Chalk bag is the a very simple item in climbing equipment. You can buy it without thinking too much. There are only several things you should consider.
Size and Shape
Small size bags work best when you don't need large amounts of chalk. Like bouldering and indoor climbing. They are also good for people who have smaller hands.
Standard size bags are the most common. They work well in nearly all conditions.
Larger bags (pots) are for shared chalk. This kind of bag is usually on the floor near the climbing wall instead of mounting on the waist.
When it comes to shape, there are two main categories - cylindrical and tapered. Some people consider tapered ones more comfortable but it's up to your personal preference.
This is very important feature. Bag should have a cord to close the bag. This prevents chalk from spilling when you're not using the bag.
Makes it easy to use the chalk by keeping the bag open.
Fleece lining helps you to distribute chalk evenly on your hands and keeps chalk inside the bag.
You can directly mount the bag on your climbing harness, but when you're climbing without harness (bouldering) you can use the waist belt to keep the bag with you.
This is a small loop where you can put a brush for cleaning the holds.