Things To Avoid
Digital Zoom. Digital zoom, to me, is worthless. It is nothing other than a trick that makes you think you're zooming in on a subject when all that's really happening is that the LCD is making the pixels larger. You're not getting any more detail. Many cameras on the market now have at least 3X optical zoom (that is, zoom that's controlled by the lens optics rather than the camera's electronics), and many of those cameras will comfortably fit in your pocket. Some of the better cameras even have 5X optical zoom, and one model from Nikon gives an amazing 10X zoom - again, in a size that will fit in your pocket. If you see a camera that says it has 12X zoom, read the specs closely: it's probably combining 3x optical zoom with 4X digital zoom (3 x 4 = 12). Don't fall for it.
Poor Body Construction. This is kind of a subjective thing, but get a camera that feels good to hold. The buttons shouldn't feel cheap and any dials or knobs shouldn't feel like they're going to break off at any time. Poor body construction can also show up in how tricky it is to reach the controls. Go to a store and hold the camera. Try taking a picture. Then try changing something - the photo mode, the focus, anything. Note the materials in the body: is it an alloy or is it all plastic? If it's all plastic, the camera may not age as well.
Slow or "Muddy" LCD. You have to try a camera in a store to check on this. Some camera's LCD's are better than others. Turn on the camera and move it fairly quickly from side to side and watch the LCD monitor. Is it keeping up? Or is it dragging a little bit. Some people may not be bothered much by this, but for folks that like seeing exactly what they want before they snap it, it matters. Also, how does the image in the LCD look? Is it pretty crisp? Or is it "muddy" with poor detail. While the LCD may be no indicator of how the final photo quality is, I feel better when I am able to check my results on-the-spot with a reliable LCD.
Proprietary Batteries. Avoid cameras that don't use AA batteries. While proprietary batteries can deliver hundreds or (in some cases) over 1,000 exposures on one charge, the big problem I have with them is, if you forget to charge that special-type battery, you may have to wait 2 hours of charging before you can use it again. With regular AA batteries, you can either run to a drug store and pick up some alkaline cells, or use NiMH (nickel metal hydride) rechargeable cells, some of which can recharge in 15 minutes. (By the way, a set of 4 AA NiMH batteries now can be had for under $10 and used hundreds of times!)
Cheap Brands. You may be tempted to buy a digital camera with a name you've never heard of, and if you're looking for a "disposable" camera, that's fine. But if you want a camera that will last longer, don't fall for the lower price tags. Often these cheap brands have no good features, no zoom, and no expandability (you can't add memory cards to them). Worst of all, some only take photos at 640 x 480 (or VGA) resolution - suitable for the web and e-mail, but not much else - and the picture quality will likely disappoint you. Stick with names like Canon, Sony, Casio, Olympus, Fuji, and if you're willing to spend a little more, Nikon.
No White Balance or ISO Control. White balance control won't matter much to you if you plan on shooting most of your photos outdoors, or if you plan on taking flash pictures most of the time. But if you like shooting existing-light photos indoors, white balance control is a must. A good camera will have settings for one or more of: sunlight, tungsten (or incandescent), and fluorescent lighting. ISO control goes hand in hand with this: it simulates different film sensitivities of films you would buy at a drug store (ISO 100, ISO 200, etc.). The higher the ISO number, the lower the light level you can shoot in, without flash. If you plan on taking at least a third of your photos indoors without flash, make sure your camera can at least touch ISO 400, or plan on using flash in those situations.