Memory is an issue, because if you buy the sony you will be stuck with their proprietary memory stick cards, which are too expensive when compared to compactflash or smartmedia. Also find out the maximum memory size you can put in it, 64MB is fine, but 128MB is obviously a bit better.
Also check out the interface. USB is the standard. How long is the cable? How accesable is the port on the camera? What type of driver is it, TWAIN or storage class? I would reccomend buying a separate card reader (they cost about $25 at Best Buy) because they tend to be quicker and can be usefull if your batteries for your camera are dead. Plus, if you loose your camera's software, you arn't stuck.
Does it use AA batteries, or a proprietary battery? AAs are larger, but you can just swap them out when they die and they are farily cheap ($15 for 4 NiMH) so you can buy lots of extras. Most other batteries are smaller and last a little bit longer than AAs, but when they are dead you are stuck, unless you shelled out $40+ for a spare battery.
There are a host of other things, like how long is the shutter delay, which is the time between when you press the button and when it actually takes the picture. Also find out how long it takes the flash to charge, on some digital cameras it can be over ten seconds!
Just remeber, the camera is only a tool: you could take a crappy digicam and take good pictures. You could also take Nikon's top of the line digital camera and take some really awful pictures.