Hi acme15. What source unit(s) and phones are you using?
I recently "upgraded" from iPod Video to iPod Classic (an upgrade of capacity only). The new iPod sounds really crappy compared to the older one and I'm guessing it's got a poor quality amplifier given that they had less space to work with in the new slimmer design. This is one example of a case where an amp will make a huge difference (the new iPod's internal amp lacks bass, seems to have a harsh peak in the 8 - 16kHz range and lacks detail and separation in the sound).
Another reason as already mentioned is that some headphones have the ability to really show off the qualities of a good amp (and the shortcomings of a bad one). I agree with Head Injury that volume is volume. Mathematically / scientifically speaking, he's 100% correct, but I've also experienced what wolfen68 is talking about with some headphones (like my Ultrasone HFI-680s) sounding much much better with amplification. I think it's more a case of the higher impedance headphones needing more grunt to create the subtle movements of the drivers that create detail and nuances in the sound.
I know that my HFI-680s come alive with amplification in comparison to my old (i.e. good sound quality) iPod. With amplification they are more detailed, have a more spacious soundstage, and reproduce the "magic" in the music which comes from harmonics and the subtlest nuances in the recording.