A lot of people find that their reciever does a pretty good job of amping their headphones (in fact most of the time the people act surprised that it drives them well). However, the thing people tend to not notice at first is that while its getting plenty of power, it is not clean. You can hear the noise in the signal. This might actually even be part of the reason it sounds good. I had an Onkyo reciever that you could hear the noise (its difficult to describe as its not really distortion, but you can tell its not a clean signal), however this actually made some of the music sound more energetic. 80s Metallica sounded great as it really added to that type of music. However other music suffered.
I think that is one of the most often overlooked aspects of dedicated headphone amps is that they offer a much cleaner signal than the headphone jack (on most products, there are exceptions and in some cases the product offers a very clean signal but not the power that is needed). I had a Meier Headfive for a while and it was very transparent, and I had not heard such a clean signal (was doing line out from an iPod using lossless). It really immersed you in the music. For some it didn't make a big difference (the aforementioned 80s Metallica might even have been a bit too polite sounding), but others it was night and day.
This has been my own personal experience. It is also the reason why a lot of people have a couple of different headphones and amps, as they can excel at different things. For instance, it'd probably be better to get the energetic sound for rock from say Grados and a quality amp that way you get the best of both worlds, a clean signal, but with energy. For other music you'd probably prefer a different sound.
Not sure if any of that makes sense, or if people agree with me, but that has been my own personal experience.