For the most part, I would respectfully disagree with this statement. Very often people get an amp and expect it to sound vastly different immediately. Unfortunately, the changes are more subtle and require extended listening to appreciate the improvements in sound quality. I have quite a few IEMs (Shure, Westone, UE, Etymotic, etc.) along with a number of different amps, both portable and stationary. I cannot think of one case where the SQ did not change......usually for the better. More detail, clarity, tighter bass, smoother vocals, etc. My advice to anyone who feels that amps do not provide a richer, more satisfying listening experience is to possibly borrow an amp, listen to it with your favorite IEMs for a few days. THEN, listen without the amp. I think you'll understand what I'm talking about. If your ears tell you there is no difference, consider yourself one of the lucky ones..........you've just saved yourself a boatload of money! :-)
That there is one of the best explanations of what a good amp will do for you that I have seen on this forum ever! Well done sir, that should be a sticky somewhere at the top of the portable amp forum!
I have yet to find even the easiest to drive IEM or headphone that has not improved it's overall presentation of music by adding a good quality (note I did not say expensive!) amp.
Think of a portable amp in the same way as you would a subwoofer properly adjusted in a speaker based system. If done properly you actually won't hear the sub or barely with the main speakers disengaged but if you took it away your music would sound less appealing and you would think that a whole lot of layers are missing to the musical experience.
Likewise I always advocate as high a bit rate as possible with your music. You may not hear much difference if you think in terms of details but boy oh boy it makes a considerable difference in terms of emotional connection to the music!