I must say, an amp doesn't make a massive difference for my HD595s, but I can notice the difference if I plug it into my home theatre receiver, it gets a lot clearer and the bass becomes more prominent.
But for a majority of IEMs, an amp gives you maybe the final 5-10% boost in audio quality, unless it's something that apparently works quite well with an amp in which case it might be as high as 25%. But my understanding for some high impedance full-sized cans, is that it can be practically the difference between sounding awful and sounding amazing. But at a lower price point that would be extremely rare, and most people would be fine without an amp - it would be silly for a headphone manufacturer to design a $50 headphone to work well with a $200 amp rather than a simple iPod headphone out. If a sub-$150 or so headphone works significantly better with a good amp then that's just bad design on the engineer's part.
Of course, a good amp can make more differences than just that if the source is awful to begin with, like many (most) onboard sound cards. It's not really just the extra 5% anymore, it's more about getting clean sound that doesn't sound like it's coming from an awfully scratched vinyl LP. My experience with Realtek onboards, at any rate, is that they muddy up the sound significantly and add a weird hissing/buzzing/ringing noise in one or both channels at random intervals...even on higher end motherboards.
But in the end, whenever people talk about how much better an amp makes most headphones sound, it's generally exaggerated. To a true audiophile even a 5% difference can be a lot, but to most they wouldn't be able to tell the difference unless they do quick A/B if at all. But I do feel a good amp can make a big difference if you're coming from a truly bad amp.
I don't have those or foresee that happening in the near future but believe me, if it were at all possible to directly hook them up to my peasant's toy called an iPod... I would. 
