I just did some basic comparison between the iPhone 4 and my iPod 5.5g thought I'd share some thoughts~
Used amp and lod as per my sig. Same volume set for all tests. Tested using TF10s.
1. Unamped Test, the iPhone 4 actually sounds better than the iPod, straight out of the headphone jacks.
2. The iPhone seems to differ only very slightly amped and unamped.
3. Test with amp, the iPod 5.5g definitely sounds better than the iPhone 4.
Sound quality ranking iPod+amp > iPhone+amp >= iPhone w/o amp > iPod w/o amp.
With my current setup, I'd say don't bother amping the iPhone~ hardly any difference... instrument separation is still all camped up with hardly any depth. Soundstage's fine though.
Stick to iPods at least for a more enjoyable music listening experience (within the "i" family that is...)