You were right about the external amplification, the 8320 and 8323 were plenty loud enough. The E6 was pretty neutral however, it really did just make it louder without coloring the sound that much or at all. What I did find it useful for was it's 2nd EQ mode that ups the bass (although I'm no sure how different/better it would be to just use "bass boost" on my iPod.)
My friend (who has the Fireye Mini & 8320) convinced me to get it by saying it made his headphones sound better but I don't really hear much of a difference besides amplification. I'm still a newb when it comes to this, would the more expensive headphone amps make a noticeable difference or do only certain headphones require an external amp? I thought alot of audiophiles always paired headphones with an external headphone amp?