I have both the E5 and E6 and I prefer the SQ of the E6. The soundstage feels wider than on the E5 and I get the feeling that instrument separation is a bit better (of course it depends on your source and headphones).
However, I think that the bass boost on the E5 is much better. It feels like bass is boosted more whereas on the E6 it feels like the overall volume is just raised. In this I agree with the review from Anything but iPod.
But with bass boost turned off there is no question - the E6 easily triumphs the E5. If I want bass to be raised, I just EQ my player instead.
I agree with starNdust that the E5 feels more solid with its aluminum casing. The plastic E6 feels more "toyish". Both amps have their strong and weak points. The E5 is by no means a bad sounding amp, so you don't need to switch it for the E6. Since they both are so cheap, you could always get the E6 later and compare them yourself.