I would still get the he-500 and buy a stereo 1/4 to 1/8 adapter at radio shack. I'd rather listen to an underpowered he-500 than an adequate he-400. E12 will give enough power for the he-500, and even the x3 will sound good with the he-500 as is because it is farily powerful by itself.
The he-500 has been given an unfair bias that it takes a ton of power to drive. I can get moderate listening levels straight out of my note 2, it just lacks a bit of soundstage and dynamic range and the bass is a bit more thin, but it is by no means unlistenable. The x3 will give you more than enough volume and will give decent punch and dynamics.
Upgrading the e12 should help, but i havent heard the x3 and e12 together with the he500. BUT my experience with the x3 with e12 on other phones and iem's is that they sound almost identical. upgrading from the e12 is really only worth it if you want a different sound sig or if you really take a leap into something like the hifi m8 or theorum.
I compared the e12 and m8 with the he500 when i had them for about 2 weeks, and really the only difference between the m8 and the e12 was a bit bigger soundstage, and smoother more refined highs, not including other options like balanced out, included dac, treble shaping.. but power was about the same.
That being said, the e12 and x3 were about the same so that should tell you that it takes a bit of an upgrade to get a negligable difference. These were all my portable options though. If you get the he-500, which i would recommend, I would also consider alpha pads. They improve both sound and comfort IMO.
Also if not being used portably or as a transportable, i would consider using your x3 as a dac and getting an emotiva mini-x or AVR and using speaker taps. I use a yamaha rx-v371 and rx-v373 and they sounded incredible with my he-500 and he-400. Beat my centrance m8 in every way. you can get either option for $150.
But bottom line- he500 isn't THAT much harder to drive than the he400 but has a more refined sound, less grain, a more full sound due to a more lush and full midrange, less tizz in the highs, and a bit more full bass and a bit lower sub bass extension.