FiiO states that their portable amps run on USB power and almost entirely cuts off its battery when plugged in. So yeah, it makes sense that it remains hot. Same thing for every other amp that uses USB or even an external power supply.
If your E12 is turned off and it isn't charging but plugged in, if it's still hot then there's a cause for concern. Is this a problem you are facing?
I solved the heating problem! My E12 also charges fully, and the charge light shows it! My E12 is perfect!
I used a 2.4A USB wall charger to charge it to full, with the charge light blinking rapidly. At first I was scared that my E12 charge light is defective and that it can't detect if the device is fully charged. But then it finished charging. Now I'm using it plugged in, but it still shows it's fully charged so it's all perfect!
Weird things:
When I was charging using the 2.4A charger with the E12 turned off, the E12 was cool all throughout! And when I turned it on again, still plugged to the charger, the charging light stays full, and after using it for an hour it is still cool!
Previously, I was using a 500mA USB wall charger. With the E12 battery drained, I tried to use it while charging. The charge light kept blinking for 24 hours with the E12 turned on. Most importantly, the E12 was running hot.
If you want to use your E12 while charging, it is best to charge it fully first so it doesn't get hot.
EDIT: Just after typing this, the charge light started to blink again. It's blinking slowly, so it means it's charging slowly even though it's plugged to a 2.4A charger (5V, 12 watts). I'm pretty sure the E12 doesn't eat 12 watts of power when driving 16-ohm headphones. I'll check again if it gets hot again. So far it's still cool.