Portable amps don't have to have a built in torroid as it runs off of batteries. A properly designed portable amp should be able to have a clean sound, but there will always be a problem of power vs battery life vs size. For higher power and the same battery lifetime, more batteries will need to be used making the size larger. If power is reduced, then it will not be capable of driving certain headphones. Overall portability is usually prioritized over everything else causing a decrease in performance, but people have managed to make better amps recently (I recall seeing a portable buffalo32 dac). With better battery technology we should be able to get better portable amps.
I had a Headroom portable Micro and a Gilmore lite at the same time, you have more room for buffer caps, power filter caps, things like toroidal transformers, input and output buffering, and all the other things that make stuff sound clean...
