Yea, its amp stage just doesn't pack enough power for many full-sized headphones. It will do fine with lower impedance phones in regards to volume/gain, but the high output impedance will alter your sound. This was the #1 flaw of the device, and is why many people won't consider it without getting it modded by RWA, MS, etc. It just really isn't a good AIO unit for most headphones/earphones.
All the more reason to go with the HifiMan HM-901 for various amp cards alone. $200-300 per amp card is much cheaper than buying a separate portable amp of similar quality, not to mention the space savings and additional cost of cables.