You're right there. You need a good source to get the best out of them. If I'd heard them for the first time out of an ipod I'd have been part of the "They're not much better than the SE535" brigade. And it's not just the bass: everything suffers. With a good DAP they sound terrific.
Some people (myself included) will amp them because of the high output impedance of their source. Just about anything will drive them to a good volume.
How would that explain the average at best sound with the PHA-2 over the 5s then? 5s DAC that good? I'm having a hard time understanding what makes the PHA-2 worth $600 based on my few hours of listening and constantly comparing with and without the PHA-2..