After hearing headphone amps, then speaker amps, I have wondered the same thing. My answers to myself were:
1. The separates I am using (preamp and speaker amp) are MUCH more expensive than any headphone amp.
that's likely true for a lot a of people, so all the more reason to do it.
2. They are totally non-portable: 75 pound preamp, almost 200 pound amp.
in a home setup, not a big deal, i would think. some of the so called best headphone amps are big and not being moved around either, although admittedly not like yours.
3. It is not turnkey like buying a headphone integrated amp is and just plugging in the headphone jack.
more threads like this by guys like you, easier it is for us the rest of us, and secondly, more likely that someone will produce a plug in box that's has adjustable parameters, esp for those that want to experiment w multiple amps/phones
4. You can damage your headphones if you don't pay attention.
true. but a properly implemented black box and clear instructions regarding grounding would go a long way towards minimizing that risk
5. The ideal impedance matching setup for each headphone will be different, so it requires fiddling, knowing something, effort, and a bit of understanding.
knowing something, effort and understanding. you hit all my weak spots. back to the black box idea. 'speaker amps and headphones - for dummies.'
It is so much fun, though, having this level of sound, essentially infinite power to drive any headphone, and the same amp to make headphone and speaker comparisons.
so true. the headphone amp world is so tiny compared to rest of the high end audio world - why pass that up?