Unless its DIY and you know what you are doing 
Well, there's that, too.... 
But Just because you build something that you call a "headphone amp" and put only headphone outputs on it and optimize it for HE-6, doesn't mean that the power supplies, transformer, out output circuits aren't going to end up being the same thing you'd use for a low powered speaker amp. I mean I could rip out the binding posts and install a 4-pin XLR on an F1 and call it a "dedicated headphone amp", but it doesn't mean it's not a speaker amp at heart. 
The biggest difference between an HP amp and a speaker amp is really the ability to drive low impedance with ease,and to deliver tremendous amounts of continuous current with massive peak swings. Most of that difference comes squarely down to the power supplies and transformer. So any "headphone" amp could be built with the same capacity power handling as any speaker amp. It's just that for any headphones other than K1k, HE-6, HE-5LE, and HE-4, such huge weight and size would be pointless because it would not be utilized.
Ultimately the question is "what is a headphone amp and how is it different from a speaker amp?" Before the headamp craze (thanks, Tyll! 
) a "headphone amp" was just a speaker amp with some resistors connected to a TRS jack. There was no difference, one just added resistance to the amp outputs.
Then the headphone amp craze started with small compact amps for headphones that didn't weigh 20lbs and measure 19x17x5". Since then we now have this split of "headphone amps versus speaker amps." But what's ultimately the difference? The headphone amp doesn't have the high current capability of the speaker amp, meaning it doesn't have massive power supplies and transformers, and yes smaller caps and the like throughout. And yes, it's more optimized for high impedance loads.
But basically the "beauty" of the headphone amp is that it can be made small, light, and relatively cool running, because headphones don't need the kind of current and swings speakers need, nor the low impedance that facilitates that. And, in theory the money saved on materials that will go unused can be put into improving the performance/materials that will be used for the application, though that theory rarely holds true, as headphone amps are a small niche, while speaker amps can run fairly mass-market, so the scale of economies tilts heavily in favor of speaker amps (until you get to really high end boutique stuff where the cost begins favoring headphone amps again.)
But then enters HE-6, HE-5LE, HE-4, and K1k. Not as low impedance as speakers but darn close considering the planars are linear, while dynamic cone speakers are rarely close to linear, so the average of 50ohm is probably not far off at all, and at that impedance, similar current and swing requirements to speakers. Meaning the idea of "downsizing" the amp has less meaning because more of those larger power handling components are utilized as they are for speakers.
So what we're left with is a lot of headphone amps that simply don't have the muscle in terms of sustained current delivery and peak handling left over, a handful of headphone amps that do have 90% of the needed muscle and excellent performance, a handful of speaker amps that lack in performace but deliver 100% of the muscle, a half dozen or less headphone amps that deliver both, and a large variety of speaker amps that deliver both.
You can DIY whatever you want, but if it delivers that kind of current and reserve power from a large enough power supply, is it really a headphone amp at all? if you stick binding posts on it can you power some passive monitors? Yes.
Edited by IEMCrazy - 11/14/12 at 8:36am
So far I haven't even plugged HE-6 into it...all my Mjolnir time has been spent with HD800 thus far (and I've still managed to spend most of my head-time with HE-6 on the normal setup.) 
