I think I get better bang for my audiophile buck with headphones, and there is a simple reason for this. I don't think it has to do with sources, construction quality or anything like that, but rather the fundamental nature of speakers vs. headphones.
Speakers are much more dependent on the acoustical nature of the space they are placed in, whereas headphones are not. Also, headphones generally do not fall victim to poorly designed crossovers and multi-driver imaging problems like speakers do (See "Orb Audio," or "Totem Acoustic's" philosophy on the single driver speaker!).
Trust me, if I could afford speakers that were nice enough to trump the comparatively cheap cans I own, I'd buy em... but for now it's headphones!
Here's a novel idea: using speakers AND open backed headphones together: great sound quality and the kidney-punch bodily effects of speakers to boot!