Well, that's just an opinion, too. My experience is the opposite. The headphone may not drastically change its characteristic, but enough to (completely!) change my mind about it and its synergy with a specific system. Also, the characteristics of a source component can be just as important when it comes to synergy, that's where you just don't seem as sensitive as myself or others. Therefore my caveat.
Let me give you a perfect example: The Amphora was one of my favorite amps with the HD800 (and Ken will be sending me one so I can do a review on it). It had an iPod in it and the resulting sound was better than other amps I heard hooked up to multi-thousand dollar sources. Certainly the iPod is not a great source, but with this amp it sounded fantastic.