I think that it's all a question of synergy, i'd rather have mid-fi sound system with components that work well together, than a high-end rig with poor synergy.
That being said, i think that since they're the ones responsable for transforming an electrical signal into actual sound,,the headphones have the most influence on the final sound signature.
I'm pretty sure that if i swap a $1000 Cambridge cd player for a $1000 Arcam cd player, the difference would probably be less obvious, than a swap between a pair of $1000 Audeze LCD-2 and a $1000 Grado GS-1000.
I believe that the headphones (or speakers), represent the vocal cords of a sound system, of course, that doesn't mean that the source and the amp, don't play an important role, i just think that they have a less obvious influence on the final result.
Now, David, while i agree that knowing what source and amp people are using can be a helpful to form a general opinion, they're are still other factors to consider, for example, the type of medium used, cd, vinyl, computer , tape, etc,,, then they're is quality of the recording itself, also, the age of the listener, because, as you probably already know, when we get older, we're usually less sensitive to higher frequencys. and last, but not least, personal préférences, some like a little more bass, some like a little more treble...
Finally, i don't think that a good sounding source and amp can turn a bad sounding headphones into good sounding one, but i do believe that in some cases, a good front end, might improve things a little. Even if i use my Yamaha DVD-S2700 and my Musical Fidelity X-CAN V2 to drive my Grado SR-80i, they don't sound at all like my PS-1000.