Read about 28 pages of this thread before my attention just wandered too far off to stay on topic.
The biggest thing I've found in determining differences in Hi-Fi gear is ear training. Musicians or those who are in constant contact (recording engineer, interested sibling, etc) with a musician have a huge advantage. The closer the person is to the core of the music the easier it will be for them to pick out differences in Hi-Fi gear. There were some mentions of this earlier, but I just wanted to further emphasize it. As a classically trained violist for approximately 12 years before even touching my first Hi-Fi gear I could tell differences right away, and since that first piece of gear my ability to do so has grown.
I work in a violin shop and when a family comes in to shop for a new instrument for the child, typically the player will be able to tell the differences readily, though they are usually very timid about playing in 'public'. This following bit is more stereotypical, but I find it more often truer than not: The mother/sibling will usually be able to tell a majority of the differences in instruments, while the father (unless a musician, himself) is left with the price tag as the only thing he can tell one another with. I've worked at this job for 10 years and I can't count the number of times the above is spot on.
Ear training is something that can be learned at any age, but the adage is always true: It's always better to learn new things at a younger age. When I listen to classical music I can pick out and follow multiple melodic and harmonic lines at a time, while my father says 'that sounds nice'.
Last Saturday I took my main rig up to work for a friend/co-worker to hear. His only experience with audio has been his ipod touch and a pair of lower end Bose. I had previously told him that I owned some nice gear and that I enjoyed it. I didn't tell him what to expect, I simply set it up and cued up some Bach Concertos that he is intimately familiar with (he is a violinist). He listened for a good hour or two on a sampling of discs I had brought. When he was done I asked for his opinion and he said and I quote "It sounds so big, like I'm right there with them." "It's amazing." "My Bose headphones are OK, but that is wow." He's not in a financial position to purchase anything like I have nor does he really feel the need to. He is happy with what he has, but he can also appreciate the difference. I completely respect his desire to use what he uses, but I felt it my obligation since he was interested that I show him where Hi-Fi can take him. He knows what is out there and chose to stay where he's at for now, which is the best decision.
Last Christmas I played an album on my gear for my sister who lives 12 hours away. She turned me on to the album a couple years prior, but when she heard a certain track she took the headphones off and said "Those are coins! I thought it was just a tambourine." She is far from a musician and is enough older than me to not experience my musical upbringing, but even the music listener with no real training can hear differences.
I make no argument that I do not seek a neutral sound free from color. In fact I state that I do in fact search for gear and the synergy there-in to make the music I listen fit the way I've heard instruments in real life. Instruments/voices/effects I haven't heard myself are subject to follow the trend placed by the sounds I have heard. I found the signature in gear I recognize as my ideal reproduction of real life. I do not impose this signature as the best for anyone but myself, but I will earnestly encourage trial of the gear I use complete with as fair an analysis as I can give.
I started with the same HD580s as the OP, bought a solid state amplifier that was well respected. I found the sound seemed less congested, but really had not changed at all. I thought head-fi was a bunch of idiots, closed my browser and only visited a few times over the next 4 years. I recently (2 yrs ago) came back armed with my HD580, but I had sold the SS amp, and bought a basic $200 tube amp thinking I'd give the other form of amp a fair try. I found the tube amp changed the sound in a way that felt more natural (not neutral) and infinitely more engaging. I had finally found what these 'fools' had been talking about. The more I listened I became convinced there was more to be had and multiplied my amp cost 3 1/2 times figuring I'd shoot the moon and give it a real go. I was rewarded immensely with the most organic natural sound I'd ever heard from any music system be it speakers or headphones. The raw power of this new amp in combination with it being balanced giving better separation presented me with a wide soundstage and the feeling of being in the music, something I'd not felt except when playing in a live orchestra.
My journey is proof of both sides of the story. Hi-fi truly does exist, but finding your own path is up to you. If it is important, you will persist and make your way to what you find is best, regardless of price. If it isn't important, find something else to obsess over. Life is dull without passion.