I think one quote I've heard sums it up best:
"Some audiophiles listen to music; other audiophiles listen to their equipment."
Some people seem able to "just hear the music and not the equipment" - while others can never seem to get past the gear.
And it's quite possible that some people really do just plain enjoy playing with the equipment more than they care about listening to the music.
(Do you have a collection of tracks that show off your equipment, or that make it easy to detect flaws in gear, even though you don't actually like the songs themselves? I do.)
There's also the issue of attention.
(Have you ever had a small crack in your windshield that, even though it's over in the corner, and not in your field of vision, but you just can't un-notice it once you know it's there?)
I personally think there is a happy middle ground.
Spending all your time, and money, trying to squeeze the last bit of performance out of your gear is usually a losing battle.
You'll never achieve perfection... and you'll never find a perfect recording anyway.
HOWEVER, just like when you have a little crack on your wibndshield, each of us have specific flaws that annoy us...
And, when you have one of those, you're never going to be able to concentrate fully on the music until you get rid of it.
(That's why I really dislike vinyl. I was always that way with surface ticks and pops on records. If I hear a single tick it "breaks the spell" and I can't focus on the music any more.)
However, I think an awful lot of people go way too far the other way.
It may be true that the world's greatest photographer can take a great picture - even with a crappy camera.
However, a really low end camera is going to limit their ability, and limit their possibilities.
For example, you can't make specific artistic adjustments to depth-of-field in your photos if your camera doesn't have a manual aperature control.
And, yes, every time I see some great photo someone took with their phone, I tend to think: "Gee, that's nice, but it would have been even better if they'd had a better camera".
Likewise, I don't want to enjoy the music I listen to
in spite of terrible flaws in my system, I would prefer to enjoy it without the distraction of obvious sonic flaws.
It's possible that the world's greatest violinist can play really well even on a discount store violin, but I'll bet he plays even better when he plays on a decent violin.
And, sorry, I'll bet even the world's greatest trumpet player sounds like crap on a fifty-nine cent plastic kazoo.
The real trick is to find the line between "meaningful improvement" and "tilting at windomills".
And, yes, that line is going to be in very different places for each of us.
And, yes, you have to acknowledge that other people may have very different priorities than yours.
For me, if I want to hear details I never heard before or alter my perception of music I'll listen closely to a piece of music I really love three or four more times. I'll read the lyrics if it has lyrics. I'll try to organize it in my mind. What our minds do with music is pretty close to magic. The lyrics will take on a different twist. I'll notice the base line, or something in the drums, or what the clarinets and flutes are doing, or that great trumpet solo, and so on. With modern technology sometimes I'll say wait a minute, what was that, and go back several seconds three or four times, and then from then on when I hear that music I have a better idea what's going on.
Maximizing fidelity is very cool but rapidly diminishing returns on that additional dollar spent hit early and hard for me. I admire and respect people who do better at the hobby than I do and if I can use their experience to identify that an obvious improvement is there for the taking for me I do it in a heartbeat. I want to hear everything on the recording (within the limits of my hearing) in a pretty good and even balance and to have control over the end sound (e.g., EQ) for my preferences.
It is a luxury of life to have a nice hifi and it can be a true wonder and fulfilling hobby and even intellectually quite interesting and demanding and engaging if done well and smartly, but in my view it's not necessary to enjoy music to the utmost. Maybe that's the cruelest audiophile myth of all--that to really appreciate the music you need the nth degree in music reproduction so you can hear the music in all its glory. And the twist of the knife is manufacturers and salesmen misleading people so that they spend their money on the wrong things and in the wrong proportions and in preposterous amounts to maximize fidelity. Then they have veered way off course and wasted time and money and have been lied to and misled in pursuit of what is for me the pot of gold at the end of the rainbow, which is to enjoy the music to the utmost.