I suspect it partly has to do with how colored consumer audio gear sounds, such as having overly emphasized bass and exaggerated warmth. When an entire market segment gets used to that kind of sonic signature, they will think that's "the standard," and when they hear actual netural/accurate audio reproduction, it sounds alien to them.
One very common phenomenon among both audiophiles and pro audio guys is this:
When they transition from being inexperienced to experienced, they pick up critical listening skills along the way, and part of the skill has a lot to do with "unlearning" the colored sound they got used to as consumers. At first, they can't get used to the neutral/accurate sound reproduction because it sounds so different; the lack of exaggerated warmth, the lack of overly emphasized bass, the clarity of transients and details--it all makes them think the sound is cold and analytical. But that is only in comparison to an exaggerated warm sound. The same can be true if the person was used to consumer audio gear that sounds excessively bright to "fake" detail in the mid and treble frequencies, such as some of the Sony headphones out there--they'll listen to neutral/accurate and think it sounds dull and dark.
Eventually, they learn that what they had become used to in the past is the colored sound, and once they spend some time with a neutral/accurate sound, they'll be able to discern all the problems with colored sonic signatures. They'll be able to hear just how bloated the bass is in the old sound, obscuring important details in the low frequencies, masking details in the low-mids, and just sound like of muddy and vague in general. They'll also hear that intentionally contrived "warmth" is often just rolled off treble, and that kind of artificial contrivance is simply subjective, and on material that really need to have the "air" properly presented, the colored sounding system will be unable to do it. If the colored sound they were used to was overly bright, they'll learn that the brightness they thought was detail was actually very artificial sounding and in fact, quite unpleasant, once their ears got used to a neutral/accurate sound.
They'll also learn that if a recording sounds fatiguing or too bright on a neutral/accurate system, then it's the recording that's the problem, not the system. If you try to remedy the recording's problem by using a colored sounding system, then everything esle you listen to on that system will be colored! That is like purposely buying a TV that is too dark and cannot be adjusted to neutral/accurate, simply because a small percentage of movies you watch seems a bit bright to you. It's ass-backwards and illogical.
So instead of buying colored sounding audio gear, what you should do is to simply use an EQ unit and create presets that contain the kind of warmth you want to remedy those overly bright recordings, and then on the majority of material that isn't too bright, you simply turn the EQ off.
The truth is, material that sounds too bright are the small minority, and it's counter-intuitive to cater your system to a small minority of inferior recordings, when the majority sounds just fine.
