I think there's a certain duality going on with music and science. Obviously, on a physical level, there are perfectly quantifiable things happening. Everything in the physical world and which obeys the laws of physics and entropy can be described by science and mathematics. But there's also a strong emotional component involved, and that's where science, for all its virtue, falls short for many people.
There are a lot of things like this. For instance, there is no scientific support for the concept of altruism, and yet the vast majority of people swear up and down that it exists and regard discussion of the contrary as dangerous and cruel (at least in westernized cultures--I can't speak definitively for other cultures). Similarly, the concept of a soul is nebulous and widely disputed. Physically, your brain is nothing more than a dense collection of neurons (a gross oversimplification, of course, but whatever), and yet emotionally people think in terms of a "core" of some sort, that functions separately from the physical reality.
Music is no different. It's nothing more, on the surface, than your brain perceiving patterned sound waves whose frequencies fall within the human hearing range. Our ears work such that the most important sounds to us (generally the sounds that other humans or dangerous predators make) are emphasized, and our sensitivity to the frequencies at the extreme limits of our hearing is quite subdued in comparison. Rhythms are a strong component of music because they imitate patterns we perceive in the real world. I imagine things like heartbeats and the simple act of walking were inspirations (incidentally, we generally prefer rhythms that are divisible by two--maybe because we have two legs?). And so on.
All this can be discussed in greater detail than somebody like me, who just gets the general gist of it, can scarcely communicate here. But there's that other component, the emotional one, which plays such a strong role in our enjoyment of music and, I believe, our evaluation of and beliefs on sound in general. That part of us doesn't like when things are reduced to mere scientific facts. There's a sense of mystery, of magic, and of subjectivity that are lost when we do that.
People love to judge, even when sometimes they're not good at it, and science takes the ability to judge away from people. In science, there's a right answer, a correct tonal balance, and the assurance that a headphone "sound stage" can never project cues forward. People don't like this perspective, and I really don't blame them. I always feel a bit let down when I discover how something that fascinated me as a kid works, and it ends up spoiling the illusion. That, I believe, is how people feel when science gets injected into sound and music. These things are personal and emotional for people, and they cherish that. On some level, all sound reproduction--particularly with headphones--is unnatural, and our ability to enjoy it at all is a function of our brains' marvelous ability to make the best of "impossible" sonic cues. There's a lot involved with headphone listening that's pure illusion, and it's a well-known fact that people don't like having their illusions shattered.
And why should we? Ignorance is bliss. I'm not saying there's anything wrong with this. When you're enjoying your favorite jam, are you thinking about all the science behind the sound production (both in the anatomic and electronic reproduction senses), or are you just enjoying the music?
I know what I'm doing. 
