Originally Posted by uchihaitachi
So Jaddie, would you say the entire mainstream range of tests an engineer uses to check audio gear pretty much covers every nook and cranny? Ie to an extent that the 'tests are not enough when actual music is not used' party really don't have any validity to their claims?
Every nook and cranny? Hard to say. Most of the tests we now use are 30 years old or older, and outside of using computers and software, we aren't really doing much testing that's different today. There've been no significant new tests added since TIM surfaced, became popular to test for, and then later was summarily dismissed. Remember, FFT analysis was around in the late 1970s, and I was driving an old Crown (Techron) TEF analyzer in the 1980s.
But ask yourself this: if we test with music as the test signal, what is the analysis mechanism? All we can really do with software (or hardware for that matter) is analyze the total spectrum over time, and look for a difference from the original, which may or may not tell the whole story. But that's not what you're getting at, is it?
The claim you refer to is usually based on listening tests vs measurements, and that's a slippery slope. Listening tests can be quite valid and revealing, but not unless expectation bias is eliminated from the test using some form of double-blind test methodology. When that is done, the ability to detect differences diminishes dramatically. The "listening is all that matters" group usually blanches at the double-blind test methodology as being too stressful, or forcing a quick decision, and thus clouding otherwise easily discernible differences. But it's also undeniable that expectation bias has a powerful and measurable effect on judgement, and for sensory based judgement to be accurate, bias must be removed.
The problem here, really, is one of belief systems. Even though my own lean strongly to scientific method, I also don't enjoy bursting peoples bubbles. If someone hears something wonderful, even if I could prove it's imagined, why would I? There's really no point. My reality is based on the tangible, provable, and repeatable. Someone else's reality may be based on legend, myth, pseudo-science, visual suggestion, or a whole palette of other influences. If we both have the enjoyment of good sound as our goal, I'm no longer certain that it matters much which is right. I've pursued truth in audio for over 4 decades, and yet I still see people becoming ecstatic listening to high-end systems with no measurable (or provable) advantage over something far more common. I follow the one principle I absorbed from my high-school economics teacher: Economics is maximizing happiness! What that means for me is as valid as what that means for someone else. For some, buying is highly enjoyable. Buying expensive and exotic things is a very powerful tweak of the pleasure center. It's as valid as maximizing sound quality for the minimum expense, or any viewpoint in-between.
Hows that for dodging the bait?