Well I personally don't dismiss subjective tests for the simple reason that we do not know that we are objectively measuring everything that is important to human hearing. Like say the way in which a signal is distorted. Maybe one amp distorts a signal differently than another, despite both having the same measured level of distortion. So it is possible we hear things that just don't show up on the tests we conduct.
However, I have studied a lot of Psychology and I do know the power of suggestion and the mind's ability to fool itself. My favourite experiment along these lines, which I can't find a link to despite my best efforts, was one allegedly conducte don speaker wire, really on the power of suggestion and peer pressure. They had a room with expert listeners in it and put a nice audio setup. They then introduce the three wire candidates, two high end cables, and normal braided copper wire, for reference. They would then play a peice of music on each, with assistants switching the wire in between each replay.
Well opinons disagreed about which of the two high end cables was best, but the experts were united that both sounded far better than the copper wire. The catch? The wires were never chainged, it was normal copper wire the whole time. All percieved changes were just in the mind of the listener.
Now this isn't proof that different wire doesn't matter (that wasn't tested), but it IS proof (or rather one study in a line of proof) of the power of the mind.
What I'd like to see more of is stuff you deal with in psychology all the time: proper blind experiements. However I also want to see them done properly. The problem is, you can't really ask someone reliably to tell if they like something in a quick A/B test. They need time to listen on a wide variety of material, espically if the change is supposed to be subtile.
It's something I'd like to try with something like a PPA sometime, and maybe will. Build two amps with just razor fine tolerances on everything, and have different opamps in them. Then, put them both in identicle boxes, rigged to be tamper resistant. But a label A on one box, and B on the other, randomize which box is labeled which. Give them to test subjects for a good period of time, a couple days at least. Let them go back and forth all they like. When their time is up, take back the boxes and ask which was better, or no difference and WHY they liked it better. See what kind of result you get.
If the subjects have no idea what the variable is, and there is no way for them to find out, then you have a pretty good chance of getting a real, unbiased opinon. Then you can see if there's any trends in what people like and, more importantly, why they like it. Just because the like results are 50/50 doesn't mean no difference was heard, it might just be different preferences. However if there is no corrilation between why somethign was liked, probably people are just making things up (since most will figure they are supposed to hear a difference, maybe as a control give some subjecst two idnenticle amps). However if you see real correlation between a given opamp and a given reason for liking it, you probably have a real difference, and can then maybe try and measure it.
Thing is, empirical tests of subjective data are TOUGH to properly design and execute, and there seems to be little intrest in teh audio industry to find out what they might reveal. If they show it's all in your head, all the audiophile firms are screwed. If they show there is osmething going on, all the pro firms are screwed. Either way you have something many people don't want to hear.
Personally, I hope to maybe do graduate research on the topic someday, but I'm not sure that will be feasable. It would take a fair bit of money to organise something like that, and I imagine grants for that sort of thing are hard to come by.
|