Help with a philosophical problem with blind testing cables

[odigg]

Quote:

Originally Posted by moonboy403 /img/forum/go_quote.gif I just read something interesting regarding audio blind testing and it might be of interest to some here: "The right brain/left brain theory applies here. You use one side of your brain for objective logical kinds of processing, and the other for subjective and emotional. When you are enjoying music, the subjective half is in play, and its process does not relate to the specs very well. The same phenomenon explains why A/B comparisons, no matter how well staged, do not resolve what audiophiles claim to hear. When you are sitting there trying to hear the differences between products, the objective side of your brain begins working and your subjective responses get locked out by the pressure to make an objective decision."

I agree with this to some degree, that's why I'd like to see blind tests where people listen a) over time and b) in a relaxed, more typical to usual listening, setting.

However, there are limits to how much protocols can be changed. Including sighted listening is flawed for a lot of reasons. Every protocol and methodological framework has flaws. Nothing is perfect. If you simply play on the fact that nothing is perfect, you're simply attacking the results of blind tests because they don't suit your opinion. Nobody can ever demonstrate a test is perfect.

It's basically then just a form of nihilism. Science doesn't move forward from that and we have to remember it was the scientific method that brought us electronics in the first place. We can't just continuously reject it because we don't like what it tells us no matter how much test protocols try to address our concerns.

I think it goes back to what royalcrown said in another thread. If you find blind testing flawed, then should reject positive results as well. You cannot reject blind testing when it has a negative results and applaud a positive result. That's just cherry picking what you want.

So there are limits to what you've said.

 

[nick_charles]

Quote:

Originally Posted by moonboy403 /img/forum/go_quote.gif I just read something interesting regarding audio blind testing and it might be of interest to some here: "The right brain/left brain theory applies here. You use one side of your brain for objective logical kinds of processing, and the other for subjective and emotional. When you are enjoying music, the subjective half is in play, and its process does not relate to the specs very well. The same phenomenon explains why A/B comparisons, no matter how well staged, do not resolve what audiophiles claim to hear. When you are sitting there trying to hear the differences between products, the objective side of your brain begins working and your subjective responses get locked out by the pressure to make an objective decision." Perhaps that might bridge the gap between the two camps? Any thought and comment?

I have done loads of ABX DBTs and have many many positive results , when of course a real difference even if relatively small exists. I do not find DBTs stressful I like doing them and I get much better results than from sighted tests.

 

[SmellyGas]

Quote:

Originally Posted by PhilS /img/forum/go_quote.gif Fair enough. You're entitled to your opinion. And my opinion is that your tone is rude and condescending, and this post and a few others convey an arrogance that I find distasteful. I won't discuss this with you further.

No, ARROGANCE is believing that you know better than hundreds of engineers and scientists who spent thousands of hours designing and performing controlled experiments, then submitting their work for peer-review and publication in major journals.

It's like a person who smokes cigarettes telling all the physicians, researchers, and epidimiologists, who have spent years publishing a body of work that proves cigarettes are harmful, that they don't know what they're doing, their methodologies are invalid, and their studies don't prove anything!

I realize I'm a minority here, but I find opinions like THAT extremely distasteful.

 

[PhilS]

Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif No, ARROGANCE is believing that you know better than hundreds of engineers and scientists who spent thousands of hours designing and performing controlled experiments, then submitting their work for peer-review and publication in major journals. It's like a person who smokes cigarettes telling all the physicians, researchers, and epidimiologists, who have spent years publishing a body of work that proves cigarettes are harmful, that they don't know what they're doing, their methodologies are invalid, and their studies don't prove anything! I realize I'm a minority here, but I find opinions like THAT extremely distasteful.

In my six years at Head-Fi, I've only put one person on my Ignore list in that entire time. You're the second.

 

[powertoold]

Quote:

Originally Posted by moonboy403 /img/forum/go_quote.gif I just read something interesting regarding audio blind testing and it might be of interest to some here: "The right brain/left brain theory applies here. You use one side of your brain for objective logical kinds of processing, and the other for subjective and emotional. When you are enjoying music, the subjective half is in play, and its process does not relate to the specs very well. The same phenomenon explains why A/B comparisons, no matter how well staged, do not resolve what audiophiles claim to hear. When you are sitting there trying to hear the differences between products, the objective side of your brain begins working and your subjective responses get locked out by the pressure to make an objective decision." Perhaps that might bridge the gap between the two camps? Any thought and comment?

That's great and all, but that would also undermine comparisons and reviews that are based on "objective logical kinds of processing." For reviews and comparisons, people claim to be able to hear differences, yet when they are tested, they don't hear them anymore. This is ridiculous. If people want to play this brain game, then it would invalidate all objective evaluations of audio.

I do understand the idea that you would be able to hear "more" if you were enjoying the music, but when people do reviews, they are both trying to enjoy the music and make objective comparisons as well.

 

[SmellyGas]

Quote:

Originally Posted by odigg /img/forum/go_quote.gif You're badly mistaken if you think everybody here who sees flaws in current ABX protocols is a used car salesman. To point to the best example I know, wavoman has a PhD in statistics and is an established researcher in his field. He is trying to modify the typical ABX protocol.

No, actually I think that most people here are highly educated, and some are even involved in a similar line of work like wavoman, but judging from some of the other criticisms people have for ABX testing, a lot of people simply are not familiar with experimental design/interpretation.

Quote:

However, if people (scientists or not) have sound objections to current ABX protocols while maintaining blinded testing, they should try to modify the protocols then come back here and report on their methodology and test results. Then we can judge for ourselves the validity of the experiment.

I completely agree with you. I think I have written along similar lines elsewhere in this forum. The problem is that people who do not engage in the scientific method will not use this line of reasoning. The common response is something like "oh well, they didn't give listeners enough time to listen to each cable, so this whole experiment is invalid." Whereas, a peer-reviewer might respond with "to satisfy objections that listening tests were too short, you should add a study arm where listeners can choose to listen longer if they wish, and then do subgroup analysis."

Quote:

And yes, I'm a scientist. I read buckloads academic articles and get cranky about methodologies and research theory, read peer reviewed articles, and all that. I think it's bad science to say "There are 20 papers and so all further research is stupid."

Woh, I never implied that further research would be stupid. The complete opposite is true. However, at the same time, stupid is not drawing appropriate conclusions from an existing body of work. If 20 blinded RCT's were published that failed to show any benefit of Heart Drug A on the heart, yet costs $1000 per treatment, it would be inappropriate for a physician to continue to prescribe Heart Drug A with the reasoning "we need more studies, I'm not quite convinced yet."

Quote:

So, for people trying to come up with new ABX protocols and who are willing to submit it to to the scientific process, I applaud you.

I think it's a valiant effort. Maybe they could get grant funding. Perhaps contact Monster Cable?