What are the arguments against double blind tests (incl. ABX)? - Page 8

If they do that, they're not scientists. This would be cherry picking or confirmation bias. However, what is the evidence that ABX doesn't work? All that is necessary to disprove the theory that it does work, is one instance of reliable scientific testing that reveals a difference not found in ABX can be reliably detected through a different method.

I've already given my examples, relating to countless tasting DBTs earlier in this thread. And in previous similar threads, I've even posted links to some specific results.

I class myself as an observer, not a scientist. Observers make their observations and scientists explain the results. Yes that's a cop out in a way, but here's the thing: most of the people on my side of the debate aren't that bothered about the science bit. Yes it's kinda interesting, but not so interesting as to spend the considerable effort to get truly scientific evidence. That's the part we expect scientists to play because that's what floats their boat and also they're better skilled at doing it. If that's not a good enough answer for the Sound Science forum, then I can understand that, but shame, because the debate won't get progressed very far. There are some on my side who can also explain themselves better scientifically, and who are doing a better job of it than me in this thread - but they are very much in the minority.  

I should also point out that there is nothing in DB ABX methodology that inherently restricts us to short term tests. There is nothing stopping someone from listening to a whole album on A, again on B, and then again on X. Or even spending a week with A, a week with B and then a week with X. The ABX method itself is not at fault here if we've decided that short term memory is no use, just the specific use of it in these cases.

Agreed. My issue is not with DBT or ABX in principle. It's with the use of structured, formal, fast switching tests for comparing certain kinds of sound (and taste) differences. As already explained in my earlier posts - they weren't that long ago. I just don't seem to be getting my points across here.

Finally, I have yet to see an instance of a subjectivist, when presented with a positive result from an ABX test (eg speakers perhaps), rejecting the findings due to inherent unreliability in the tests.

I've already said that speakers and headphones are easily differentiated in ABX tests, primarily because they have a non-flat frequency response and relatively high distortions, which are the type of sound differences that are  easily identified in fast switching tests. 

Can you evidence scientists shooting down arguments against blind testing in flames and only remembering the evidence that makes then right? What is the overwhelming observational evidence that there are problems with blind testing and what applications are you referring to?

Yes, the whole point of my earlier "prediction" is that it's happening right now. Actually, this particular thread is pretty civilized compared to similar ones in the past.  

You have made some very strong remarks against science which I think you need to evidence them.

I haven't said anything at all against science. Only that some scientists don't behave very scientifically, by not taking the time to properly investigate all sides of the evidence.

One of the complete fails of the audio industry is to submit their claims for proper independent study, by scientists. For example, no cable maker has ever put their cables to be tested at a university engineering department to validate their claims about enhanced sound quality.

True. And the reason that they get away with it is because most of their customers don't really care about the science bit.

However, I did start a thread once about Acuity and two cable manufacturers, where exactly this was intended. Sadly, after the hype, the initial results were disappointingly amateurish, at least as far as the sound scientists here were concerned. That group still publishes results I think.

There is no flaw in a properly conducted blind test. It is a case that many audiophiles do not know what to do with the results of those tests, especially since they are contrary to what they expected/hoped for.

You're just following my prediction again.

BTW, I'm not trying to say that one side is all good and the other all bad. Just that some types of tests are unreliable for certain types of sound (and taste) differences.

I am sorry, but that is nonsense. Blind tests that are positive show an audible difference but blind tests that are negative do not show an inaudible difference?!

For you to dismiss one set of results and accept another from exactly the same test procedure is very bad science indeed.

In a way he is right.. you cannot really prove a negative in this way.

Quote:

For that reason, ABX testing is mainly a tool for proving that a difference does exist, i.e. the burden of proof should be on those who claim there is an audible difference, and they can use reliable positive results (which do not even have to be in a majority) to back it up.

The "blindness" of the test is a necessary condition to exclude biases that are already proven to exist.