[b0dhi]

Quote:

Originally Posted by Shark_Jump /img/forum/go_quote.gif Yes, I am not arguing this. I was just stating that if the ABX listening test is designed correctly then placebo should no impact on the final result of the test.

I'm not sure if this is true. The listener can't get a significant result due to placebo in an ABX. ABX guards against this. However, it's possible that the listener believes they can't hear a difference (whether they can or not), and their responses can be (subconsciously) random. ABX can't guard against this. I.e., ABX can't differentiate between a "real" random guess and a placebo-induced random guess.

 

[eucariote]

^ If the tester is truly blind to the source, visual cues, preconceptions etc. won't have an influence. More importantly, if enough samples are taken to eliminate a type I (<5% chance of false positive) or type II error (<5% chance of false negative), hypothesis testing is designed to eliminate the effects of chance variation. If the tester is blind to the source, and can choose the better headphone/cable, etc. with a frequency that that would occur less than 5% due to random variation assuming a binomial distribution of outcomes, than the effect is real. If the choices fall within a 95% distribution of random outcomes, than the effect is not real. This is the same technique that is used to say that 'x' has an effect on 'y' in medical and scientific journals (what I do for a living). Lives depend on the validity of DBT, statistics and hypothesis testing. It's good enough for audio equipment.

 

[b0dhi]

Quote:

Originally Posted by eucariote /img/forum/go_quote.gif ^ If the tester is truly blind to the source, visual cues, preconceptions etc. won't have an influence. More importantly, if enough samples are taken to eliminate a type I (<5% chance of false positive) or type II error (<5% chance of false negative), hypothesis testing is designed to eliminate the effects of chance variation. If the tester is blind to the source, and can choose the better headphone/cable, etc. with a frequency that that would occur less than 5% due to random variation assuming a binomial distribution of outcomes, than the effect is real. If the choices fall within a 95% distribution of random outcomes, than the effect is not real. This is the same technique that is used to say that 'x' has an effect on 'y' in medical and scientific journals (what I do for a living). Lives depend on the validity of DBT, statistics and hypothesis testing. It's good enough for audio equipment.

The tested may have a strong belief that cables sound the same. If the tested also knows that the ABX is testing audibility of cables (for example), this can lead to the same type of "placebo" effect that cable "believers" can be influenced by, except in this case they "hear" all cables sounding the same. Due to this, their responses emulate random guesses, since this is what they "hear". Note that for this type of error to occur, the tester doesn't need to see the cables.

Not that this type of error can't be guarded against. I just rarely if ever see audio tests designed to do so.

 

[haloxt]

B0dhi put it much better than I could. Sure is nice to know I'm not the only one who reached that crazy conclusion.

 

[eucariote]

Quote:

Originally Posted by b0dhi /img/forum/go_quote.gif The tested may have a strong belief that cables sound the same. If the tested also knows that the ABX is testing audibility of cables (for example), this can lead to the same type of "placebo" effect that cable "believers" can be influenced by, except in this case they "hear" all cables sounding the same. Due to this, their responses emulate random guesses, since this is what they "hear". Note that for this type of error to occur, the tester doesn't need to see the cables. Not that this type of error can't be guarded against. I just rarely if ever see audio tests designed to do so.

Interesting point. You could set up some general expectation like 'the audio is coming from two different systems' and not even tell them that just the cables, etc. are being changed. Even so, forcing a choice (these tests are called 'two alternative forced choice') often has very clear results even if the tested does not believe that they are perceiving any differences. An example of this is blindsight. If a person has their visual cortex blown out, they insist that they cannot see anything, and that is true, insofar as seeing is largely conscious experience of activity in primary and associative visual cortex. However the superior colliculus receives a visual input that branches off early from inputs going to visual cortex, which can do enough processing to allow people to choose certain kinds of visual stimuli correctly almost 100% of the time despite their insistence that they are not seeing anything. And if the stimuli are identical/imperceptibly different, then those choices really will be random and never reach statistical significance.

 

[aimlink]

Quote:

Originally Posted by b0dhi /img/forum/go_quote.gif The tested may have a strong belief that cables sound the same. If the tested also knows that the ABX is testing audibility of cables (for example), this can lead to the same type of "placebo" effect that cable "believers" can be influenced by, except in this case they "hear" all cables sounding the same. Due to this, their responses emulate random guesses, since this is what they "hear". Note that for this type of error to occur, the tester doesn't need to see the cables. Not that this type of error can't be guarded against. I just rarely if ever see audio tests designed to do so.

The reason for the blinding is to minimize, not prevent the placebo effect or guessing. The only way to make the results not be influenced by these unwanted influences it to make many attempts to tell the difference and then assess the responses.

 

[Shark_Jump]

Interesting.

I thought a properly set up ABX test mitigates against all extraneous effects so they are statistically insignificant.

I didn't realise there were these issues, I didn't realise it was 'open to question' so to speak.

 

[Shark_Jump]

Could you give the impression that one of the cables has an 'error' in it.

Therby encouraging a response.

 

[b0dhi]

Quote:

Originally Posted by Shark_Jump /img/forum/go_quote.gif Could you give the impression that one of the cables has an 'error' in it. Therby encouraging a response.

Yes, this is one of the methods used. Wavoman has made some excellent posts on this topic over the years, I definitely recommend checking those out.