What are the arguments against double blind tests (incl. ABX)?
Mar 17, 2012 at 12:11 PM Post #166 of 209
The first picture is more balanced, but if you were asked to ABX them, viewing A, B, and X half an hour apart, it's would be quite probable that you would fail the ABX despite having a preference for A.


Bad example, if you ask me. I downloaded both pictures and viewed them back to back with fast switching to detect any differences, which is exactly how I would conduct an ABX test. I immediately saw one difference, a small shadow in the bottom right corner. The rest is, as far as I can tell, identical. I have no idea what you mean by "more balanced". Also, I don't have a preference for either one.
 
Mar 17, 2012 at 12:12 PM Post #167 of 209


Quote:
Oh, I didn't mean you! It was a figure of speech. If someone claims cable X makes a night and day difference, they better own up to it. That's what I meant. Bad, bad wording. Bad!



Sorry, I think I owe you an apology, sir!
Let us now metaphorically shake hands!
beerchug.gif

 
I agree, if you make a BIG statement like "OMG! I can hear NIGHT AND DAY differences between brand X and brand Y!" components then it should hold up to DBT whether or not DBT is actually flawed.
Personally I believe that I can hear the differences between some cables, the difference is subtle, but I think I would feel too much pressure in a DBT test. 
That's just a theory as I have never participated in a true DBT, so what do I know?
I have particitated in a few casual ones. I remember one casual ABX where a friend and I could pick out the difference between a Threshold power amp and another brand but could not hear the difference between two different models of Threshold power amps. Obviously this is what we call anecdotal.
Personally, I have never heard NIGHT! AND! DAY! difference in cables or any other equipment other than loudspeakers, headphones and (deep breath!) turntables!
redface.gif

 
 
Mar 17, 2012 at 12:13 PM Post #168 of 209
Quote:
Originally Posted by khaos974 /img/forum/go_quote.gif
 
The first picture is more balanced, but if you were asked to ABX them, viewing A, B, and X half an hour apart, it's would be quite probable that you would fail the ABX despite having a preference for A.


Why require half an hour, when there is no such limit (minimum or maximum) in audio ABX tests ? In fact, the easiest way to tell the two pictures apart is if they can be shown with rapid switching at the same position on the screen; that makes the change more obvious than seeing both side by side.
 
 
Mar 17, 2012 at 12:19 PM Post #169 of 209
Quote:
Bad example, if you ask me. I downloaded both pictures and viewed them back to back with fast switching to detect any differences, which is exactly how I would conduct an ABX test. I immediately saw one difference, a small shadow in the bottom right corner. The rest is, as far as I can tell, identical. I have no idea what you mean by "more balanced". Also, I don't have a preference for either one.

 
Quote:
Originally Posted by stv014 /img/forum/go_quote.gif
 
Why require half an hour, when there is no such limit (minimum or maximum) in audio ABX tests ? In fact, the easiest way to tell the two pictures apart is if they can be shown with rapid switching at the same position on the screen; that makes the change more obvious than seeing both side by side.


Yes, the difference is obvious with fast switching, but since audio memory is notoriously short term compared to visual memory, I made the hypothesis that waiting half an hour in a visual test would be like fast switching for audio, that's the argument, what if fast switching is audio is still not fast enough? Anyway, see you tomorrow.
 
 
 
Mar 17, 2012 at 12:39 PM Post #171 of 209
Quote:
Originally Posted by khaos974 /img/forum/go_quote.gif
 
I made the hypothesis that waiting half an hour in a visual test would be like fast switching for audio


Well, as you noted, this (with the arbitrary choice of a time interval of half an hour) is only a hypothesis that is not proven. The best way to find out what kind of testing is most effective for audio in practice is to test the tests themselves using audio files with intentionally added subtle differences. In fact, this type of experiment was already done, and ABX testing (with fast switching allowed) was found to produce more accurate results.
 
 
Mar 17, 2012 at 12:43 PM Post #172 of 209
Quote:
Is there any other type of tests that would undeniably prove the existence of audible differences? One that DBT/ABX detractors would approve of, hopefully?


It's simpler than all this. All that matters is if you can hear a difference. Now, if 500 tests of other people shows no statistical significance, then I'll accept that too for myself. But for those who don't accept blind tests of others, the answer is to do your own tests! Learning how to do such tests also makes one a more knowledgeable audiophile. This is the beauty of ABX software. People can test themselves honestly, and know for certain what matters or not to them. You think you can tell the degradation of MP3 compression at 320 kbps? Take four sample tracks, encode them, and test yourself. This is far more productive than arguing for weeks on end in an audio forum.
biggrin.gif
By testing yourself, in half an hour you'll have your answer.
 
--Ethan
 
Mar 17, 2012 at 12:45 PM Post #173 of 209


Quote:
 
You are way out of line when you claim that I am making "wild claims".

Firstly, there are a lot of people out there whom I suspect would not be good at DBT as I suspect they would probably choose a neutral stance rather than look like a fool.


It's easy to design a blind test to prevent this being a factor. E.g. if you're testing 320 br MP3 against FLAC then you also include 128 or 64 br MP3. Plus you multiple trials on the same individuals - if you have two guys in a group of 10 who identify an "improvement" in each of 8 trials, that means a lot. The protocols for blind testing were worked out decades ago, they're easily applied.
 
Mar 17, 2012 at 5:20 PM Post #174 of 209
If you use a seamless transition you would have to rely less on memory, you could simply listen for any changes during the switch.
 
Mar 17, 2012 at 6:15 PM Post #175 of 209
Quote:
If you use a seamless transition you would have to rely less on memory, you could simply listen for any changes during the switch.


Exactly. This is the same reason why I suggested that the picture test of khaos974 is easiest if the pictures can be switched quickly at the same position on the screen. Of course, the requirement of a seamless transition makes it even more important to ensure the best possible level matching and sample accurate synchronization. Although a very short fade out/in at the transition would also help eliminating any small glitches that could give spurious clues.
 
 
Mar 18, 2012 at 7:29 AM Post #176 of 209
How can you be so sure memory loss is an issue? Could it not be that your memory is fine and it remembers what you heard before and there is either no difference or a difference so small it no longer matters?
 
If memory loss is such an issue, do any of you audition hifi? If so how, since as products are switched, you will not according to you remember the previous sound.
 
Mar 18, 2012 at 7:56 AM Post #177 of 209
Quote:
How can you be so sure memory loss is an issue? Could it not be that your memory is fine and it remembers what you heard before and there is either no difference or a difference so small it no longer matters?
 
If memory loss is such an issue, do any of you audition hifi? If so how, since as products are switched, you will not according to you remember the previous sound.


Stop putting words in my mouth, I never wrote that I was sure memory loss is an issue, I made several conjectures, I speculated, I asked, I used "may"/"would", I never wrote I was sure of anything: I'm not, I'm simply asking if there could be difference small enough not to be ABXable but big enough to be perceivable (if subconsciously).
 
It's somewhat an extension of the fact that even failing 10 time an ABX testing, you can still succeed once you are told what to listen to, you have the capacity to hear a difference, but that difference doesn't register consciously, and thus you fail the ABX.
 
 
 
Mar 18, 2012 at 8:03 AM Post #178 of 209
Khaos974, I was responding to the two posts above mine, not yours.
 
Mar 18, 2012 at 8:10 AM Post #179 of 209
Quote:
Khaos974, I was responding to the two posts above mine, not yours.


My bad.
 
Anyway, I was thinking about another case of classical ABX conditions possibly failing to show a perceivable phenomenon, suppose fast switch with shortish listening sessions.
 
Let's imagine a signal with a loud 22 kHz frequency in it, you don't hear that 22 kHz, you can't ABX it, but if that 22 kHz was played all day long at high volumes it still may be possible to be irritated by it.
 
 
Mar 18, 2012 at 8:34 AM Post #180 of 209
if that 22 kHz was played all day long at high volumes it still may be possible to be irritated by it.


Would it, if you can't hear it? When I was a kid, I was walking through the entrance hall of a large retailer, and I suddently covered my ears because I was hearing a painful, high pitch tone. I don't know what was emitting it, but no-one else seemed to be bothered. Adults simply couldn't hear what my young ears could.

Anyway, that's a rather extreme example, that doesn't relate to any of the differences that audio components might show.
 

Users who are viewing this thread

Back
Top