1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.

    Dismiss Notice

What are the arguments against double blind tests (incl. ABX)?

Discussion in 'Sound Science' started by skamp, Mar 7, 2012.
First
 
Back
2 3 4 5 6 7 8 9 10 11
13 14
Next
 
Last
  1. skamp

    Bad example, if you ask me. I downloaded both pictures and viewed them back to back with fast switching to detect any differences, which is exactly how I would conduct an ABX test. I immediately saw one difference, a small shadow in the bottom right corner. The rest is, as far as I can tell, identical. I have no idea what you mean by "more balanced". Also, I don't have a preference for either one.
     
  2. Chris J


    Quote:


    Sorry, I think I owe you an apology, sir!
    Let us now metaphorically shake hands!
    [​IMG]
     
    I agree, if you make a BIG statement like "OMG! I can hear NIGHT AND DAY differences between brand X and brand Y!" components then it should hold up to DBT whether or not DBT is actually flawed.
    Personally I believe that I can hear the differences between some cables, the difference is subtle, but I think I would feel too much pressure in a DBT test. 
    That's just a theory as I have never participated in a true DBT, so what do I know?
    I have particitated in a few casual ones. I remember one casual ABX where a friend and I could pick out the difference between a Threshold power amp and another brand but could not hear the difference between two different models of Threshold power amps. Obviously this is what we call anecdotal.
    Personally, I have never heard NIGHT! AND! DAY! difference in cables or any other equipment other than loudspeakers, headphones and (deep breath!) turntables![​IMG]
     
     
  3. stv014
    Quote:

    Why require half an hour, when there is no such limit (minimum or maximum) in audio ABX tests ? In fact, the easiest way to tell the two pictures apart is if they can be shown with rapid switching at the same position on the screen; that makes the change more obvious than seeing both side by side.
     
     
  4. khaos974
    Quote:
     
    Quote:

    Yes, the difference is obvious with fast switching, but since audio memory is notoriously short term compared to visual memory, I made the hypothesis that waiting half an hour in a visual test would be like fast switching for audio, that's the argument, what if fast switching is audio is still not fast enough? Anyway, see you tomorrow.
     
     
     
  5. skamp

    Experience shows that it is.
     
  6. stv014
    Quote:

    Well, as you noted, this (with the arbitrary choice of a time interval of half an hour) is only a hypothesis that is not proven. The best way to find out what kind of testing is most effective for audio in practice is to test the tests themselves using audio files with intentionally added subtle differences. In fact, this type of experiment was already done, and ABX testing (with fast switching allowed) was found to produce more accurate results.
     
     
  7. EthanWiner
    Quote:

    It's simpler than all this. All that matters is if you can hear a difference. Now, if 500 tests of other people shows no statistical significance, then I'll accept that too for myself. But for those who don't accept blind tests of others, the answer is to do your own tests! Learning how to do such tests also makes one a more knowledgeable audiophile. This is the beauty of ABX software. People can test themselves honestly, and know for certain what matters or not to them. You think you can tell the degradation of MP3 compression at 320 kbps? Take four sample tracks, encode them, and test yourself. This is far more productive than arguing for weeks on end in an audio forum. [​IMG] By testing yourself, in half an hour you'll have your answer.
     
    --Ethan
     
  8. scuttle


    Quote:

    It's easy to design a blind test to prevent this being a factor. E.g. if you're testing 320 br MP3 against FLAC then you also include 128 or 64 br MP3. Plus you multiple trials on the same individuals - if you have two guys in a group of 10 who identify an "improvement" in each of 8 trials, that means a lot. The protocols for blind testing were worked out decades ago, they're easily applied.
     
  9. Draygonn
    If you use a seamless transition you would have to rely less on memory, you could simply listen for any changes during the switch.
     
  10. stv014
    Quote:

    Exactly. This is the same reason why I suggested that the picture test of khaos974 is easiest if the pictures can be switched quickly at the same position on the screen. Of course, the requirement of a seamless transition makes it even more important to ensure the best possible level matching and sample accurate synchronization. Although a very short fade out/in at the transition would also help eliminating any small glitches that could give spurious clues.
     
     
  11. Prog Rock Man
    How can you be so sure memory loss is an issue? Could it not be that your memory is fine and it remembers what you heard before and there is either no difference or a difference so small it no longer matters?
     
    If memory loss is such an issue, do any of you audition hifi? If so how, since as products are switched, you will not according to you remember the previous sound.
     
  12. khaos974
    Quote:

    Stop putting words in my mouth, I never wrote that I was sure memory loss is an issue, I made several conjectures, I speculated, I asked, I used "may"/"would", I never wrote I was sure of anything: I'm not, I'm simply asking if there could be difference small enough not to be ABXable but big enough to be perceivable (if subconsciously).
     
    It's somewhat an extension of the fact that even failing 10 time an ABX testing, you can still succeed once you are told what to listen to, you have the capacity to hear a difference, but that difference doesn't register consciously, and thus you fail the ABX.
     
     
     
  13. Prog Rock Man
    Khaos974, I was responding to the two posts above mine, not yours.
     
  14. khaos974
    Quote:

    My bad.
     
    Anyway, I was thinking about another case of classical ABX conditions possibly failing to show a perceivable phenomenon, suppose fast switch with shortish listening sessions.
     
    Let's imagine a signal with a loud 22 kHz frequency in it, you don't hear that 22 kHz, you can't ABX it, but if that 22 kHz was played all day long at high volumes it still may be possible to be irritated by it.
     
     
  15. skamp

    Would it, if you can't hear it? When I was a kid, I was walking through the entrance hall of a large retailer, and I suddently covered my ears because I was hearing a painful, high pitch tone. I don't know what was emitting it, but no-one else seemed to be bothered. Adults simply couldn't hear what my young ears could.

    Anyway, that's a rather extreme example, that doesn't relate to any of the differences that audio components might show.
     
First
 
Back
2 3 4 5 6 7 8 9 10 11
13 14
Next
 
Last

Share This Page