1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.

    Dismiss Notice

What are the arguments against double blind tests (incl. ABX)?

Discussion in 'Sound Science' started by skamp, Mar 7, 2012.
First
 
Back
4 5 6 7 8 9 10 11 12 13
  1. Mischa23v

    I agree but, unfortunately in this matter, we can’t all be the same way. Take my advice and try some relaxing yoga, or just listen to some relaxing tunes and let it go.
     

     
    Quote:


     
     
     
  2. skamp
    If anything, the member in question dropped the ball by refusing to debate, making a point without backing it up. Unsubstanciated claims are a dime a dozen. I won't be losing any sleep over it.
     
  3. liamstrain


    Quote:


    Aye, probably the best advice. You can keep the yoga though. :wink:
     
  4. Prog Rock Man
    Listening to music whilst posting keeps me happy, no matter which way the debate is going. [​IMG]
     
  5. Chris J


     
    Personally I had dropped out for a few reasons:
     
     - I am a DBT sceptic, I neither believe nor dis-believe in the results of DBT
     - I feel out of my depth discussing DBT, it is way out of my expertise
     - therefore, I would rather lurk and do this................
     

     

    PRM, I like your signature  "It is rather striking how in a hobby based on the creations of science....etc"   who is Wapiti?
     
    Quote:


     
     
  6. TheAttorney


    Quote:
    If this was directed at me, then let me explain why I backed out:
    In several similar threads in the past, various people, and sometimes I, have posted links to evidence and got into discussions where, no matter what effort was applied, all ended up in stalemate. Each time losing a few more people, vowing never to return.
     
    For this thread, which has been more civilised than most, the OP seemed to be inviting input from "the opposition" and, as there was so little forthcoming, I contributed my thoughts. But as all my ideas were met with "gimme hard evidence", I realised that I could either spend a lot of time and effort  bringing up such evidence, or just cut my losses and run. I hope you see the irony in expecting a subjectivist to bring along hard evidence :xf_eek:)
     
    It was my mistake for thinking that ideas and suggestions alone were of any interest at all in the Science forum. No hard feeling though from my side - this was just the wrong place for me, and I shouldn't have even started. So, no more contributions in the future to such threads, but I'll continue lurking because the topic is still intriguing.   
     
  7. liamstrain
    No, it was not directed at you. But thank you for explaining your position further. :)
     
    I do see the irony in expecting the subjectivist to provide evidence... but not as much as the irony in their expecting us discontinue something that can provide useful data, without offering a compelling case (in particular showing why the data a DBT or ABX generates would be invalid for these uses) for why we should do so. Ideas and suggestions are fantastic - they are the seed of discovery - but without evidence to support them, they are not enough to invalidate existing data and methodologies.  :)
     
  8. skamp
    I don't believe it was directed at you.

    I started this thread because I don't think I've heard all the arguments against DBTs. It's kinda hard to be absolutely certain of something if you haven't had the opportunity to reflect on possible flaws in your reasoning.

    I haven't heard any compelling argument against DBTs so far. I feel the discussion has been productive, however: it has helped me determine with more clarity what DBTs prove and what they don't prove.

    What I've observed however, is that subjectivists tend to refuse submitting to such tests while providing what I consider to be poor excuses. The conclusion I make is that they're anticipating failure, and don't want to give objectivists more empirical evidence of inaudibility. What we have overall is simply lack of proof either way; unfortunately for the subjectivists, the burden of proof lies on them (can't prove a negative), and they're not delivering.

    My take: I think I said earlier that the subjectivists shouldn't be afraid of DBTs, and they should start by submitting to tests that they have a chance of passing. ABXing lossy codecs with foobar at bitrates that approach transparency is something anyone can do, and it would reveal just what kind of subtle differences can be heard. I believe it would demystify the whole thing.

    Of course, the results could be embarrassing: some people have reported that they can't even ABX a 128kbps MP3. The difference is, the people who admit to it usually never made any claims beforehand. And that's where I think lies the bigger problem with people on both sides of the fence: ego gets in the way. It takes a big man to admit to being wrong.

    I think a good way of eliminating ego is to only believe in, and claim, what is proven. I'd say that's the objectivist way, though we're not immune to exageration either.

    That said, I doubt anyone will blame me for consciously enjoying my placebo :)
     
  9. Prog Rock Man


    Quote:

    Another forum member.
     
     
  10. Mkubota1 Contributor
     
    Quote:

    LOL... thanks for that laugh!
     
    After a good amount of reading and skimming, I find the lack of rebuttals here is disappointing.  I guess if you ever find yourself in a room full of these people and want some privacy…
     
    Anyhow, I thought of an example that might bolster the argument of not being able to detect small differences via fast switching.  Let's say you had an incline or treadmill where the grade or steepness could be changed from 2% to 2.1%.  One might argue that most people would not be able to tell the difference between the two grades whether it was a second or a few minutes between adjustments.  But if you walked for an hour or two at the different grades there would be a difference in your level of fatigue.  The same could apply to sound- especially the fatigue (or lack thereof) part.  The problem is, there is no time limit to ABX/DBT testing.  Oh well.
     
     
    Quote:

    Can we at least get them to say the following?:  "Placebo and expectation bias are trivial issues when evaluating, and they can be overcome within oneself."
     
     
     
  11. Chris J


    Quote:


    I talked to a friend of mine about this.............to make a real long real story short you would need an advanced degree in Sociology or Psychology to come up with a solid rebuttal.
    And I don't have a degree in Sociology or Psychology.
     
     
  12. AKG240mkII
     
    Quote:
     
     


    Actually, all you need is a blind-test ..
     
     
  13. Chris J
     
    Quote:
     
    You are entirely missing the point.
    The point is, the test DOES NOT WORK!
    And another thing...............You know more about Psychology than he does....................?
     
  14. liamstrain
     
    Quote:
     
    There are some effective arguments against them for some types of data (both ABX and Double Blind). But for other types of data - especially user experiences and preferences, they remain valid when effectively designed and implemented. 
     
    I have yet to see an argument that invalidates the test methods outright. Your friend is welcome to share some of the evidence he may have. A few of us here have other advanced degrees that rely on statistical data, testing methodology and other related fields and I'm sure we would be curious to learn from him. 
     
    All that is a polite way of saying - your statement that you have a friend who has a PhD that states they can be invalidated, is not an argument against them. It is a fallacy. An appeal to authority and entirely unsupported at that. No different than my stating that I have a friend who has a PhD in sociology, and a father in-law who has PhD in psychology who states that they are valid. 
     
    *shrug*
     
First
 
Back
4 5 6 7 8 9 10 11 12 13

Share This Page