What are the arguments against double blind tests (incl. ABX)?
Mar 21, 2012 at 4:27 PM Post #196 of 209

[size=medium]I agree but, unfortunately in this matter, we can’t all be the same way. Take my advice and try some relaxing yoga, or just listen to some relaxing tunes and let it go. [/size]
 

 
Quote:
If I thought the person I was referring to was a child, I'd agree... this was a longstanding adult member. 
 
Also, I don't see these things as winning or losing ... the truth matters. I'll happily lose, if the end result is a more accurate understanding. But that's just me. I have a very hard time when people wantonly dismiss facts and the tools used to determine them without even the courtesy of explaining their reasons for doing so. 



 
 
 
Mar 21, 2012 at 4:49 PM Post #198 of 209


Quote:
[size=medium]listen to some relaxing tunes and let it go. [/size]
 



Aye, probably the best advice. You can keep the yoga though. :wink:
 
Mar 21, 2012 at 4:58 PM Post #199 of 209
Listening to music whilst posting keeps me happy, no matter which way the debate is going.
bigsmile_face.gif

 
Mar 21, 2012 at 7:50 PM Post #200 of 209


 
Personally I had dropped out for a few reasons:
 
 - I am a DBT sceptic, I neither believe nor dis-believe in the results of DBT
 - I feel out of my depth discussing DBT, it is way out of my expertise
 - therefore, I would rather lurk and do this................
 

[size=medium]Take my advice and try some relaxing yoga, or just listen to some relaxing tunes and let it go. [/size]

 

 

PRM, I like your signature  "It is rather striking how in a hobby based on the creations of science....etc"   who is Wapiti?
 
Quote:
Listening to music whilst posting keeps me happy, no matter which way the debate is going.
bigsmile_face.gif



 
 
Mar 22, 2012 at 9:35 AM Post #201 of 209


Quote:
liamstrain said:


.....Declining to discuss here, naturally - and just saying "do a search" when questioned further. 
 
Very frustrating.

If this was directed at me, then let me explain why I backed out:
In several similar threads in the past, various people, and sometimes I, have posted links to evidence and got into discussions where, no matter what effort was applied, all ended up in stalemate. Each time losing a few more people, vowing never to return.
 
For this thread, which has been more civilised than most, the OP seemed to be inviting input from "the opposition" and, as there was so little forthcoming, I contributed my thoughts. But as all my ideas were met with "gimme hard evidence", I realised that I could either spend a lot of time and effort  bringing up such evidence, or just cut my losses and run. I hope you see the irony in expecting a subjectivist to bring along hard evidence :xf_eek:)
 
It was my mistake for thinking that ideas and suggestions alone were of any interest at all in the Science forum. No hard feeling though from my side - this was just the wrong place for me, and I shouldn't have even started. So, no more contributions in the future to such threads, but I'll continue lurking because the topic is still intriguing.   
 
Mar 22, 2012 at 10:14 AM Post #202 of 209
No, it was not directed at you. But thank you for explaining your position further. :)
 
I do see the irony in expecting the subjectivist to provide evidence... but not as much as the irony in their expecting us discontinue something that can provide useful data, without offering a compelling case (in particular showing why the data a DBT or ABX generates would be invalid for these uses) for why we should do so. Ideas and suggestions are fantastic - they are the seed of discovery - but without evidence to support them, they are not enough to invalidate existing data and methodologies.  :)
 
Mar 22, 2012 at 10:27 AM Post #203 of 209
I don't believe it was directed at you.

I started this thread because I don't think I've heard all the arguments against DBTs. It's kinda hard to be absolutely certain of something if you haven't had the opportunity to reflect on possible flaws in your reasoning.

I haven't heard any compelling argument against DBTs so far. I feel the discussion has been productive, however: it has helped me determine with more clarity what DBTs prove and what they don't prove.

What I've observed however, is that subjectivists tend to refuse submitting to such tests while providing what I consider to be poor excuses. The conclusion I make is that they're anticipating failure, and don't want to give objectivists more empirical evidence of inaudibility. What we have overall is simply lack of proof either way; unfortunately for the subjectivists, the burden of proof lies on them (can't prove a negative), and they're not delivering.

My take: I think I said earlier that the subjectivists shouldn't be afraid of DBTs, and they should start by submitting to tests that they have a chance of passing. ABXing lossy codecs with foobar at bitrates that approach transparency is something anyone can do, and it would reveal just what kind of subtle differences can be heard. I believe it would demystify the whole thing.

Of course, the results could be embarrassing: some people have reported that they can't even ABX a 128kbps MP3. The difference is, the people who admit to it usually never made any claims beforehand. And that's where I think lies the bigger problem with people on both sides of the fence: ego gets in the way. It takes a big man to admit to being wrong.

I think a good way of eliminating ego is to only believe in, and claim, what is proven. I'd say that's the objectivist way, though we're not immune to exageration either.

That said, I doubt anyone will blame me for consciously enjoying my placebo :)
 
Mar 22, 2012 at 1:28 PM Post #204 of 209


Quote:
 
 

PRM, I like your signature  "It is rather striking how in a hobby based on the creations of science....etc"   who is Wapiti?
 


 


Another forum member.
 
 
Apr 14, 2012 at 5:33 AM Post #205 of 209
 
Quote:
Edit: it just occured to me that they might not visit the Sound Science forum :-/


LOL... thanks for that laugh!
 
After a good amount of reading and skimming, I find the lack of rebuttals here is disappointing.  I guess if you ever find yourself in a room full of these people and want some privacy…
 
Anyhow, I thought of an example that might bolster the argument of not being able to detect small differences via fast switching.  Let's say you had an incline or treadmill where the grade or steepness could be changed from 2% to 2.1%.  One might argue that most people would not be able to tell the difference between the two grades whether it was a second or a few minutes between adjustments.  But if you walked for an hour or two at the different grades there would be a difference in your level of fatigue.  The same could apply to sound- especially the fatigue (or lack thereof) part.  The problem is, there is no time limit to ABX/DBT testing.  Oh well.
 
 
Quote:
Yep, you're right. It's very frustrating. These threads come and go, and each time there's some interesting food for thought, and each time it ends up stalemate. Which is why I will join the all the other people and not contribute to such threads any more. Ever! The frustrating thing for me is my earlier quote below. This probably matches the frustration of the scientists expecting scientific proof for everything. The topic is inherently interesting, but I'm past caring now. Time to move on.


Can we at least get them to say the following?:  "Placebo and expectation bias are trivial issues when evaluating, and they can be overcome within oneself."
 
 
 
Apr 15, 2012 at 6:59 PM Post #206 of 209


Quote:
 

LOL... thanks for that laugh!
 
After a good amount of reading and skimming, I find the lack of rebuttals here is disappointing.  I guess if you ever find yourself in a room full of these people and want some privacy…
 



I talked to a friend of mine about this.............to make a real long real story short you would need an advanced degree in Sociology or Psychology to come up with a solid rebuttal.
And I don't have a degree in Sociology or Psychology.
 
 
Apr 20, 2012 at 4:55 PM Post #207 of 209
 
Quote:
I talked to a friend of mine about this.............to make a real long real story short you would need an advanced degree in Sociology or Psychology to come up with a solid rebuttal.
And I don't have a degree in Sociology or Psychology.
 

 
 


Actually, all you need is a blind-test ..
 
 
Apr 20, 2012 at 5:46 PM Post #208 of 209
 
Quote:
 
 
 


Actually, all you need is a blind-test ..
 

 
You are entirely missing the point.
The point is, the test DOES NOT WORK!
And another thing...............You know more about Psychology than he does....................?
 
Apr 20, 2012 at 6:27 PM Post #209 of 209
 
Quote:
 
The point is, the test DOES NOT WORK!
 
And another thing...............You know more about Psychology than he does....................?

 
There are some effective arguments against them for some types of data (both ABX and Double Blind). But for other types of data - especially user experiences and preferences, they remain valid when effectively designed and implemented. 
 
I have yet to see an argument that invalidates the test methods outright. Your friend is welcome to share some of the evidence he may have. A few of us here have other advanced degrees that rely on statistical data, testing methodology and other related fields and I'm sure we would be curious to learn from him. 
 
All that is a polite way of saying - your statement that you have a friend who has a PhD that states they can be invalidated, is not an argument against them. It is a fallacy. An appeal to authority and entirely unsupported at that. No different than my stating that I have a friend who has a PhD in sociology, and a father in-law who has PhD in psychology who states that they are valid. 
 
*shrug*
 

Users who are viewing this thread

Back
Top