Difficulty of blind testing
Jun 10, 2009 at 9:14 PM Post #61 of 117
Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif
I'll back-pedal a little and say that even though a test cannot directly control the use of the attention, it certainly can influence it. For example, a quick-switching protocol, or one that uses segments like you describe, has a very strong influence on how someone uses their attention. See my post earlier in this thread for my explanation why this can block access to information.


We will have to differ on this one, I find it much more effectve than long term listening especialy if the differences are small.


Quote:

If 100 subjects (who are all using their attention in their own way) can't detect something, it could mean:
  1. No one can detect it.
  2. The protocol itself makes the subjects insensitive to the key differences.
  3. It can be detected, but only by an unusual use of the attention.


It is rash to say "No one" since there may always be one undiscovered individual out there with greater discrimination talents , best you could say the odds against it lengthen and once you have several 1000s with the same result you start taking its pragmatically as read and stop testing for it so keenly. Since audio is less critical than say drug testing (where you really do have to be extreme) this is rational.

As for 2 and 3 perhaps but that is not informative or predictive and then requires you to devise alternative and suitably rigorous protocols, it does not let you fall back to "my wife was in the kitchen and she heard the dfference"
wink_face.gif


Quote:

Remember that some people spend much more time introspecting on how they use their attention. Being a musician and constantly working with small differences in sound could influence this. So could being an instrument-builder. Someone who practices mindfulness meditation would be influenced.


Maybe, but what "real" evidence for this do you have, you are speculating a lot here


Quote:

It comes down to this: most audiophiles get the impression of differences between something controversial (like cables) in either initial impressions, or long-term living-with-the-component. Very, very few DBTs replicate these conditions.


I would venture that is why we have so much faulty evidence on the effect of cables, the assessment is done in a wholly uncontrolled manner where bias an expectation simply cannot be removed.

When Wlson did listening tests on some speakers they got one sample of listeners to be able to see the speakers and the others could not, without fail the sighted listeners always gave the speakers (visually impressive and expensive looking) higher ratings , it is just a human thing, similarly a big blingy(sp) cable compared to a stock cable....

Quote:

EDIT: you are clearly listening to sound. I listen for music. That right there is a huge difference in how we use our attention.


I listen to sound only to see if I can tell a difference, normally I listen to music, but if I listen to music when trying to tell the difference between sounds I cannot tell samples apart so easily because my concentration is focussing on different things.

That said one could run a set of experiments where you have varying levels of difference between stimuli and you have different groups assigned and required to use one strategy or another.
 
Jun 11, 2009 at 9:54 PM Post #62 of 117
Quote:

Originally Posted by nick_charles /img/forum/go_quote.gif

I listen to sound only to see if I can tell a difference, normally I listen to music, but if I listen to music when trying to tell the difference between sounds I cannot tell samples apart so easily because my concentration is focussing on different things.



I have no doubt that is true for the kinds of differences that are being tested by you. But I doubt it's true for all differences. People who advocate "snippets" or "listening to sound as sound" are making an unfounded assumption. It is this: any difference which affects the experience of sound as music, can also be experience in sound as sound. In this thread:

http://www.head-fi.org/forums/f133/e...risons-428624/

I explain why this is an unfounded and unlikely assumption, and why it arises from a particular kind of illusion.
Quote:

That said one could run a set of experiments where you have varying levels of difference between stimuli and you have different groups assigned and required to use one strategy or another.


I think this is a good idea.
 
Jun 12, 2009 at 4:51 AM Post #63 of 117
Quote:

Originally Posted by xolp /img/forum/go_quote.gif
... are you intending to also listen to an unknown X for some time and identify it?


No, my wiring won't allow for that. The analog connection from my CDP/DAC to my amp will go either thru a high-end RCA cable pair or very cheap one, depending on how I move a jumper pair on the front of a patch panel. I won't know which, because I can't see the back of the patch panel, and I won't have been the one to wire it. (All other signal paths in this chain will be high-end -- the jumper and the patch panel were custom made -- and the source, electronics, etc. will all be beyond criticism that they are not good enough to reveal cable differences).

I'll have local Head-Fi'ers over to see it ... already have some volunteers, although I am still months away. I will switch many times, when I want to, and see if I can hear differences. Controlled but relaxed. No excuses possible.

I have argued in other threads that directly testing A vs B is as good, and perhaps better, than A/B/X. Anyway I can't do A/B/X in my set-up, so that point is moot. Regardless of whether you agree with that or not, no one would deny that if I can, or cannot, hear a difference between, or state a preference for, A or B over many trials, I will have reached an interesting conclusion.

This test is half-blind (I know the switch has been made but don't know which is which). If I feel that I can reliably tell the difference, then I will try it single-blind (turn my back and have my wife move the jumper, randomly giving me some A/A and B/B tests which will smoke out the truth).

My fondest hope is that I can tell the difference, can replicate that blind, and that I prefer one cable to the other (boy it would be cool if I liked the cheap one better!), or else have no strong preference even though I can tell the difference. Second best would be to never hear any difference, even though I know I switched the cable.

Unfortunately, this is what I think will happen instead: I will be convinced I can tell the difference between A and B. Every time! But I will fail to do so blind. 50% chance that I will say I prefer A and it is the cheap cable, but with my luck it will be the expensive cable I claim (but cannot repeat when blind) that I like -- then I am right back to ground zero -- I am worried that the expensive cable really does sound better, and something is wrong with the blind test. But I will fight against that conclusion!

One way or the other this question will be settled for me.
 
Jun 13, 2009 at 10:11 AM Post #64 of 117
If you can hear a difference you can hear a difference PERIOD. What is so hard to understand? There is no need to bring others into the test, only yourself. If you can tell the difference 90% of the time, why bother with what others can hear? The whole point of the test is to determine if you've spent your money for a reason. What is so hard to understand here?? Exactly as I've stated before, there will always be naysayers who will come up with a reason for the test to be invalid.
 
Jun 13, 2009 at 11:16 AM Post #65 of 117
Quote:

Originally Posted by saintalfonzo /img/forum/go_quote.gif
If you can hear a difference you can hear a difference PERIOD. What is so hard to understand? There is no need to bring others into the test, only yourself. If you can tell the difference 90% of the time, why bother with what others can hear? The whole point of the test is to determine if you've spent your money for a reason. What is so hard to understand here?? Exactly as I've stated before, there will always be naysayers who will come up with a reason for the test to be invalid.


It never was about the test or the cables really. Its about people wanting to be able to say "Cables all sound alike" so they can A.) Rationalize using crap cables and B.) Go around telling other people that they waisted their money because the price of said cable is above a fictitious number that is to be determined at the time of the conversation.

I will also add that most of the "I know what your capable of hearing better than you know yourself" people don't use cheap cables.

Either they all sound alike and the $1.50 RatShacks will do or "Shut the Hell Up"!!!

Class Dismissed.
 
Jun 13, 2009 at 10:16 PM Post #66 of 117
Quote:

Originally Posted by olblueyez /img/forum/go_quote.gif
It never was about the test or the cables really. Its about people wanting to be able to say "Cables all sound alike" so they can A.) Rationalize using crap cables and B.) Go around telling other people that they waisted their money because the price of said cable is above a fictitious number that is to be determined at the time of the conversation.

I will also add that most of the "I know what your capable of hearing better than you know yourself" people don't use cheap cables.

Either they all sound alike and the $1.50 RatShacks will do or "Shut the Hell Up"!!!

Class Dismissed.




so what is it that compels you to post in the "Sound Science" forum? - you have a perfectly protected playground safe from contrary views in the "DBT Prohibited" Cable forum

you could present evidence for your position but putting an offensive caricature like the above "shut up" into the mouths of those you disagree with hardly advances rational discussion

for symmetry re DBT/non-DBT forum rules how about posting a link to your own "personal opinions only" thread in the Cable forum and quit being abusive of the discussion here
 
Jun 13, 2009 at 10:35 PM Post #67 of 117
olblueyez is trying to make you guys use your brains right for a change.

"When the artist's mind closes against the new ideas that are the mind's strength, as the sap is the trees, the brain becomes rigid, and arid, and neither philosophy, poetry nor painting can be produced thereby."
 
Jun 14, 2009 at 3:11 AM Post #68 of 117
Quote:

Originally Posted by jcx /img/forum/go_quote.gif
so what is it that compels you to post in the "Sound Science" forum? - you have a perfectly protected playground safe from contrary views in the "DBT Prohibited" Cable forum


I know... it is kind of amusing. These guys will chase you out of the cable forum because of your 'agenda' and then follow you here with great arguments such as 'shut up' or 'use your brains for a change.' Too funny.
 
Jun 14, 2009 at 3:38 AM Post #69 of 117
Quote:

Originally Posted by jcx /img/forum/go_quote.gif
so what is it that compels you to post in the "Sound Science" forum? - you have a perfectly protected playground safe from contrary views in the "DBT Prohibited" Cable forum


Umm.. excuse me? Cable forum "safe from contrary views"? Last time I posted a message there, half the replies were mocking or dismissive. Apparently as long as you don't mention DBT, you can be as much of an ******* as you want.
 
Jun 14, 2009 at 4:29 PM Post #70 of 117
Quote:

Originally Posted by olblueyez /img/forum/go_quote.gif
It never was about the test or the cables really. Its about people wanting to be able to say "Cables all sound alike" so they can A.) Rationalize using crap cables and B.) Go around telling other people that they waisted their money because the price of said cable is above a fictitious number that is to be determined at the time of the conversation.
.



The same argument can be used to the position against testing - people wish to rationalize purchases.

(Caveat: I use rat shack cables. Some rat shack rca-1/4" adapters are noisy.)

Speaking as a social scientist, I would argue that the people most vocal on either side of the argument are very likely those least certain about their position - otherwise they would have no stake in what positions others hold.

Me, I just like a good debate, especially when the two sides have different worldviews.
 
Jun 14, 2009 at 4:32 PM Post #71 of 117
"Speaking as a social scientist, I would argue that the people most vocal on either side of the argument are very likely those least certain about their position - otherwise they would have no stake in what positions others hold."

olblueyez isn't being vocal on the pro-cable side, he's just pointing out how absurd anti-cablers are.
 
Jun 14, 2009 at 4:35 PM Post #72 of 117
Quote:

Originally Posted by haloxt /img/forum/go_quote.gif
"Speaking as a social scientist, I would argue that the people most vocal on either side of the argument are very likely those least certain about their position - otherwise they would have no stake in what positions others hold."

olblueyez isn't being vocal on the pro-cable side, he's just pointing out how absurd anti-cablers are.



I didn't say he was
smily_headphones1.gif
 
Jun 14, 2009 at 4:43 PM Post #73 of 117
Quote:

Originally Posted by ph0rk /img/forum/go_quote.gif
Me, I just like a good debate, especially when the two sides have different worldviews.


To me, the most interesting part of the debate is epistemology; that is, how we know things about audio. One side says that conventional DBT technology is a good way to know things; on the other side myself,, I suggest that simple introspection of the process of becoming conscious of sound reveals the problems in conventional DBTs. This is the "thoughtful" version of each side; there is also the more "reactive" version.

The reactive version of the pro-cablers is the person who says "I heard it, I know it, I don't want to think about it any more." This is motivated in part by wanting to be certain of their knowledge.

The reactive version of the anti-cabler, pro-DBT is similar: "We ran the test, end of story." I also think this is motivated by wanting to be certain, by wanting no holes and gray areas.

In each case, I think it is anxiety about the unknown and the potentially unknowable which motivates the person.
 
Jun 14, 2009 at 4:46 PM Post #74 of 117
Quote:

Originally Posted by haloxt /img/forum/go_quote.gif
olblueyez isn't being vocal on the pro-cable side, he's just pointing out how absurd anti-cablers are.


There's plenty of practical evidence that differences in cables cannot be heard. There are scientific measurements which prove that any interference introduced by a cable is orders of magnitude below what a human being is capable of hearing and there is far more anecdotal evidence against audible differences in cables than there are for audible differences.

Perhaps you would care to explain why anti-cablers are more absurd than pro-cablers? Or are you just saying this because you have no logical or rational argument and you're just being defensive?

G
 
Jun 14, 2009 at 4:47 PM Post #75 of 117
Quote:

Originally Posted by saintalfonzo /img/forum/go_quote.gif
If you can hear a difference you can hear a difference PERIOD. What is so hard to understand? There is no need to bring others into the test, only yourself. If you can tell the difference 90% of the time, why bother with what others can hear? The whole point of the test is to determine if you've spent your money for a reason. What is so hard to understand here?? Exactly as I've stated before, there will always be naysayers who will come up with a reason for the test to be invalid.


SeeAnyBlackDots.jpg


The black dots really are there because you can see them. Right?

Do not underestimate your brain. It is extremely difficult to control for suggestion, expectations, the influence of beliefs, not to mention the quirks that make you see - and hear - things that aren't there.

The only way to get rid of those problems is if people can't see what cable they're listening to.

No matter the methodology, no one has ever passed a blind test.

The problem isn't the methodology.
 

Users who are viewing this thread

Back
Top