[olblueyez]

Quote:

Originally Posted by xolp /img/forum/go_quote.gif I think one major problem with DBT advocates is their tone. DBT is NOT the universal answer to "Are there no audible differences between cables" or "Are expensive cables a waste of money" or worse "Are expensive cable buying audiphiles stupid". DBT is the answer to "Can I hear audible differences between cables" or "Are expensive cables a waste of money FOR ME"

Best post I have seen in quite some time. The newbie knows.

 

[xolp]

Quote:

Originally Posted by BIG POPPA /img/forum/go_quote.gif "I love that other issue about measurement vs listening—when people promote the blind panel test, which of course I have no faith in, or even interest in, other than to laugh at. If you were choosing a piano for a concert hall, would you get two or three name pianists that you knew and respected, or would you put it up to a blind committee to help you pick a concert-hall instrument" This is from David Manley of VTL in a Stereophile interview. Can't argue with the man.

I beg to differ. I'm not too sure whats the situation in the world of concert instruments, but I'm pretty sure the world of audio is chock full of advertising, preferential treatment for reviewers etc.

If these 3 name pianists were sponsored by Steinway, Bosendorfer and Yamaha, which pianos would you think each of them would choose.

But bear in mind as well, the ABX that we are suggesting is to isolate if they are any audible differences. The blind test vs reviewer that the quote suggests is to determine the best of the lot and the use of DBT in these situations is much more controversial.

 

[BIG POPPA]

To me when does DBT benefit on a piece of equipment? 100, 1,000, 100,000 dollars? Really, for most of us this is just a hobby. Have most of us been to meets to listen to gear? Probably not. What is DBT going to do to help the person interested in a piece of gear? How is it practical in a real world situation? Will DBT make us a better Head-Fier? Nice to wonder? Have better things to do like listen to music on my rig..... Very Often.

 

[mike1127]

Quote:

Originally Posted by nick_charles /img/forum/go_quote.gif I am a big fan of blind tests I have tried blind tests with complete tracks and I find it much harder to compare two tracks when the first track was 3 or 5 minutes ago it is very hard to hold details in memory. If the differences are gross then it might be easy but for subtle differences that span of time is not going to help much...I have had much more success with blind tests with fast switching between segments.

Regarding the duration of memory for sounds, I am aware that some kinds of psychology experiments have shown it is short. I'd like to know more about the studies that determined this. For I think the situation is complicated by several factors.

For one thing, there are phenomena which seem to suggest we can hold quite a detailed conceptual memory of a sound over time. For another, when I compare equipment, I don't really compare the "sound" so I don't need to "remember the sound." To take these one at a time:

First the phenomena:

When we are used to the sound of something---our spouse, or car engine---we can tell when it has changed from the norm. We store some kind of conceptual representation of sound and can compare it to the sound coming into our ears at that moment.

Many people experience aural hallucinations in which some familiar music plays, like an internal radio. I've never had a really fully detailed one, but reading a book on the phenomena, it seems that some people have astonishingly realistic hallucinations. So the sound is "stored" in there somewhere.

We can debate how accurate these memories of sound are, but the point is that the situation MUST be more complex than the simple statement: "aural memory is short." If we want to understand these additional phenomena, additional research is needed.

Now the second point. I don't really compare sound. I compare the reactions I have to the music. Here are some examples of reactions:

1. Articulation of fast notes is very light.
2. Opening chord makes me think of moving my arms in a big circle.
3. Decay length on harpsichord seems not to match Leonhardt's tempo... perhaps the system is misrepresenting it.
4. Inner voices come to my attention.
5. Vibrato is not audible.

I have hundreds of types of reactions like these. So what I do is take notes on my reactions, and compare them. Sometimes my reactions are strikingly different from system A to system B. And sometimes not. Just now I did a test of two cables in which I had nearly the same reactions to each cable, with only two real notable differences.

And YES I think these tests are uncontrolled. I am experimenting with protocol and timing that seems to make me most sensitive, but I think a blind test is the only way to remove my biases. I don't have the resources to do a blind test at the moment---no nearby audiophile friends to help with the setup. So I do the best I can.

 

[wavoman]

It seems easy to do a blind test. But it is not. I believe there are indeed contamination effects (I thank the OP for that wording) which are not easily solved by randomization. Even testing a single subject, and spreading the tests out over time (both of which solve certain problems) run in to difficulties.

I want blind tests to work. I want to know if cables make a difference. I am not attacking blind testing in an attempt to avoid their conclusions.

Think this way: a lot of us (trained, experieced listeners) hear differences in cables in our systems, but fail blind tests. So there are two explanations:

(a) We are all fooled by the placebo effect. We spent good money on the cables, we want there to be a difference, so we hear one.

(b) There is something about blind testing that we don't fully understand yet, and it makes passing them difficult even though we should pass.

Everyone immediately says (a). But I (and the OP) say -- could be (b). Now we have to be careful. We have to come up with a way to test and falsify (b), or this is just a meaningless defense that you could always invoke no matter what, and that is not my goal.

This does not happen in commercial taste testing (although I understand it has happened with wine). When people say they generally prefer Pepsi to Coke (or vice versa) in their daily lives, they are very able to tell them apart in a blind test. Ditto for some ice cream brands.

Why not audio?

I am wiring a new listening room. I really would like to use cheap cables. I bought one high-end analog interconnect off eBay to test (non-blind). I assumed, I hoped, I would not hear a difference. Damn it, I did. But I can't do it blind. My decision on cable quality will have a multi-thousand dollar impact, and due to WAF, will affect whether I can replace my crap 2002 Camry, which I really want to. Stakes for me are high.

Here is my multi-month blind test plan. I have invested in ultra-high-end custom built audiophile patch panels (RCA and XLR) "normal" style (back-to-front wiring). They should have virtually no impact on SQ (top of the line wire, solder, connectors). I have pushed my racks close together so that you cannot see the backs of any gear, not the backs of the patch panels. I am going to have a buddy wire both analog and digital interconnects thru the backs and fronts in alternate pathways so that I have both cheap and expensive cables in play, and the selection is made between A and B via front patch cords -- which I also had custom made with ultra-expensive cable and connectors (just a few).

Now I can control whether I listen to A or B, but have no idea which is (in the back) the cheap stuff or the good stuff. I will live with A for weeks, with B for weeks, etc. etc. I will switch when I want. The moment of truth will come, eventually, at least for me personally. I'll post.

Added: note that I have every economic incentive NOT to cheat -- I could move the racks and peek, but I don't want to.

 

[xolp]

Wonderful wavoman.

I think its attitudes like yours that will push a more methodical approach to evaluating audio components. Really look forward to your conclusion.

Edit: I just re read your post, are you intending to also listen to an unknown X for some time and identify it?

 

[mike1127]

Quote:

Originally Posted by wavoman /img/forum/go_quote.gif It seems easy to do a blind test. But it is not. I believe there are indeed contamination effects (I thank the OP for that wording) which are not easily solved by randomization. Even testing a single subject, and spreading the tests out over time (both of which solve certain problems) run in to difficulties. I want blind tests to work. I want to know if cables make a difference. I am not attacking blind testing in an attempt to avoid their conclusions.

Thanks for a thoughtful post on this thread (not too many of them) and I am in agreement, both about the difficulty in finding a protocol that can really control the test subject's use of their attention, and the fact that valid blind testing has big financial ramifications.
Quote:

Think this way: a lot of us (trained, experieced listeners) hear differences in cables in our systems, but fail blind tests. So there are two explanations: (a) We are all fooled by the placebo effect. We spent good money on the cables, we want there to be a difference, so we hear one. (b) There is something about blind testing that we don't fully understand yet, and it makes passing them difficult even though we should pass. Everyone immediately says (a). But I (and the OP) say -- could be (b). Now we have to be careful. We have to come up with a way to test and falsify (b), or this is just a meaningless defense that you could always invoke no matter what, and that is not my goal.

Right. And in fact I don't think (b) is a mere speculation---all you have to do is introspect a little bit on how you listen to music and you'll find many problems. As I mentioned a few posts up, one very basic and obvious problem is that people tend to notice different things about the music with each repetition. In fact, a lot of music is designed to cause that, because it is too rich to take everything in.

Quote:

This does not happen in commercial taste testing (although I understand it has happened with wine). When people say they generally prefer Pepsi to Coke (or vice versa) in their daily lives, they are very able to tell them apart in a blind test. Ditto for some ice cream brands. Why not audio?

One of my observations is that music listening seems to be more an activity than a passive reception of sensory impressions. This makes it distinct from food testing (although some chefs might say that food fires up their imaginations and souls).

Good luck with your plan. Let us know the results.

 

[saintalfonzo]

The only difficulty in blind testing is (for some people) accepting the results. People who cannot justify having spent a lot of money on something by being able to accurately identify the difference in a blind test will simply come up with infinite excuses as to why the test is faulty. Denial is a way of life for some.

 

[haloxt]

What happens when people try listening critically for cable differences while under a time constraint or other similar circumstance is they strain in an attempt to concentrate and often suffer fatigue and headaches and their accompanying reduction in mental health. They are then completely unreliable for cable tests. The fact of the matter is no cable DBT should be taken seriously unless they take into account the detrimental effect of nervousness on the senses and mental function. This also relates to medical placebo but not in the way anti-cablers repeat redundantly: when people think they're getting medicine but it's just sugar pills they can relax and that is what improves their physical and mental conditions. Maybe we should give anti-cablers sugar pills so they stop being so irrational here on head-fi?

Quoting a paragraph by Dr. William H. Bates, "The idea that the attention can be forced is a very common one and is very bad for the eyes. It is greatly encouraged by popular writers, but is contrary to the teachings of more reliable psychologists who know that forced attention can only be momentary, and that it is a great strain upon the mind and the whole body. Ladd records that the subject of an experiment to determine reaction-time under concentrated attention often "though sitting quiet, sweats profusely."

 

[ILikeMusic]

Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif This makes it sound like you are not actually interested in understanding perception of music, but just "defusing an objection." In fact it's a bit duplicitous to act like "take your time" is a valid answer, then later reveal you think fast switching is most accurate.

Not at all. I didn't 'reveal' anything because it's no a secret that fast switching is easier for most people, and my opinion whether fast or slow is better is unimportant anyway. I was only trying to point out that fast sampling isn't a valid reason to dismiss DBT (as we see happening once again in the preceding post) . Quick samples, slow samples, whatever... the bottom line does not change.

 

[nick_charles]

Quote:

Originally Posted by wavoman /img/forum/go_quote.gif (b) There is something about blind testing that we don't fully understand yet, and it makes passing them difficult even though we should pass.

I have done many blind tests between components and on the effect of filters, I have even detected the difference between a 24/96 file and a 16/44.1 file (admittedly from a small, and I do mean small, artifact in one in the first 90ms and not in the other) and have been able to identify differences *if the stimuli are different enough*

NB many of these positives were done with a TBAAM a far from boutique item.


, and others have done positive DBTs with codecs such as ADD here who correctly identified 320K mp3 vs lossless, a decidedly non trivial task, other blind tests have shown people capable of detecting stimulus differences of 0.1db or very small amounts of distortion. In the forum I pointed to in a different thread one listener correctly and consistently identified 10ns of jitter. Blind tests can be very sensitive if carried out properly. There are several positive DBTs on the web for amps and cd players.

Obviously folks abilities do differ, this is why you use loads and loads of subjects(Meyer and Moran, Blech and Yang) . In another famous blind test 4 listeners (out of 110 admittedly) were capable of telling the difference between DSD and high res PCM both of which are capable of incredible accuracy, low noise and low distortion.


Good luck with your cable blind tests, if you do detect differences consistently please also measure the differences between the cables.

 

[aristos_achaion]

Quote:

Originally Posted by saintalfonzo /img/forum/go_quote.gif The only difficulty in blind testing is (for some people) accepting the results.

Likely a bigger problem is knowing what the results mean. Even if, in a blind trial, you get a statistically significant result one way or another, you have to know how to interpret the results. For instance, if I test a suitably large group of the general populace and a statistically significant number are unable to to tell a good cable from a stock one, what can I conclude about audiophiles? Absolutely nothing; I've concluded that the general populace doesn't know good audio equipment from bad, which we already know...that's why Skullcandies exist. To get any interesting results, we'd have to test a significant number of audiophiles who believe in cables. Notice I say "statistically significant"...just testing one or two people would tell us nothing.

It seems kind of silly to me to either dismiss on principle a known & respected scientific technique or to worship it without knowing how to critically examine the techniques and results. Ironic that the DBT is building up so much prejudice around here, given that's exactly what the DBT is designed to eliminate.

 

[mike1127]

Quote:

Originally Posted by aristos_achaion /img/forum/go_quote.gif Likely a bigger problem is knowing what the results mean. Even if, in a blind trial, you get a statistically significant result one way or another, you have to know how to interpret the results.

Exactly. I accept the results. I don't think they are easy to interpret.

Also, I don't think that the published DBTs were able to control the subject's use of their attention, or get them to listen to music as music and not sound. So what appears on the outside to be a "controlled test" is the furthest thing from it.

 

[nick_charles]

Quote:

Originally Posted by mike1127 /img/forum/go_quote.gif Also, I don't think that the published DBTs were able to control the subject's use of their attention, or get them to listen to music as music and not sound. So what appears on the outside to be a "controlled test" is the furthest thing from it.

When you have tested over 100 subjects and they all show the same result it does not really matter what strategy different individuals were using in fact telling them precisely how to do it may be counterproductive since each person may have their own strategy that works best for them. I for instance will go over the same segment again and again and again trying to detect a slight artifact.

Where the differences are big enough some people will hear them , where they are not big enough nobody will. I have several DBTs with positive results and several with negative results, the difference between positive and negative is always the magnitude of the differences not my atttude or belief. Sometimes this can be as little as a 90ms blip in an intro.

For instance I did not think I would be able to DBT my CD players yet I could do so consistently, why , because there are substanial measurable differences at frequencies and levels that actually mattered. I could put a cliff face low pass flter on samples at 17K and never in a miillion years hear the difference but at 9K or 13K I can detect the difference.

I love DBTs I do not get overly stressed by them , I think they are fun, they are equally interesting whether they are negative or positive because they tell me something about my capabilities. Everyone who professes an interest in high fidelity should be required to do blind tests at the very least to challenge their assumptions.

 

[mike1127]

Quote:

Originally Posted by nick_charles /img/forum/go_quote.gif When you have tested over 100 subjects and they all show the same result it does not really matter what strategy different individuals were using in fact telling them precisely how to do it may be counterproductive since each person may have their own strategy that works best for them. I for instance will go over the same segment again and again and again trying to detect a slight artifact.

I'll back-pedal a little and say that even though a test cannot directly control the use of the attention, it certainly can influence it. For example, a quick-switching protocol, or one that uses segments like you describe, has a very strong influence on how someone uses their attention. See my post earlier in this thread for my explanation why this can block access to information.

If 100 subjects (who are all using their attention in their own way) can't detect something, it could mean:

1. No one can detect it.
2. The protocol itself makes the subjects insensitive to the key differences.
3. It can be detected, but only by an unusual use of the attention.

Remember that some people spend much more time introspecting on how they use their attention. Being a musician and constantly working with small differences in sound could influence this. So could being an instrument-builder. Someone who practices mindfulness meditation would be influenced.

It comes down to this: most audiophiles get the impression of differences between something controversial (like cables) in either initial impressions, or long-term living-with-the-component. Very, very few DBTs replicate these conditions.

EDIT: you are clearly listening to sound. I listen for music. That right there is a huge difference in how we use our attention.