Designing an experiment to detect differences between cables
post-769265
Thread Starter
Post #1 of 93

Orpheus

Headphoneus Supremus
Joined
Aug 17, 2002
Messages
3,126
Reaction score
15
Joined
Aug 17, 2002
Posts
3,126
Likes
15
the question: "Can cables audibly affect the tonality of your system?"

as we all know, our hearing is physically capable of sensing frequencies between 20hz-20khz. as we grow older, this range decreases. there are measurable differences between cables in respect to "tonality" but most of this exists much beyond our hearing capabilities. there is little measurable difference among cables within our hearing range. yet, people believe that cables can make one's system sound "brighter" or "duller", etc.. there have been reports of better spatial clarity ("soundstage") and other benefits as well. so, besides the well know fact that various cable construncts reject ambient noise better, can they affect other areas of sound?--what i call "tonality."

the goal is to devise a test that both believers and non-believers agree on. then we will execute this test and report the results.

so, let's do it.

post your ideas here.
 
     Share This Post       
post-769361
Post #2 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Messages
7,820
Reaction score
23
Joined
Aug 12, 2001
Posts
7,820
Likes
23
There are two kinds of error that you're going to have to control for.

Type I error: You say that an effect is real when in fact it is not. This is where alpha= 0.05 and all that comes into in the statistical analysis etc. This is where DBT shines. It's particularly good at not revealing false positives.

Type II error: You say that an effect is not real when in fact it is. This is where DBT has fallen on its face through poor design. It produces a lot of false negative results through failure to consider Type II error. The statistic that is used is referred to as beta, and also has to be less than 0.05 for a result to be considered meaningful. However, calculating beta is a trick, because you need information on variance for a real difference to compute the N needed to even run the experiment with enough people for a negative result to be meaningful. It gets very circular here.

Coming back from that digression and back to the experiment:

Step 1: Define an experimental hypothesis. Cables sound the same. In statistics, we'd call this the null hypothesis. That part's the easiest.

Step 2: Define our independent and dependent variables. Independent variables is easy. Cables. OK, that's out of the way for a bit, but we'll come back to it soon enough in deciding how we want to expose people to the different levels of the independent variable.

Now we need a dependent variable. Exactly what are we going to measure?
 
     Share This Post       
post-769649
Post #3 of 93

daycart1

Ceremonial old guy, bifocals and all.All the philosophy he can muster can't solve the mysterious double-post.
Joined
Mar 28, 2002
Messages
4,710
Reaction score
12
Joined
Mar 28, 2002
Posts
4,710
Likes
12
Quote:

Originally posted by Hirsch
Now we need a dependent variable. Exactly what are we going to measure?


OK, I'll bite. The subject judges whether two samples are the same kind of cable or a different kind. (Since one kind of cable might differ from one sample to the next, that should be controlled for as well). So the null hypothesis is that subjects can not discriminate same cable pairs from different cable pairs.

This seems to be the most basic test.
 
     Share This Post       
post-769700
Post #4 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Messages
7,820
Reaction score
23
Joined
Aug 12, 2001
Posts
7,820
Likes
23
Quote:

Originally posted by daycart1
OK, I'll bite. The subject judges whether two samples are the same kind of cable or a different kind. (Since one kind of cable might differ from one sample to the next, that should be controlled for as well). So the null hypothesis is that subjects can not discriminate same cable pairs from different cable pairs.

This seems to be the most basic test.


OK. Our procedure so far is that we let the subject listen to two different sets of cables. The subject then has to make a judgement of "same" or "different". We'll go with a given that the subject cannot be aware of which cable is which. We'll also throw in some dummy trials where there really is no difference.

Now, we can start putting an experiment together. Taking it one step at a time... How do we get subjects? How do we determine how many subjects we need?
 
     Share This Post       
post-769962
Post #5 of 93

ServinginEcuador

Founder of the Head-Fi Pay-to-Post Program.
Joined
Sep 1, 2002
Messages
8,384
Reaction score
14
Joined
Sep 1, 2002
Posts
8,384
Likes
14
Dean,

If you want to see for yourself wait until June 19th when we have some really good cables on hand. I have had a few that didn't change much, and one or two that did.
 
     Share This Post       
post-769985
Post #6 of 93

Orpheus

Headphoneus Supremus
Joined
Aug 17, 2002
Messages
3,126
Reaction score
15
Joined
Aug 17, 2002
Posts
3,126
Likes
15
Quote:

If you want to see for yourself wait until June 19th when we have some really good cables on hand. I have had a few that didn't change much, and one or two that did.


well, the point is to devise an experiment to sorta provide some answers. it's no secret i don't "believe" in cables. not the way some here do anyway. at the last meet there were plenty of really expensive cables too. and i've played around with some also in my own personal system. i even have the hd600 cardas cable. i have never heard any differences. ever.

but right now, i am looking for an objective test. i am willing to put aside my own beliefs for this experiment.

but i should note, i do believe in quality. i make my own cables, and i believe they perform better than what i can buy on the market. i also sell cables as some of you know. and i believe in the performance of what i make--not in this "tonality" grey area though, but in the measurable performance.
 
     Share This Post       
post-770017
Post #7 of 93

tortie

Headphoneus Supremus
Joined
Aug 12, 2003
Messages
2,812
Reaction score
13
Joined
Aug 12, 2003
Posts
2,812
Likes
13
I hope this test will actually be done. Im sure a lot of people are interested in the test results.
 
     Share This Post       
post-770031
Post #8 of 93

Wodgy

Headphoneus Supremus
Joined
Feb 24, 2002
Messages
4,657
Reaction score
12
Joined
Feb 24, 2002
Posts
4,657
Likes
12
If you're going to do the test, you should use long cables, perhaps 30 feet, and certainly no less than 20 feet. If you use shorter cables, it will be easier for opponents to claim that the experiment is biased towards the hypothesis that cables make no difference.
 
     Share This Post       
post-770033
Post #9 of 93

Iron_Dreamer

Member of the Trade: HeadAmp
Landscape-Photo-Fi
Organizer for Can Jam '09
Joined
Mar 28, 2003
Messages
9,527
Reaction score
74
Joined
Mar 28, 2003
Posts
9,527
Likes
74
Wodgy, considering that most people use shorter cables in their systems, what would be the point of using such long cables, since it would not be realistic? If cables on make a distance when they are of that length, then why bother?
 
     Share This Post       
post-770054
Post #10 of 93

Orpheus

Headphoneus Supremus
Joined
Aug 17, 2002
Messages
3,126
Reaction score
15
Joined
Aug 17, 2002
Posts
3,126
Likes
15
Quote:

If you're going to do the test, you should use long cables, perhaps 30 feet, and certainly no less than 20 feet. If you use shorter cables, it will be easier for opponents to claim that the experiment is biased towards the hypothesis that cables make no difference.


i disagree. almost no home system uses such long cables. yet people still claim to hear the differences, even among 1/2m cables! even if we use 1m cables, and people argue about the results, we can still conclude whether 1m cables make a difference or not, as that's what was used. you know what i mean?
 
     Share This Post       
post-770117
Post #11 of 93

Edwood

1/2 hamster, 1/2 Turkish∙ Blueteething
Joined
Nov 11, 2003
Messages
20,230
Reaction score
225
Location
Cage
Joined
Nov 11, 2003
Location
Cage
Posts
20,230
Likes
225
I use much shorter cables.

9 Feet.




Woooo. Are we going to do forbidden......DBT?!?!



Well, we are in the member's lounge.

-Ed
 
     Share This Post       
post-770135
Post #12 of 93

Orpheus

Headphoneus Supremus
Joined
Aug 17, 2002
Messages
3,126
Reaction score
15
Joined
Aug 17, 2002
Posts
3,126
Likes
15
so, let's get this rolling.

Hirsch, i think you have the most experience--and i know you know a lot about sampling. you tried this once, and you know what wasn't so great, and what would be better. what would you suggest?
 
     Share This Post       
post-770145
Post #13 of 93

Wodgy

Headphoneus Supremus
Joined
Feb 24, 2002
Messages
4,657
Reaction score
12
Joined
Feb 24, 2002
Posts
4,657
Likes
12
Quote:

Originally posted by Orpheus
i disagree. almost no home system uses such long cables. yet people still claim to hear the differences, even among 1/2m cables! even if we use 1m cables, and people argue about the results, we can still conclude whether 1m cables make a difference or not, as that's what was used. you know what i mean?


I know what you mean, but that's not how to design a proper scientific experiment. When you have a hypothesis, you need to choose experimental conditions that give the least possible benefit of the doubt towards the hypothesis.

For instance, when trying to prove that an algorithm I've developed is faster than something someone else developed, I cannot benchmark a good implementation of my algorithm against a mediocre implementation of theirs. The peer reviewers hate that. It's fairer to choose an good implementation of mine, and the best possible implementation of theirs. Only then is it reasonable to be able to conclude that mine is faster.

In your case, the hypothesis is that cables make no difference. If you bias the experiment towards your hypothesis by choosing short cables, your experimental design is flawed. Especially because other variables you cannot control are already biased toward your hypothesis (e.g. the test is necessarily of a much shorter duration than realistic listening periods (where it could be argued that people might be able to learn to discern differences), and the test will inevitably use music some participants are not intimately familiar with, etc.). With such variables that you cannot reasonably control being biased towards the hypothesis, you really need to bias cable length and other variables against it.

Or, to put it more succinctly: the skeptic will say that you were scared that using longer cables would magnify any potential differences, so you picked shorter cables.
 
     Share This Post       
post-770299
Post #14 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Messages
7,820
Reaction score
23
Joined
Aug 12, 2001
Posts
7,820
Likes
23
I stuck a post in the other thread, indicating that I would be moderating this one closely. All positive contributions welcome, but if it starts going in ridiculous tangents that don't address the question, I'll get those posts out of here so we can actually hold a real discussion. Agreement is not necessary, but politeness is. (Hope you don't mind me doing this, Orpheus).

The sampling question is a loaded one. In a real sense, the sample need only be one. If one person can be shown to reliable detect cable differences in a properly designed setup, then there is no longer any question as to whether or not the cables are different. The question shifts, and becomes "why do some people hear cable differences, while others aren't sensitive to them". However, any given person may or may not be sensitive to the differences. So, we need to run a group of people, and need to determine who they are. There are some forms of experimental bias that we don't really have to worry about. Probably our best subject pool would be people that already claim that they can detect differences. Since they self-report that they already have the skill that we're looking for, it would save a lot of auditory testing. Now we have to figure out how many we need.

WRT cable length, IMO we should be using cables of the length that people typically use in their systems. It does no good to show that longer cables, which are in fact magnifiers, may show a difference when those are for the most part not the ones that people are paying big bucks for (well, they would but it would be even bigger bucks).

So, I think we refine the question a bit, and rephrase it something like "Do the cables that people use in their home systems make a difference?"

Note that by doing this, we're biasing the experiment a bit in favor of the cable skeptics, by reducing the size of any effects of cables by putting a limit on their length. There's less of a chance of getting a positive result. OTOH, we'd also be getting results that more people can generalize to their home systems, which may be worth more in the long run (oops. just saw the pun
)
 
     Share This Post       
post-770306
Post #15 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Messages
7,820
Reaction score
23
Joined
Aug 12, 2001
Posts
7,820
Likes
23
Quote:

Originally posted by Edwood
Woooo. Are we going to do forbidden......DBT?!?!



DBT is only forbidden in the Cable Forum, so it's legal here. OTOH, as I've indicated, this thread is going to be moderated closely so that things don't get out of hand.
 
     Share This Post       

Users Who Are Viewing This Thread (Users: 0, Guests: 1)

Top