Designing an experiment to detect differences between cables
Apr 16, 2004 at 9:21 PM Thread Starter Post #1 of 93

Orpheus

Headphoneus Supremus
Joined
Aug 17, 2002
Posts
3,126
Likes
20
the question: "Can cables audibly affect the tonality of your system?"

as we all know, our hearing is physically capable of sensing frequencies between 20hz-20khz. as we grow older, this range decreases. there are measurable differences between cables in respect to "tonality" but most of this exists much beyond our hearing capabilities. there is little measurable difference among cables within our hearing range. yet, people believe that cables can make one's system sound "brighter" or "duller", etc.. there have been reports of better spatial clarity ("soundstage") and other benefits as well. so, besides the well know fact that various cable construncts reject ambient noise better, can they affect other areas of sound?--what i call "tonality."

the goal is to devise a test that both believers and non-believers agree on. then we will execute this test and report the results.

so, let's do it.

post your ideas here.
 
Apr 16, 2004 at 10:11 PM Post #2 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Posts
7,822
Likes
35
There are two kinds of error that you're going to have to control for.

Type I error: You say that an effect is real when in fact it is not. This is where alpha= 0.05 and all that comes into in the statistical analysis etc. This is where DBT shines. It's particularly good at not revealing false positives.

Type II error: You say that an effect is not real when in fact it is. This is where DBT has fallen on its face through poor design. It produces a lot of false negative results through failure to consider Type II error. The statistic that is used is referred to as beta, and also has to be less than 0.05 for a result to be considered meaningful. However, calculating beta is a trick, because you need information on variance for a real difference to compute the N needed to even run the experiment with enough people for a negative result to be meaningful. It gets very circular here.

Coming back from that digression and back to the experiment:

Step 1: Define an experimental hypothesis. Cables sound the same. In statistics, we'd call this the null hypothesis. That part's the easiest.

Step 2: Define our independent and dependent variables. Independent variables is easy. Cables. OK, that's out of the way for a bit, but we'll come back to it soon enough in deciding how we want to expose people to the different levels of the independent variable.

Now we need a dependent variable. Exactly what are we going to measure?
 
Apr 17, 2004 at 1:20 AM Post #3 of 93

daycart1

Ceremonial old guy, bifocals and all.All the philosophy he can muster can't solve the mysterious double-post.
Joined
Mar 28, 2002
Posts
4,710
Likes
12
Quote:

Originally posted by Hirsch
Now we need a dependent variable. Exactly what are we going to measure?


OK, I'll bite. The subject judges whether two samples are the same kind of cable or a different kind. (Since one kind of cable might differ from one sample to the next, that should be controlled for as well). So the null hypothesis is that subjects can not discriminate same cable pairs from different cable pairs.

This seems to be the most basic test.
smily_headphones1.gif
 
Apr 17, 2004 at 1:59 AM Post #4 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Posts
7,822
Likes
35
Quote:

Originally posted by daycart1
OK, I'll bite. The subject judges whether two samples are the same kind of cable or a different kind. (Since one kind of cable might differ from one sample to the next, that should be controlled for as well). So the null hypothesis is that subjects can not discriminate same cable pairs from different cable pairs.

This seems to be the most basic test.
smily_headphones1.gif


OK. Our procedure so far is that we let the subject listen to two different sets of cables. The subject then has to make a judgement of "same" or "different". We'll go with a given that the subject cannot be aware of which cable is which. We'll also throw in some dummy trials where there really is no difference.

Now, we can start putting an experiment together. Taking it one step at a time... How do we get subjects? How do we determine how many subjects we need?
 
Apr 17, 2004 at 5:44 AM Post #5 of 93

ServinginEcuador

Founder of the Head-Fi Pay-to-Post Program.
Joined
Sep 1, 2002
Posts
8,384
Likes
14
Dean,

If you want to see for yourself wait until June 19th when we have some really good cables on hand. I have had a few that didn't change much, and one or two that did.
 
Apr 17, 2004 at 6:07 AM Post #6 of 93

Orpheus

Headphoneus Supremus
Joined
Aug 17, 2002
Posts
3,126
Likes
20
Quote:

If you want to see for yourself wait until June 19th when we have some really good cables on hand. I have had a few that didn't change much, and one or two that did.


well, the point is to devise an experiment to sorta provide some answers. it's no secret i don't "believe" in cables. not the way some here do anyway. at the last meet there were plenty of really expensive cables too. and i've played around with some also in my own personal system. i even have the hd600 cardas cable. i have never heard any differences. ever.

but right now, i am looking for an objective test. i am willing to put aside my own beliefs for this experiment.

but i should note, i do believe in quality. i make my own cables, and i believe they perform better than what i can buy on the market. i also sell cables as some of you know. and i believe in the performance of what i make--not in this "tonality" grey area though, but in the measurable performance.
 
Apr 17, 2004 at 7:21 AM Post #8 of 93

Wodgy

Headphoneus Supremus
Joined
Feb 24, 2002
Posts
4,657
Likes
12
If you're going to do the test, you should use long cables, perhaps 30 feet, and certainly no less than 20 feet. If you use shorter cables, it will be easier for opponents to claim that the experiment is biased towards the hypothesis that cables make no difference.
 
Apr 17, 2004 at 7:27 AM Post #9 of 93
Joined
Mar 28, 2003
Posts
9,530
Likes
81
Location
Los Angeles
Wodgy, considering that most people use shorter cables in their systems, what would be the point of using such long cables, since it would not be realistic? If cables on make a distance when they are of that length, then why bother?
 
Apr 17, 2004 at 7:46 AM Post #10 of 93

Orpheus

Headphoneus Supremus
Joined
Aug 17, 2002
Posts
3,126
Likes
20
Quote:

If you're going to do the test, you should use long cables, perhaps 30 feet, and certainly no less than 20 feet. If you use shorter cables, it will be easier for opponents to claim that the experiment is biased towards the hypothesis that cables make no difference.


i disagree. almost no home system uses such long cables. yet people still claim to hear the differences, even among 1/2m cables! even if we use 1m cables, and people argue about the results, we can still conclude whether 1m cables make a difference or not, as that's what was used. you know what i mean?
 
Apr 17, 2004 at 10:34 AM Post #13 of 93

Wodgy

Headphoneus Supremus
Joined
Feb 24, 2002
Posts
4,657
Likes
12
Quote:

Originally posted by Orpheus
i disagree. almost no home system uses such long cables. yet people still claim to hear the differences, even among 1/2m cables! even if we use 1m cables, and people argue about the results, we can still conclude whether 1m cables make a difference or not, as that's what was used. you know what i mean?


I know what you mean, but that's not how to design a proper scientific experiment. When you have a hypothesis, you need to choose experimental conditions that give the least possible benefit of the doubt towards the hypothesis.

For instance, when trying to prove that an algorithm I've developed is faster than something someone else developed, I cannot benchmark a good implementation of my algorithm against a mediocre implementation of theirs. The peer reviewers hate that. It's fairer to choose an good implementation of mine, and the best possible implementation of theirs. Only then is it reasonable to be able to conclude that mine is faster.

In your case, the hypothesis is that cables make no difference. If you bias the experiment towards your hypothesis by choosing short cables, your experimental design is flawed. Especially because other variables you cannot control are already biased toward your hypothesis (e.g. the test is necessarily of a much shorter duration than realistic listening periods (where it could be argued that people might be able to learn to discern differences), and the test will inevitably use music some participants are not intimately familiar with, etc.). With such variables that you cannot reasonably control being biased towards the hypothesis, you really need to bias cable length and other variables against it.

Or, to put it more succinctly: the skeptic will say that you were scared that using longer cables would magnify any potential differences, so you picked shorter cables.
 
Apr 17, 2004 at 3:28 PM Post #14 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Posts
7,822
Likes
35
I stuck a post in the other thread, indicating that I would be moderating this one closely. All positive contributions welcome, but if it starts going in ridiculous tangents that don't address the question, I'll get those posts out of here so we can actually hold a real discussion. Agreement is not necessary, but politeness is. (Hope you don't mind me doing this, Orpheus).

The sampling question is a loaded one. In a real sense, the sample need only be one. If one person can be shown to reliable detect cable differences in a properly designed setup, then there is no longer any question as to whether or not the cables are different. The question shifts, and becomes "why do some people hear cable differences, while others aren't sensitive to them". However, any given person may or may not be sensitive to the differences. So, we need to run a group of people, and need to determine who they are. There are some forms of experimental bias that we don't really have to worry about. Probably our best subject pool would be people that already claim that they can detect differences. Since they self-report that they already have the skill that we're looking for, it would save a lot of auditory testing. Now we have to figure out how many we need.

WRT cable length, IMO we should be using cables of the length that people typically use in their systems. It does no good to show that longer cables, which are in fact magnifiers, may show a difference when those are for the most part not the ones that people are paying big bucks for (well, they would but it would be even bigger bucks).

So, I think we refine the question a bit, and rephrase it something like "Do the cables that people use in their home systems make a difference?"

Note that by doing this, we're biasing the experiment a bit in favor of the cable skeptics, by reducing the size of any effects of cables by putting a limit on their length. There's less of a chance of getting a positive result. OTOH, we'd also be getting results that more people can generalize to their home systems, which may be worth more in the long run (oops. just saw the pun
tongue.gif
)
 
Apr 17, 2004 at 3:36 PM Post #15 of 93

Hirsch

Why is there a chaplain standing over his wallet?
Joined
Aug 12, 2001
Posts
7,822
Likes
35
Quote:

Originally posted by Edwood
Woooo. Are we going to do forbidden......DBT?!?!



DBT is only forbidden in the Cable Forum, so it's legal here. OTOH, as I've indicated, this thread is going to be moderated closely so that things don't get out of hand.
 

Users who are viewing this thread

Top