or Connect
Head-Fi.org › Forums › Equipment Forums › Sound Science › Audiophile cables, an interesting question.
New Posts  All Forums:Forum Nav:

# Audiophile cables, an interesting question. - Page 41

Quote:
Originally Posted by cjl

Also, as an interesting note, you don't need to score close to 100% to show statistical significance. An ABX or A/B test can have a positive outcome even if you only get the correct answer 51% of the time. The closer your outcome is to 50% though, the more trials are needed to show that the result is unlikely to be purely chance driven. In your case, the sum of your last three trials is 19/30. While this is actually somewhat unlikely from chance alone, it isn't nearly unlikely enough to be considered statistically significant (if I did the math right, the chances of getting at least 19/30 correct by pure chance are around 14%). If you continue to get ~63% correct through many more trials (which is not an expected outcome based on the current data, I should add), the probability of a 63% success rate occurring by chance alone drops below 5% once you have successfully achieved 32/50 correct (so a 32/50 would be considered a possible positive result for an ABX test). If you tighten up the required P value to 1% (since 5% still means that by chance alone, 1 in 20 random trials will achieve that kind of significance), you would need a score of 62/98 to show a significant result.

Thank you cjl. From memory alone, 19 out of 30 trials is unlikely to be significant. But you are right, there is a statistical test that is based on the total number of samples also. Could you point me to a webpage that has the mathematics on this? I'd like to regain a better understanding of this myself.

Quote:
Originally Posted by esldude

I commend you x838nwy for doing this.  You are experiencing what at least many of us have.  You just know, feel it in your gut, they aren't the same.  Then with a blind test, well you feel differences are small, but you just know they are there.  Yet the test results indicate there is no audible difference likely.

I have tried to conceive of a test like the two red colored tiles you speak about.  My first idea was separate cables on each channel listening over headphones.  But that doesn't work.  When differences I artificially induce in one channel become quite perceptible in such a concurrent test they are already larger differences than I detect with an ABX in Foobar.

Also, though you have the subjective experience of seeing two tiles 'at the same time side by side' your eyes actually switch rapidly from one to the other, and do some trickery so you think you are looking at both.  Using software that could instantly switch from one color to other on my computer screen I found I could more easily pick two very close colors than if I had them side by side.  The key is instant (in my case 1/60th of a second) switching.  Putting in a dead time of 5 seconds lowered the sensitivity of color discrimination for me.

Finally, the logic of using music for your test seems good.  It has been known for some time smaller differences can be detected with proper test signals in blind testing.  For your cable test I would suggest pink noise.  At not too high a level mind you.  This way you have a constancy of signal character, can rapidly switch between and make your choices.  Just a few seconds before switching makes the test less time consuming to do a fair number of trials.  I find it less tiresome in practice than using music myself.  This is if you wish to give it one more go.

It takes time to turn loose of one idea and get comfortable with another one. You had the guts to actually try this out.  I have found few people willing to do such a thing.  Most just bluster about how obvious it is and never quite commit themselves to testing the proposition cables sound different.  Or attack the methodology if they do and get results they don't like.

As you get comfortable with the idea cables are generally transparent and fully interchangeable it is somewhat freeing.  You can use whatever inexpensive cable is at hand with no worries.  It is a part of the audio world that is cut and dried.  Leaves more money and time to focus on things that really matter.

The switching concept is a good one and I'm interested as I think it will make for a fairer test. I have not tried playing pink noise through it either, so it'd be interesting if I could have the system play the same tone through A, say then through A or B and I have to say if the second playback is the same as the first. It'd probably no adhere too closely to a definition of a DBX...

Quote:
Originally Posted by bigshot

If you go to that much trouble and still can't discern a definite improvement, I think it's time to chalk it up to not mattering and move on to frying bigger fish. The whole point is to improve the sound of your stereo system, not to become a testing engineer.

Well, my conclusion is pretty much what I've highlight in bold type. It's been kindda fun and interesting to actually put stuff to the test, but it at least gives me a better sense of proportion to plan my system's direction. I think I've been as fair as I could have been - in fact even biased in expecting a difference, so I think I gave the cables concerned a decent chance. I hope I haven't put Nordost or their Blue Heaven cables in the spotlight and that is not my intention. They're simply the most costly interconnects I own, are at the right length and of the right sort of plugs for testing  against my \$7 wonder-cable.

I have to say one last thing though - all my tests have been with analogue interconnects. I am not claiming that the same will or will not apply to other types of interconnects, tonearm wires, speaker cables or power cords. And while I currently have a highly modified opinion on interconnects (in comparison to the opinion I held prior to the tests), I remain open to reasonable suggestions on ways to improve my tests or other approaches.

With things like this, there remains always a "garage fairy" thing - if someone were to say that in garages exist fairies and I want to prove that they are fiction, looking in my garage for fairies every couple of hours and not finding any does not constitute the proof I am looking for as my methods may be incorrect or my knowledge/skills insufficient. The difficulties in proving negatives is a challenge. But for this one, I think I am comfortable with what I found. I shall not be spending unreasonable sums on money on my analogue interconnects.

p.s. There IS a possibility that I have stumbled upon the most amazing \$7 cables in the history of hifi, being a gnat's hair's width from a Blue Heaven. But that's highly, highly unlikely.

Have a great week everybody :)

The formula in simple form is N/2 + square root of N  where N is the number of trials.

So for 30 trials 21 would reach 95% confidence level.

Results required for a 95% confidence level:

Number of trials 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Minimum number correct 9 9 10 10 11 12 12 13 13 14 15 15 16 16 17

18

http://lacinato.com/cm/software/othersoft/abx

This is a piece of ABX software I haven't noticed before.  Works on Mac, Linux or Windows.  So I haven't used it, but it looks simple and useful if you don't have Foobar available.

Edited by esldude - 5/4/14 at 8:00pm
Quote:
Originally Posted by bigshot

If you go to that much trouble and still can't discern a definite improvement, I think it's time to chalk it up to not mattering and move on to frying bigger fish. The whole point is to improve the sound of your stereo system, not to become a testing engineer.

While I agree with this from the viewpoint of trying to set up an audio system, a positive result of any kind with an A/B or ABX test using cables would still be an astonishing thing (and very relevant to the sound science section). This is still true even if the difference is very subtle, and the tester only gets ~60% correct (but has sufficient trials to show significance). There is no known mechanism (or at least no mechanism with which I am familiar) that would cause two 1.5m cables of reasonable gauge (and without any obvious sound-altering properties) to sound audibly different to human perception, so that would still be an amazing result.

Quote:
Originally Posted by x838nwy

Thank you cjl. From memory alone, 19 out of 30 trials is unlikely to be significant. But you are right, there is a statistical test that is based on the total number of samples also. Could you point me to a webpage that has the mathematics on this? I'd like to regain a better understanding of this myself.

I was getting it out of my college statistics textbook, so I can't give you the direct link unfortunately. However, you can get some understanding from the following pages (mostly wikipedia, but that isn't necessarily a bad thing for basic info):

http://en.wikipedia.org/wiki/Statistical_significance <-- Basic overview of statistical significance

http://en.wikipedia.org/wiki/One-_and_two-tailed_tests <--- for the purposes of this article, an ABX test is a one-tailed test

http://en.wikipedia.org/wiki/Bernoulli_trial

http://en.wikipedia.org/wiki/Checking_whether_a_coin_is_fair <--- this one is pretty good at explaining the statistics, and although it is using a different application for the statistics, the math is similar

http://home.provide.net/~djcarlst/abx_bino.htm <--- Here's a table showing probabilities of obtaining various results in an ABX via random chance (it only goes up to 20 trials though). It does however show nicely how the probabilities scale, and how, for example, 16/20 would actually be a substantially better result for an ABX than 7/7.

(Sorry I can't go into full detail right now - it's late and I should be getting to bed. If I have time tomorrow, I might get around to typing up a more complete explanation of the math though...)

Edited by cjl - 5/4/14 at 10:50pm
Quote:
Originally Posted by cjl

I was getting it out of my college statistics textbook, so I can't give you the direct link unfortunately. However, you can get some understanding from the following pages (mostly wikipedia, but that isn't necessarily a bad thing for basic info):

http://en.wikipedia.org/wiki/Statistical_significance <-- Basic overview of statistical significance

http://en.wikipedia.org/wiki/One-_and_two-tailed_tests <--- for the purposes of this article, an ABX test is a one-tailed test

http://en.wikipedia.org/wiki/Bernoulli_trial

http://en.wikipedia.org/wiki/Checking_whether_a_coin_is_fair <--- this one is pretty good at explaining the statistics, and although it is using a different application for the statistics, the math is similar

http://home.provide.net/~djcarlst/abx_bino.htm <--- Here's a table showing probabilities of obtaining various results in an ABX via random chance (it only goes up to 20 trials though). It does however show nicely how the probabilities scale, and how, for example, 16/20 would actually be a substantially better result for an ABX than 7/7.

(Sorry I can't go into full detail right now - it's late and I should be getting to bed. If I have time tomorrow, I might get around to typing up a more complete explanation of the math though...)

Thanks cjl!!

Back when I was at university, I avoided statistics like a plague. A advice from me, GET A GOOD HANDLE ON STATISTICS specially if you're going into engineering or management. Even the little I know has given me quite an edge on a few occasions. Will take a quick read and see.

Have there been any audio industry whistle blowers?
Quote:
Originally Posted by Agharta

Have there been any audio industry whistle blowers?

How do you mean?

se
I mean has anyone exposed inside scams etc?
Quote:
Originally Posted by Agharta

I mean has anyone exposed inside scams etc?

Oh. Yes. But being a Member of the Trade, I'm afraid I can't talk about them here. Sorry.

se
I would love to read about such things, if someone could meet me under the bridge at midnight. Wear a USB cable round your neck so I can identify you.
Quote:
Originally Posted by Agharta

I would love to read about such things, if someone could meet me under the bridge at midnight. Wear a USB cable round your neck so I can identify you.

se
Quote:
Originally Posted by Agharta

I would love to read about such things, if someone could meet me under the bridge at midnight. Wear a USB cable round your neck so I can identify you.

That's a bad idea to wear a USB cable! someone might hear you!

Cheers

Quote:
Originally Posted by ab initio

That's a bad idea to wear a USB cable! someone might hear you!

Not if it's a Ninja USB cable.

se
Quote:
Originally Posted by bigshot

If you go to that much trouble and still can't discern a definite improvement, I think it's time to chalk it up to not mattering and move on to frying bigger fish. The whole point is to improve the sound of your stereo system, not to become a testing engineer.

Well Bigshot, it's finally happened: you've posted something on this topic that I agree with 100% .

Back to the results:  As others have said, these are inconclusive. At around 60% detection, the sample would need to be much bigger to be statistically valid. Looks like that's something that x838nwy is not prepared to pursue further - and who can blame him!

It's important to not cherry pick the results to suit ones biases. Just as x838nwy shouldn't filter only the best sequences, then others shouldn't be quite so eager to conclude that the tests proved that there was no difference - statistical analysis bias works both ways.

The 1st set of 15 tests also gave a result of around 60%. That’s 45 tests in total, albeit different cables compared. Still not statistically conclusive, but it's kinda getting interesting - along the lines that cjl made.

Anyway, I'm not trying to talk up the results here, just get the balance right. The key thing I'm getting from this is that x838nwy felt that the differences (if they ever existed) have turned out to be smaller than he expected - and certainly small enough to not worry about any further.

x838nwy, I'd still be interested to see a correlation between short term and long term tests. At this point I don't think it's necessary to do it blind. Just leave the Nordost cable in for a few weeks and enjoy listening to music, then swap back to the stock cable. If you still don't hear much difference, then game over. If you do hear a more substantial difference, then more blind tests are required to remove that pesky expectation bias. OTOH, if you can't be bothered with all that anymore, then I fully understand.

Quote:
Originally Posted by Agharta

I would love to read about such things, if someone could meet me under the bridge at midnight. Wear a USB cable round your neck so I can identify you.

There's actually a bunch of very interesting stuff on the Lampizator website where the guy took apart a whole bunch of different hi-end cd players/transports to see what was inside (Krell, for one) only to find out that a great many of them were actually Philips CD players in a nice housing, had terrible design and/or cheap parts all around.

http://lampizator.eu/lampizator/references/krell%20cd300/Krell%20CD300.html

The Krell one is here, there are many other "hi end" ones but I forgot most of the names.