Also, as an interesting note, you don't need to score close to 100% to show statistical significance. An ABX or A/B test can have a positive outcome even if you only get the correct answer 51% of the time. The closer your outcome is to 50% though, the more trials are needed to show that the result is unlikely to be purely chance driven. In your case, the sum of your last three trials is 19/30. While this is actually somewhat unlikely from chance alone, it isn't nearly unlikely enough to be considered statistically significant (if I did the math right, the chances of getting at least 19/30 correct by pure chance are around 14%). If you continue to get ~63% correct through many more trials (which is not an expected outcome based on the current data, I should add), the probability of a 63% success rate occurring by chance alone drops below 5% once you have successfully achieved 32/50 correct (so a 32/50 would be considered a possible positive result for an ABX test). If you tighten up the required P value to 1% (since 5% still means that by chance alone, 1 in 20 random trials will achieve that kind of significance), you would need a score of 62/98 to show a significant result.
Thank you cjl. From memory alone, 19 out of 30 trials is unlikely to be significant. But you are right, there is a statistical test that is based on the total number of samples also. Could you point me to a webpage that has the mathematics on this? I'd like to regain a better understanding of this myself.
I commend you x838nwy for doing this. You are experiencing what at least many of us have. You just know, feel it in your gut, they aren't the same. Then with a blind test, well you feel differences are small, but you just know they are there. Yet the test results indicate there is no audible difference likely.
I have tried to conceive of a test like the two red colored tiles you speak about. My first idea was separate cables on each channel listening over headphones. But that doesn't work. When differences I artificially induce in one channel become quite perceptible in such a concurrent test they are already larger differences than I detect with an ABX in Foobar.
Also, though you have the subjective experience of seeing two tiles 'at the same time side by side' your eyes actually switch rapidly from one to the other, and do some trickery so you think you are looking at both. Using software that could instantly switch from one color to other on my computer screen I found I could more easily pick two very close colors than if I had them side by side. The key is instant (in my case 1/60th of a second) switching. Putting in a dead time of 5 seconds lowered the sensitivity of color discrimination for me.
Finally, the logic of using music for your test seems good. It has been known for some time smaller differences can be detected with proper test signals in blind testing. For your cable test I would suggest pink noise. At not too high a level mind you. This way you have a constancy of signal character, can rapidly switch between and make your choices. Just a few seconds before switching makes the test less time consuming to do a fair number of trials. I find it less tiresome in practice than using music myself. This is if you wish to give it one more go.
It takes time to turn loose of one idea and get comfortable with another one. You had the guts to actually try this out. I have found few people willing to do such a thing. Most just bluster about how obvious it is and never quite commit themselves to testing the proposition cables sound different. Or attack the methodology if they do and get results they don't like.
As you get comfortable with the idea cables are generally transparent and fully interchangeable it is somewhat freeing. You can use whatever inexpensive cable is at hand with no worries. It is a part of the audio world that is cut and dried. Leaves more money and time to focus on things that really matter.
The switching concept is a good one and I'm interested as I think it will make for a fairer test. I have not tried playing pink noise through it either, so it'd be interesting if I could have the system play the same tone through A, say then through A or B and I have to say if the second playback is the same as the first. It'd probably no adhere too closely to a definition of a DBX...
Well, my conclusion is pretty much what I've highlight in bold type. It's been kindda fun and interesting to actually put stuff to the test, but it at least gives me a better sense of proportion to plan my system's direction. I think I've been as fair as I could have been - in fact even biased in expecting a difference, so I think I gave the cables concerned a decent chance. I hope I haven't put Nordost or their Blue Heaven cables in the spotlight and that is not my intention. They're simply the most costly interconnects I own, are at the right length and of the right sort of plugs for testing against my $7 wonder-cable.
I have to say one last thing though - all my tests have been with analogue interconnects. I am not claiming that the same will or will not apply to other types of interconnects, tonearm wires, speaker cables or power cords. And while I currently have a highly modified opinion on interconnects (in comparison to the opinion I held prior to the tests), I remain open to reasonable suggestions on ways to improve my tests or other approaches.
With things like this, there remains always a "garage fairy" thing - if someone were to say that in garages exist fairies and I want to prove that they are fiction, looking in my garage for fairies every couple of hours and not finding any does not constitute the proof I am looking for as my methods may be incorrect or my knowledge/skills insufficient. The difficulties in proving negatives is a challenge. But for this one, I think I am comfortable with what I found. I shall not be spending unreasonable sums on money on my analogue interconnects.
p.s. There IS a possibility that I have stumbled upon the most amazing $7 cables in the history of hifi, being a gnat's hair's width from a Blue Heaven. But that's highly, highly unlikely.
Have a great week everybody :)