Quote:
It's just since a couple of your posts back that I now fully understand what you've been getting at and how DBT's could fail in detecting a genuine, though subtle difference. However, I must admit, in the defence of cable skeptics, that if someone claims that he can easily tell a difference between the cables that he uses and that one provides a night and day improvement over another or that it's like a veil has been lifted in the sound when using one cable as opposed to another, then it really aught to be easy to detect a difference in a blind test environment more than 70% of the times. No?
In my experience, I do think I hear a difference with a couple cables that I have when using them with my K702's and now my HD800's. In neither of the cases is the difference like night and day. It's subtle and for the life of me, the differences fade when I attempt to A/B the cables in question to pinpoint or better characterise the differences. When I use one for a while and switch to the other the difference is more apparent. I've never formally volume matched and for this reason, I cannot be too confident about my findings. The same applies with amps that I've switched and DAC's as well. Never really night and day differences. Never... and for this reason, I can sympathise with those who have similar skepticism about the claims made about the wonders of some of the more spectacular looking tube amps out there. There's much placebo effect based on expectations built into those designs. I've heard night and day differences only with cans and the difference is way more drastic when switching to one after a long period of listening solely to the other.
I agree that a positive test proves without doubt what it claims.
It's the negative result that is the crunch and this is where beeman's criticism takes center-stage. The DBT statistical criteria to prove validity cannot differentiate between guessing and a genuine but low positive pickup rate. But you've said it already... 'Why pay so much for such a small benefit, even if it turned out to be demonstrably real, whatever the methodology that's uncovered to demonstrate it with a reasonable degree of accepted validity.'
I'm happy to see the discussion going this route. It's very scientific in that it's looking at the methodology and how we interpret and use statistical analysis. Statistical analysis isn't perfect and furthermore, it's very much divorced from experimental methodology and its validity. You could use wonderful statistical analysis but end up with rubbish results because of poor statistical application or poor experimental methodology.
Now I get what beeman is getting at, and his problem is NOT with DBT's and Statistical Hypothesis testing, but with the interpretation of the results. Crap I had already written a defensive and not so nice post aimlink! I had to start all over again!!
Quote:
Aimlink wrote:
It's just since a couple of your posts back that I now fully understand what you've been getting at and how DBT's could fail in detecting a genuine, though subtle difference.
Thanks! Sorry if I did a poor job in conveying the concept.
However, I must admit, in the defence of cable skeptics, that if someone claims that he can easily tell a difference between the cables that he uses and that one provides a night and day improvement over another or that it's like a veil has been lifted in the sound when using one cable as opposed to another, then it really aught to be easy to detect a difference in a blind test environment more than 70% of the times. No?
I'm in agreement and agreed, folks need to tone down their aggrandized statements.
In neither of the cases is the difference like night and day. It's subtle and for the life of me, the differences fade when I attempt to A/B the cables in question to pinpoint or better characterise the differences.
As a suggestion, go a step further down into the sub-conscious area of your mind and evaluate your emotional response as opposed to evaluating your hearing response. In the simple, does it put a smile on your face or does it take the smile away as your emotions register disappointment? I'm serious when I suggest checking for the above.
Feel free to give us references to that analogue nature you are talking about, remember we scientists are one of the most open minded people (we are wrong way more often we would like
), but without references or clear definitions a scientist cannot work.
The last paragraph you say is not scientifically valid (which says nothing about it being true), so I won't say anything about that.
I want to clarify, a Null Hypothesis test does not require that particular individual has more than 70% correct, it is all about the average, because by seeing the average you control the risk you mention of being wrong (although it never discards it completely). First to be statistically valid it cannot be a test performed by only one individual, but by many, so the fact that one only gets a low percentage (say 50%) good proves absolutely nothing, neither that other one has 100%, but is the average what is important (and is related to the law of large numbers). Basically you are trying to minimize the risk that the result you are seeing was just a guess (that is why the freaking coin appears, lets just forget that).
That is why statistical hypothesis tests deal with probabilities, even with an awesomely well performed experiment there is risk of error (the famous p-value), technically speaking you cannot say the null hypothesis is accepted, but it casts doubt to whether it is not true (just as correlation is not causation, but hints at something happening). Yeah, that is why the result is to say: there is no evidence to reject the null hypothesis, that is very important.
So scientifically I'll repeat: there is no evidence (statistically significant) to say that people hear differences in cables, that's it, no more. I never said there can be no difference, but the results give me doubts, and with doubts a scientist prefers to err in the side of accepting the Null hypothesis since false positives generally are more much damaging than false negatives (the idea is that its better to set free a guilty guy than to imprison an innocent).
Here and
here are little wikipedia articles (which is easy-ish and not that thorough, but not bad at all), there are (valid) critiques to the method (it was the reason why I mentioned the Bayesian Statistician thing), in case you are interested. Anyways, have some beer for standing in what you believe in, I respect that!
(I just saw your last post with some references, so disregard what I said about that earlier, I'm tired of editing this post)