Sorry, for the sake of brevity I was lax and incorrect. I appreciate that I may not understand the finer details of statistics.
As I understand it, CLT says if every test subject has a random (or the same) chance of getting it right every time a choice must be made, then you will get a normal distribution of test subject performance. If you have a deviation from the normal distribution, it may be significant. Am I wrong about this?
The way that the Clarkson test and the Atkinson amplifier test panned out, you got a normal distribution, but only the audio reviewers and industry professionals scored full or near full marks, as a whole the results were basically normally distributed. That SHOULD have signified to the testers that there is a correlation between training and performance, that the population is not equal and that the normal distribution, might not mean that test score is random. However Clarkson told the people who scored full marks or close to full marks, no they really can't hear it, they were just the "lucky coins" in the population that by accident got the answers all right, as the normal distribution still held. I appreciate that people better at stats will not do such a thing, but this is the way the abx tests and those most publicly upheld, were run.
In this case and I should have been more specific about this, using JUST the presence of a normal distribution, is not adequate to say that the test shows that nobody can hear the difference in amps... So yes, I agree with you...
My example pointed out ONLY that the standard deviation in objectively measured performance is as absurd as using the average 100m sprint times to say that people can only run the 100m sprint in x seconds, the other results are just by chance.
So if you are going to use normal stats, you better ask the right question - which is one of the harder things to do, or you have to test and re-test those performing well - if that corresponds to some factor that may prejudice their scores - to confirm they do in fact score differently than the rest of the population. To be honest the breadth of human experience that could be grouped at a single show... would seem to indicate that there might be a need for this regardless of any apparent correlations.
Or you have to perform a multivariate analysis that includes background information, to see if there is any correlation between training and the observed trends... but you may have to classify the data objectively or subjectively and then we will probably fight a little more
What I also do not mean to say is that everybody should buy the most expensive DAC, AMP, Cables they can afford, cause they WILL hear a difference.
Quote:
As someone who has taught graduate statistics in a psych department (that is, measuring human behavior) for more than a decade and a half, this is... wildly incorrect.
There are some serious issues with the way the ABX crowd draws conclusions from statistics—most critically, a failure to reject the null does not justify concluding that two things are equivalent (you basically can never "prove" two things are the same; you can only "prove" differences)—but what you're talking about here is not one of them. The Central Limit Theorem is a key theorem in statistics, but it's not about extreme values.
Hunh? That makes no sense. In what way do "the stats" indicate that? Which stats?
If the question is "does the distribution in the population contain any values less than X?" the stats are pretty clear: if you observe any values less than X, then it does. We can observe values less than X (that is, we see people do it in less than 10 seconds), then the correct conclusion is that there are definitely population values less than X. So I'm not sure what you mean here.
That's not what an ABX test asks. You could ask that question, but it would not then be an ABX test if you did. An ABX test asks "Can people tell the difference?" It's not about preference. There's no point in determining preference if people can't discriminate the two things in the first place.
