Hmm the quote system isn't working right now.
"liamstrain - Is there evidence that rapid switching ABX is the only kind of ABX we are recommending? I have no problem with more flexible test formats to allow for more natural listening."
Not anyone in this thread I think, sorry if it came across like that, but I think I've seen at least one person insist on a volume-matched 0.1 second rapid switch ABX format for detecting differences in their amplifier versus more upmarket ones, or a modified version of the same amp.
I think rapid switching ABX is very likely the best method for detecting minor differences in volume, but it seems dangerous to extrapolate on that to all perceivable sonics. As a sensory analogy, if you rapid switched between two different perfumes, you could be tricked into perceiving them as the same, while if you took a very quick break or smelled coffee inbetween perfume A and perfume B, you could more correctly identify the subtle differences. If you took a 10 minute break, they could smell different due to the more typical reasons like expectation bias via visual impact, price, etc.
"Prog Rock Man - Past results show that taking the 10 who 'passed' will result in another random 'passes' and you could go on until you had just the one who has 'passed' then all. But now prove that is nothing more than random itself."
You can't incorporate previous test results into future tests. You can assume the 10 who passed were only guessing, but that is purely an assumption and not statistics or science.
Let alone one of those 10 who passed might have said "wow system A sounded so good!" when he left the room, or pointed violently at the left system while in the room, influencing others to choose the left system. If it dispelled some myths about super expensive audiophile CD players and CD racks not introducing very audible differences, then OK that's fine, but I still view the test as a novelty...
"There are thousands of sighted test result, just read What Hifi for dozens each month into all sorts of hifi products from cables to speakers."
I see your point there. I'm not familiar with What HiFi, but perhaps they
can't hear the differences?, i.e. they're just fabricating them for other incentives like profit, or suffering from the visual impact, price, or other emotional influences...
If you think more reviews should be conducted blind without having any idea what the product is, I have no issues there at all! I think most people have decided if they like the car or not before they've turned the engine on, and someone decided audio should be all expensive leather and chrome too, not just sound. So, some people think audio
is mostly just expensive leather and chrome, which isn't true!
"2 - that different parts of the hifi chain perform differently. So no cable has ever passed an ABX test and they do badly in blind comparison, amps get mixed results, bit rates do better again and speakers do best of all in both blind comparison and ABX tests."
bit rates do well because their digital nature makes them convinient to ABX, unlike physical components, people can
practice bit rates and eventually find the differences.
There are people (like myself) which have a hard time with ABX'ing bit rates, and Wikipedia says that 256kbps MP2 is 100% transparent, i.e.
no one can hear the difference versus CD quality, proven by extensive blind-testing, actually they write "in the most critical conditions ever implemented" -
http://en.wikipedia.org/wiki/MPEG-1#Quality
I think the ABX study they're referring to involved 61 listeners with something like 20,000 trials, in the end only 1 of them could detect a difference, that should underline that you
can't collect the data and dismiss the positive results.
Another case in point, Wikipedia says more and more people now
prefer MP3 compression to lossless -
http://en.wikipedia.org/wiki/MP3#Audio_quality, which means you
can't dismiss preference.
The
Pras & Gustavino study took that into account. Basically, some subjects kept selecting the wrong answer with such significance, that it becomes statistically significant, like losing money at a roulette table until it reaches a 1/1000 chance of losing with such significance. Most studies don't take that into account.
"The clear conclusion is spend your money on speakers/headphones and not on cables."
Yes.