Originally Posted by hciman77
Are you suggesting that sighted listening tests can be as reliable as blind tests, if so how come when sighted tests are followed by blind tests the differences detected by sighted tests often disappear i.e in Masters and Clark amongst others ?. The subjects experience a real certainty that big differences exist and describe them in great detail but when denied visual
cues they cannot detect the difference.
Sighted tests are fine but they allow all sorts of human biases to come into play i.e expectations, for instance I have an irrational liking for Rotel gear, I have a Rotel CD player (I have owned several in fact) and a Rotel Integrated amp, if I was asked to listen to a Rotel and a NAD my predisposition would be to prefer the Rotel even if they sounded identical.
You should read "Toole and Olive" (1996) in it they showed how sighted listeners perception of subjective quality were heavily biased by knowledge of what they are listening to i.e brand, physical apperance and so on and that when stripped of these cues the results were very different under otherwise identical listening conditions.
What I am suggesting is that DBT's are not necessarily better than sighted tests, nor are sighted tests necessarily better. What I am saying is both are merely testing methodologies and the results are part of a statistical design which has a large influence on the generalizability of the results. In addition there is no DBT, there are only specific DBT's that have been run. Each is essentially unique in detail and these details are critical to the interpretation of the results.
My larger point is that I, like many others do not care. I have no reason to delude myself that something is better in my system so a DBT is less valuable to me than extended listening. As an example, if you compare my 2-channel system today to the system 2 years ago only the transducers (speakers and headphones) are the same, all of the electronics are different as are IC's, power cords and speaker cables. Some items are more expensive and some are less expensive but all sound better to me (or are filling in until I find something that sounds better) than the items they replaced.
Anyway, I am not stating that DBT's are bad, just that in their implementation they might not really add anything and to the layman they may seem "more scientific" and thus receive more credibility than they deserve. I did appreciate your initial few posts for adding a much needed reality check but the last few seemed to get a little preachy for me.
Originally Posted by regal
DBT's aren't perfect. Ears are on humans, not robots. To get a statistically valid test several replications are required. People have short attention spans, there is a lot of psychological and physiological factors that influence the tests as much as the factors you examining. You end up with tests that can take weeks to be truly valid.
Exactly. Replication is crucial to statistical power both in its magnitude and in its implementation, i.e random, paired, nested, etc.. Much of the limitations in the scientific tests of components is because they do not take account of the biological and psychological factors since the researchers are not experts in those fields.