Another issue with this test is that Tyll knew he was looking for a difference and he heard both headphones before the test. He was also familiar with the songs being played and knew what small nuances to listen to.
But then again, I don't think Tyll claims anywhere in the article that this is objectively conclusive proof that burn-in exists and is audible. Its great that this attempt to get some more data in this great big argument was made and I applaud it.
How is any of that an issue in a blind comparison? None of that should matter if he doesn't know which device is on his noggin. Looking for a difference doesn't allow to find consistancy in randomness if not there.
It's been ignored but I said early on that the way to confirm in this instance would be to fully break in the new pair and repeat the test. It would give a control of whether the two phones were actually close enough to identical in original build to be meaningful.
Obviously one test either way doesn't 'prove' anything but if we're going to be headstrong and scientific, we should approach this rationally. Listening material doesn't matter and source doesn't matter as long as it can drive the device and is of a high enough quality not to mask things. Testing procedure and a control matter. In this case the control would be to break in the new phones and compare again and when I say break in, I mean like over 1000 hours so that we won't be talking about variance in time.
Edited by goodvibes - 9/9/11 at 6:48am