Quote:
** And yes the tests are always framed as "encoder" or "codec" tests. When are they clarified within a subset (age range, gender equality [which is nearly as important as age with audio limits], etc.)? This is only important in how the results are perceived and Hydrogen is SO concerned with perception (even shutting down the site for while once) that I'm surprised at least the equipment isn't addressed in tests posted (even if the tests are only loosely associated with HA, HA is pretty stringent in post rules). Again they're some of the most sound tests out there ... just not are conclusive as their siters keep implying. |
It is not so difficult why equipment is not in TOS#8. It really does not matter on the larger scheme of things. An ABX test can only prove if an audible difference exists, not that it does not exist (the difficulty with proving a negative). In other words, in individual listening tests negative results are quite unintresting because nothing has been proven, so other exact conditions (like equipment and age) do not matter. In case of a positive result, the exact conditions do not matter either, because we have only the codec to blame for the audible difference. Indeed, because a condition of a proper ABX test is that both source and encoded file should be played on the same equipment. IOW: the factor equipment, nor age, nor anything else you can think of does not add or change anything about the presented result.
For a larger scale (not individual) listening tests factors like equipment age and gender are only useful to know if they are controlable variables. If every participant would have given these details they could not in anyway be used to make the tests more accurate.
When is knowing equipment useful? When the same samples are used, with the same people on equipment setup X1, X2 ... Xn.
When is knowing age useful? When the same samples are used, with the same equipment, on different 'sets' of people, each with a different age range.
HA listening tests are not organised in this way. Perhaps they should, but at best I think it remains very unclear how these factors would really influence the result, if at all. I can ABX LAME 320 kbps on some samples with $5 headphones. Not that those headphones are great, or LAME is pretty bad, it was just that those samples prove to be very difficult.
Personally, I believe that sample selection, listener training, and listening environment are factors that weigh much more than equipment or age.