Originally Posted by Sir Nobax
Only comparing different codec's by listening is not scientific.
If we are going to accept the Wikipedia as an authoritative statement on the issue, then you are wrong, as the Wikipedia specifically identifies a codec listening test
as a scientific study:
|A codec listening test is a scientific study designed to compare two or more lossy audio codecs, usually with respect to perceived fidelity and/or compression efficiency.
Most tests take the form of a double-blind comparison, referred to as "ABX" or "ABC/HR" testing. In an ABX test, the listener has to identify an unknown sample X as being A or B, with A (the original) and B (the encoded version) available for reference. The outcome of a test must be statistically significant. This setup ensures that the listener is not biased by his/her expectations, and that the outcome is not likely to be the result of chance. In an ABC/HR test, if sample X can be identified reliably, the listener can assign a score as a subjective judgement of the quality.
If sample X cannot be determined reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between samples A and B. This usually indicates that the encoded version will actually be transparent to the listener.
There are various software packages available for individuals to perform this type of testing themselves with minimal assistance.
Independent of this article's characterization of a codec listening test as a scientific study, as you can see from the description above, a proper ABX test meets all of the criteria for a "scientific study" that are described in the article that you cite.
|What makes you think every human hears the same thing, if i had a hearing loss between 3khz and 5khz and the Psychoacoustic model used in the codec thinks that a tone of 4khz would overshadow the 2khz tone in a specific music file the psychoacoustic model is flawed in this case. So if i heard the source i would hear the 2khz better then the 4khz one, in the compressed file the codec filtered out the 2khz tone making my experience different from the source.
I guess my response to this is, "so what?" This example does not support your argument that the graphs are a better way to compare codecs than listening tests.
As a threshold matter, you identify a potential weakness in the psychoacoustic model. Though you do no explicitly state what the relevance of that potential weakness is to your argument, I am inferring that it is your position that the graphs depict the performance in an objective manner, which can be applied to the entire population, whereas a listening test can be applied only to the specific subject of that listening test. Thus, the graph will allow you to evaluate a codec even for those outlier cases, whereas a listening test will not. (Please feel free to correct me if I have misinterpreted what you are trying to say.)
There are several flaws in this reasoning. First, the inference that listening tests cannot be applied to the population as a whole is flawed. It is true that any one listening test will apply only to the subject of that test. However, if the test is conducted over a statistically significant portion of the population, then results can be obtained that are predictive of the whole population.
Second, your hypothesis that the psychoacoustic model will not work effectively for people with hearing loss between 3kHz and 5kHz is completely untested. Therefore, you cannot draw any conclusion with respect to whether the graphs are a better way of evaluating codecs for that subsegment of the population. As it so happens, I do
have hearing loss between 3kHz and 5kHz. That does not make the graphs any more or less useful at predicting whether any particular codec will be transparent to me. It is still the case that they only way for me to tell whether any particular codec is transparent to me is to conduct a listening test.
Third, even if we assume arguendo
that your hypothesis regarding the application of psychoacoustic models to people with hearing loss is correct, your attempt to prove your general case using that specific example that is neither persuasive nor an example of valid scientific methodology. Your argument that these graphs are more useful than listening tests because they apply to the entire population, whereas listening tests do not, is demonstrably false. Take for example, the graphs that you've posted that show a loss-pass filter at 16kHz. For the listener with hearing loss of all
frequencies above 10kHz, those graphs impart no meaningful information whatsoever.