[Filburt]

Quote:

Originally Posted by Sir Nobax /img/forum/go_quote.gif however it gives an IDEA how it sounds.

If you understand what I tried to explain to you, it would follow that the graphs alone do not give you an idea of how it sounds. Psychoacoustics is much more complex than this, and a solid analysis will require more than what information is presented in these graphs.

Quote:

Originally Posted by audiomagnate /img/forum/go_quote.gif Get used to it. Anything remotely based on real science around here gets flamed. You have to understand that accurate sound reproduction is more of a religion than a science to most of these people, and you're an atheist who's telling them there is no god. You should have started a thread about how changing the power cord on your CD player made the most amazing improvement in sound you've ever heard through your KSC-75s. Then you'ld be feeling the love right now. I think the complexity of the test wave form, combined with how easy it is to see aberrations, is a great idea. I think it's excellent, well thought out stuff. Keep up the good work!

Is this in response to my post? My criticism of his graphs and conclusions was scientific. If anything, the conclusions being made here about codecs based on their respective graphs are the sort of undersupported conjecturing and pseudo-science you seem to be criticising. You have to ABX codecs in order to get relevant data with which to judge lossy codecs. They're based on psychoacoustics, which inherently reduces to a perceptual domain. However, psychoacoustics is a scientific field, based on data and theories resting upon it and after data collection you can make analysis and develop scientific models and theories, along with make predictions based off of them. In order to do so, though, you still need that initial data.

Furthermore, psy models are primarily tuned for _real music_; not test tones that are unlikely to mirror a realistic scenario, created to try to trick the psy model. Those tests are useful when trying to isolate a problem, or improve performance after having solved problems with actual music, but the failure of a psy model on something highly synthetic is hardly dispositive of the global performance of the model. HE-AAC, for example, uses SBR which can actually work fairly well with many songs, especially when taking into account a low target bitrate, but it is unlikely to perform well in some circumstances, such as probably those of this type. Another thing to consider is that what shows as visible loss in some areas may not necessarily indicate poor performance. The fact of the matter is, as per the very foundations of psychoacoustics, you cannot hear all which is available in the source data, and it's the job of the psy model to figure out what you can and cannot hear. A graph that may show some visible loss may indicate the psy model was able to better figure out what was inaudible than that of another codec, and so was able to dedicate more bandwidth to that which is audible in the source.

I'll reiterate my earlier point. Lossy compression's focus is maximisation of quantitative loss with minimisation of qualitative loss. These graphs are unlikely to be sufficient for the purposes of making a well-reasoned, defensible analysis of the qualitative performance of a codec, especially without a working background in psychoacoustic theory and analysis.

 

[Thelonious Monk]

thanks to Filburt and Febs for putting it better than i could have. it's not an elitism or blind faith thing.. and i'm really against stuff like expensive power cables and ERS paper and other stuff like that.

so yeah, to put it simply, you can't hear everything that's there. ABX testing is the best way, and yes, it IS scientific.

these types of graphs will never show you anything but "amounts", depending on what you're measuring. they won't really show you what the actual difference is.


[size=xx-small]that was really poorly worded, i know, but i'm really quite too tired to put on my serious boots[/size]

 

[Sir Nobax]

Quote:

Originally Posted by Filburt /img/forum/go_quote.gif If you understand what I tried to explain to you, it would follow that the graphs alone do not give you an idea of how it sounds. Psychoacoustics is much more complex than this, and a solid analysis will require more than what information is presented in these graphs.

What makes you think every human hears the same thing, if i had a hearing loss between 3khz and 5khz and the Psychoacoustic model used in the codec thinks that a tone of 4khz would overshadow the 2khz tone in a specific music file the psychoacoustic model is flawed in this case. So if i heard the source i would hear the 2khz better then the 4khz one, in the compressed file the codec filtered out the 2khz tone making my experience different from the source.

But with the pictures you can't see wich 128kbps encoding of different codecs (without parametric stereo and Spectral band replication) is better sounding to you, you only see how accuratly it reproduces the original sound. But with the 256kbps WMA 10 Pro graph you can clearly see that it is WAY better then the MP3 at 256kbps, if someone could hear the difference is something else, and thats why spectrographs are more accurate then listening tests in my opinion.

...and can anyone recommand a ATRAC encoder??

 

[Sir Nobax]

Quote:

Originally Posted by Thelonious Monk /img/forum/go_quote.gif so yeah, to put it simply, you can't hear everything that's there. ABX testing is the best way, and yes, it IS scientific.

Quote:

Originally Posted by Wikipedia Wikipedia:Scientific method is a body of techniques for investigating phenomena and acquiring new knowledge, as well as for correcting and integrating previous knowledge. It is based on gathering observable, empirical, measurable evidence, subject to specific principles of reasoning.

Only comparing different codec's by listening is [size=medium]not[/size] scientific.

If you would combine ABX-tests with my results you would come a lot closer, having both empirically evidence and real measurements.

 

[db597]

Quote:

Originally Posted by wnmnkh /img/forum/go_quote.gif How people can say that graphs do not represent the codec quality? I do not understand.

It does show where the data loss occurs for different encoders - and is useful in doing comparisons between them. What it can't predict is whether it will sound good, as it doesn't factor in psychological factors (i.e. the human ear can't detect missing data if its placed in certain places - our brains are flawed and just don't notice a difference).

It's exploiting flaws in human perception that's the goal of lossy compressors. The good ones are more effective at placing data loss in areas which we can't hear. Having lots of black holes and missing bits doesn't mean poor sound quality, though it means poor reproduction.

On the flip side, I think these graphs are useful for other purposes. e.g. you can compare the different encoders to each other, and see which is the most efficient at a given bit rate. People always said Blade is inferior to LAME - these graphs prove it. Bravo Sir Nobax!

 

[db597]

Quote:

Originally Posted by Sir Nobax /img/forum/go_quote.gif My DT550 and my HQ-1700

Thanks for doing those tests Sir Nobax. Super!

I suspect the loss of the fine lines are more exaggerated than in reality - it's not recording the full range of waves from the headphones - or attenuation by the time they reach the mic. If these are accurate, we'd never hear a difference between encoders, as the headphones massacre it to the point that the source has an insignificant impact.

It's like in the lab - we could never take very accurate measurements (experimental error!). Very tough. I wonder if turning up the volume helps?

 

[Joshatdot]

How did you generate the Test Wave? Where can I get it, or generate it?

 

[Febs]

Quote:

Originally Posted by Sir Nobax /img/forum/go_quote.gif Only comparing different codec's by listening is [size=medium]not[/size] scientific.

If we are going to accept the Wikipedia as an authoritative statement on the issue, then you are wrong, as the Wikipedia specifically identifies a codec listening test as a scientific study:

Quote:

A codec listening test is a scientific study designed to compare two or more lossy audio codecs, usually with respect to perceived fidelity and/or compression efficiency. Most tests take the form of a double-blind comparison, referred to as "ABX" or "ABC/HR" testing. In an ABX test, the listener has to identify an unknown sample X as being A or B, with A (the original) and B (the encoded version) available for reference. The outcome of a test must be statistically significant. This setup ensures that the listener is not biased by his/her expectations, and that the outcome is not likely to be the result of chance. In an ABC/HR test, if sample X can be identified reliably, the listener can assign a score as a subjective judgement of the quality. If sample X cannot be determined reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between samples A and B. This usually indicates that the encoded version will actually be transparent to the listener. There are various software packages available for individuals to perform this type of testing themselves with minimal assistance.

Independent of this article's characterization of a codec listening test as a scientific study, as you can see from the description above, a proper ABX test meets all of the criteria for a "scientific study" that are described in the article that you cite.

Quote:

What makes you think every human hears the same thing, if i had a hearing loss between 3khz and 5khz and the Psychoacoustic model used in the codec thinks that a tone of 4khz would overshadow the 2khz tone in a specific music file the psychoacoustic model is flawed in this case. So if i heard the source i would hear the 2khz better then the 4khz one, in the compressed file the codec filtered out the 2khz tone making my experience different from the source.

I guess my response to this is, "so what?" This example does not support your argument that the graphs are a better way to compare codecs than listening tests.

As a threshold matter, you identify a potential weakness in the psychoacoustic model. Though you do no explicitly state what the relevance of that potential weakness is to your argument, I am inferring that it is your position that the graphs depict the performance in an objective manner, which can be applied to the entire population, whereas a listening test can be applied only to the specific subject of that listening test. Thus, the graph will allow you to evaluate a codec even for those outlier cases, whereas a listening test will not. (Please feel free to correct me if I have misinterpreted what you are trying to say.)

There are several flaws in this reasoning. First, the inference that listening tests cannot be applied to the population as a whole is flawed. It is true that any one listening test will apply only to the subject of that test. However, if the test is conducted over a statistically significant portion of the population, then results can be obtained that are predictive of the whole population.

Second, your hypothesis that the psychoacoustic model will not work effectively for people with hearing loss between 3kHz and 5kHz is completely untested. Therefore, you cannot draw any conclusion with respect to whether the graphs are a better way of evaluating codecs for that subsegment of the population. As it so happens, I do have hearing loss between 3kHz and 5kHz. That does not make the graphs any more or less useful at predicting whether any particular codec will be transparent to me. It is still the case that they only way for me to tell whether any particular codec is transparent to me is to conduct a listening test.

Third, even if we assume arguendo that your hypothesis regarding the application of psychoacoustic models to people with hearing loss is correct, your attempt to prove your general case using that specific example that is neither persuasive nor an example of valid scientific methodology. Your argument that these graphs are more useful than listening tests because they apply to the entire population, whereas listening tests do not, is demonstrably false. Take for example, the graphs that you've posted that show a loss-pass filter at 16kHz. For the listener with hearing loss of all frequencies above 10kHz, those graphs impart no meaningful information whatsoever.

 

[Sir Nobax]

I've uploaded the Test Wave, everybody can grab it from the first page and HERE too.

The tone was generated with an old version of CoolEdit Pro wich i can't seem to remember, i tried a few yesterday night but it where all different versions.
The spectographs are also made in CoolEdit Pro.

Quote:

Originally Posted by Gurra1980 One thing that i didn't understand was that WMA9.2 was worse than AAC I can hear the difference from AAC and lossless at much higher bitrates than WMA9.2. What program did you use to encode with? Did you use WMP11? no you could not have done that because it can't encode higher than 192kbps. Try to do it with WMP11.

Remember that i made CBR WMA 9.2's in my test, also if you encode music within WMP11 on the Pro-setting it is WMA 10 Pro, however i've never used WMP11 to encode music, i use DBpowerAmp, when i had WMP 10 installed i could convert the WMA9.1 files, the moment that i installed WMP11 my options where different, suddenly i could encode to WMA 9.2 and WMA 10 Pro but i was'nt able to encode WMA 9.1's.

 

[Gurra1980]

Quote:

Originally Posted by Sir Nobax /img/forum/go_quote.gif I've uploaded the Test Wave, everybody can grab it from the first page and HERE too. The tone was generated with an old version of CoolEdit Pro wich i can't seem to remember, i tried a few yesterday night but it where all different versions. The spectographs are also made in CoolEdit Pro. Remember that i made CBR WMA 9.2's in my test, also if you encode music within WMP11 on the Pro-setting it is WMA 10 Pro, however i've never used WMP11 to encode music, i use DBpowerAmp, when i had WMP 10 installed i could convert the WMA9.1 files, the moment that i installed WMP11 my options where different, suddenly i could encode to WMA 9.2 and WMA 10 Pro but i was'nt able to encode WMA 9.1's.

Thanx for all the time you have taken to share your testresults with us, it's always very interesting with such tests, I think that using both ABX testing and mesurments like these are always the best way, just look at respected Hi-Fi mags, most of the time they are using both methodes when testing stuff.


It would be interesting if you could do a test with WMA9.2 encoded in WMP11, but you don't have to, just if you want.

 

[Sir Nobax]

Quote:

Originally Posted by Gurra1980 /img/forum/go_quote.gif It would be interesting if you could do a test with WMA9.2 encoded in WMP11, but you don't have to, just if you want.

My guess would be that the results will be the same, as DBpowerAmp uses the same codec (only the bitrate goes higher). I'll have a look at it tonight though, just to confirm it

.

 

[hYdrociTy]

thanks mister nobax! looks like its going to be aac224 vbr for me now! no more flac.... battery killing flac....

I don't know what you are all arguing about and I really don't care to read it(especially those references to esoteric "academic" concepts and articles), but I was looking for specific things that all his graphs show. Maybe you don't like looking at pictures because you think your ears are all that... well mine aren't. I look at the hf rolloff on the cbr vs vbr, and his graphs show me what I was looking for. How you people say a graph is not accurate when you clearly see some of the rolloff? if something is not there, that means it's not there. no "audiophile" ears are going to bring it back...or.. maybe when hf is cut off, it 'sounds good'. well if it does then kudos.

id rather listen to the ugly truth than a pretty lie.

 

[Sir Nobax]

Quote:

Originally Posted by Febs /img/forum/go_quote.gif Your argument that these graphs are more useful than listening tests because they apply to the entire population, whereas listening tests do not, is demonstrably false. Take for example, the graphs that you've posted that show a loss-pass filter at 16kHz. For the listener with hearing loss of all frequencies above 10kHz, those graphs impart no meaningful information whatsoever.

....wth, someone with a hearing loss of all 10khz+ tones won't care if he is listening a 128kbps Lame with --lowpass(10) or a uncompressed file, he won't hear the difference. And still then, he knows that last one is better if that person would take a look at the spectographs, he could'nt know that otherwise.

 

[Febs]

Quote:

Originally Posted by hYdrociTy /img/forum/go_quote.gif How you people say a graph is not accurate when you clearly see some of the rolloff?

I never said that. If you "really don't care to read" what I'm saying, then please don't comment on it or try to characterize it. I do not dispute that the graphs convey information about the codecs. What I dispute is the assertion that the graphs are more effective than actual listening when it comes to evaluating the effectiveness of lossy compression that uses perceptual encoding schemes. In other words, is the information that is conveyed by the graph meaningful in determining how the codec sounds? I suggest that you cannot answer that question without conducting listening tests.

Quote:

if something is not there, that means it's not there.

But that does not mean that the absence of that "something" is audible. The only way to evaluate that is through listening tests.

 

[Febs]

Quote:

Originally Posted by Sir Nobax /img/forum/go_quote.gif ....wth, someone with a hearing loss of all 10khz+ tones won't care if he is listening a 128kbps Lame with --lowpass(10) or a uncompressed file, he won't hear the difference.

How do you know that? Answer: you don't. You're making a lot of unfounded assumptions.

Quote:

And still then, he knows that last one is better if that person would take a look at the spectographs, he could'nt know that otherwise.

"Better"? How is it "better"? You're using an undefined, subjective term. If you mean that it is "better" in that it discards less information than some other encoder, I suppose I would agree with you. But that doesn't mean that it sounds "better" or that the perceptual encoder, whose job it is to discard inaudible information, is doing a "better" job.