[Phronesis]

"Notice that, despite deviations, both distributions have similar bell shapes. Furthermore, all reliable p-values are in favor of the null hypothesis stated, some of them in high agreement. So, based on the data obtained, the most reasonable conclusion is that we can’t hear the difference between CD audio and iTunes plus. And this is true in all the cases considered—being young, with our sense of hearing at its peak, having musical training or using excellent audio gear doesn’t seem to help."

Yes, the author drew a conclusion which isn't really supported by his own results. The whole point of calculating specific p-values is to get a probability that a difference has been found, rather than resorting to eyeballing shapes of distributions, and choosing an arbitrary p-value of 0.05 as a limit. I would be interested in knowing the author's background.

 

[KeithEmo]

I really need to point something out here that people keep forgetting.

Statistical results ONLY provide a valid answer to statistical questions.

If you want to know "how fast the average person can run" then you're looking for a statistical result.
If you want to know "whether most people can run a mile in under five minutes or not" then you're looking for a statistical answer.
And, if you want to know "if most people can tell the difference between lossless and lossy files most of the time" then that is also a statistical answer.
But, if you want to know "how fast the fastest human being can run a mile" that is NOT a statistical answer... it's a simple single number.
And, if you want to know whether "humans can EVER notice a difference between lossy and lossless files" then THAT is a simple yes or no question.
(And, that being the case, if a single human being can be shown to do so, you have your answer.)

So, if that's what you REALLY want to know...
Then our best bet for finding out would probably be to buy a plane ticket for the guy who was supposedly able to do so on that test.
Because, if we test him, and find out that he really can reliably do so, then we have our answer... to that specific question.
(However, it probably says little about "what most people can hear".)

And, yes, in science, you have to be extremely careful to ask the right question if you want the right answer.

If you ask "how fast can a human run a mile" - there is only one correct answer:
"The current record is held by Hicham El Guerrouj, who ran the mile in 3:43.13"
There is no point in looking to statistics for the answer to that question....
And no statistical result that can possibly alter that answer....
(And, unless you find someone faster, no result of any test you can perform will alter that answer in any way.)

"Notice that, despite deviations, both distributions have similar bell shapes. Furthermore, all reliable p-values are in favor of the null hypothesis stated, some of them in high agreement. So, based on the data obtained, the most reasonable conclusion is that we can’t hear the difference between CD audio and iTunes plus. And this is true in all the cases considered—being young, with our sense of hearing at its peak, having musical training or using excellent audio gear doesn’t seem to help."

 

[Sgt. Ear Ache]

In the end, I'm perfectly happy to accept the conclusion that .05% of the people on this planet "might" be able to reliably hear the difference between a 256kb lossy file and a lossless one. I know I'm not in that .05% group. Nor are most of us...

It would be lovely if folks selling gear or services on the basis of lossless being better were to say something like ",05% of you might be able to hear improved sound quality thanks to our lossless audio"

 

[Phronesis]

In the end, I'm perfectly happy to accept the conclusion that .05% of the people on this planet "might" be able to reliably hear the difference between a 256kb lossy file and a lossless one. I know I'm not in that .05% group. Nor are most of us...

It would be lovely if folks selling gear or services on the basis of lossless being better were to say something like ",05% of you might be able to hear improved sound quality thanks to our lossless audio"

But if you quote a number like that, it needs to be backed up by good research. Otherwise, you'll have to use words like might, seems, about, etc. when stating your tentative conclusions.

Maybe we can all agree that it seems quite unlikely that the vast majority of listeners really notice significant differences between lossless and lossy with the gear they normally use, for the music they normally listen to, the way they normally listen to the music. I do think the evidence so far supports that conclusion pretty well.

 

[Sgt. Ear Ache]

But if you quote a number like that, it needs to be backed up by good research. Otherwise, you'll have to use words like might, seems, about, etc. when stating your tentative conclusions.

Maybe we can all agree that it seems quite unlikely that the vast majority of listeners really notice significant differences between lossless and lossy with the gear they normally use, for the music they normally listen to, the way they normally listen to the music. I do think the evidence so far supports that conclusion pretty well.

lol. OK.

 

[castleofargh]

I absolutely agree....
However, this leads us right around the circle, and back to a very basic question:
WHICH claim is the one that we are supposed to reject without proof?
If we accept the criterion that "we shouldn't accept ANY claim without proof" then we simply have two unproven claims.
I should also note that failure to prove that one of those claims is true does NOT prove by default that the other is true.

One person claims that "lossless files sound audibly the same as lossy files".
Another claims that, since lossy files can be shown to be measurably quite different, it seems likely that they will be audibly different.
(I personally suspect that, because lossy files are measurably very different, it seems likely that there will turn out to be situations where those differences lead to audible differences... but I make no claim to have tested it either way.)

It is a logical fallacy to assume that either of those claims is some sort of "default assumption"; by your criterion there is no such thing as a default assumption..
Neither claim is "obviously true" or "obviously likely to be true" or "obviously unlikely to be true".
They are BOTH "just empty claims" until and unless valid and relevent proof is presented.
We need to see actual proof before assuming that EITHER of those claims is true.
(Note that the scale and scope of the proof must be appropriate; for example, if you test five subjects and three pieces of equipment, you cannot reasonably generalize your results to "everyone" or "all equipment".)

I should also point out that most lossy compression methods, including the popular MP3, are not standardized.
If you compress the same original file into a "320k VBR MP3 file" using different encoders, you will end up with different results.
Each encoder uses its own "judgment" to decide what to discard.
They are all based on the same basic assumptions - but the details vary considerably.
Therefore, at a very minimum, when making this sort of claim, you must specify the EXACT encoder, version, and settings that were used.
(There is no reason to assume that different encoders, or the same encoder with slightly different settings, will produce equally "audibly transparent" results.)
And, yes, this can be a problem if you purchase MP3 files, because vendors often neglect to tell you what compressor and settings they used.
(Of course, the solution there is to find an encoder whose performance YOU trust, then compress your own files from lossless originals.)

we all have assumptions, no issue there. it's not like I'm going to delete @Phronesis' list of hypotheses because it's forbidden to think about possibilities. my point was mainly that we have no requirement to bother with something if there is no data supporting it. I believe it's an important tool for science to filter out what deserves attention and what is bound to lead nowhere because it's not knowledge based. I don't use that as a filter to tell what is true or not, data will help me weight in on this so if I have none, I usually won't have a clue(maybe some assumptions ^_^).

beyond that, our decisions don't have to rely exclusively only on data, that's what makes us humans and different from each others. but arguments and even more so, claims about what happens or what could happen in the real world, should be founded on data! not on gut feelings, insecurity, and desires for something to be true.
people don't want to use lossy format, good for them, I wish for everybody to enjoy music no matter the format and resolution they use. it really has nothing to do with me TBH. I'll do what I want, you do the same, everybody's happy. but if somebody brings up some claim of audibility to justify his format choice, then IMO he should be prepared to back up his claim or to shut up about it. maybe it doesn't look nice, but nobody asked or forced him to make that statement. he could have said that he used a format because he wanted to and we would all have been like: "hmmm, ok". if he decides to come with a justification that involves objective reality, he opens the 24th level of hell where all the skeptics and objectivists reside, and he has only himself to blame for that.

and yes different encoders can do a bunch of things differently, even more so when you can add lines of command. and yes over the years codecs have changed. I always wish for us to stop talking in vague concepts and tackle actual situations with a lot of specific details. that's what this topic should IMO be about. actual situations, actual experiments, and then we discuss the errors, how the results align with previous tests(or not) and what to do of that new data. talking about lossy vs lossless in general without even bothering with bitrate is a waste of time IMO.

 

[bigshot]

In the end, I'm perfectly happy to accept the conclusion that .05% of the people on this planet "might" be able to reliably hear the difference between a 256kb lossy file and a lossless one.

I'd like to take the "might" out of that sentence and know for sure. But whenever I find someone who says they can definitely hear a difference, they suddenly get cold feet when it comes to doing a controlled listening test. They start coming up with excuses and claim that tests don't prove anything. That tells me where they're coming from quite clearly. I would be delighted to find someone who can actually do this. I don't know if I ever will. The audiophile world is full of generally accepted concepts that no one has ever bothered to prove. They just assume that it exists because other people tell them it does.

 

[Sgt. Ear Ache]

I'd like to take the "might" out of that sentence and know for sure. But whenever I find someone who says they can definitely hear a difference, they suddenly get cold feet when it comes to doing a controlled listening test. They start coming up with excuses and claim that tests don't prove anything. That tells me where they're coming from quite clearly. I would be delighted to find someone who can actually do this. I don't know if I ever will. The audiophile world is full of generally accepted concepts that no one has ever bothered to prove. They just assume that it exists because other people tell them it does.

yup, you and I know the truth.

 

[KeithEmo]

The author actually drew TWO conclusions....

One conclusion was that "we" - presumably referring to some sort of average listener - cannot "usually" tell the difference.
The other conclusion was that at least one test subject COULD reliably tell the difference.
And, from this, he concluded that apparently SOME people can hear a difference SOME of the time.

There is no contradiction.

The problem, however, is in the context of THIS PARTICULAR FORUM.
Even though it isn't being explicitly stated, most of us here aren't really concerned with "what matters to most typical iTunes customers".
If that was the case, then we already know that "most iTunes customers" and "most Spotify customers" and "most Sirius XM customers" find lossy compression to sound quite acceptable.
(Although, as I mentioned in another post, many of them probably don't have a specific opinion of whether it sounds exactly like the CD or not - they simply consider it to sound "good enough to satisfy them".)

In contrast, we are concerned with "what matters and is audible to a specific segment of the market who consider themselves to be audiophiles".
We assume these are the people who have the best equipment, listen the most carefully, and are the most critical of even the tiniest audible differences.
These are also the people who most demand that we speak in absolutes.
(You're not going to convince me, or most other audiophiles, that "I won't hear something because 'most' people can't hear it.")

Yes, the author drew a conclusion which isn't really supported by his own results. The whole point of calculating specific p-values to get a probability that a difference has been found, rather than resorting to eyeballing shapes of distributions, and choosing an arbitrary p-value of 0.05 as a limit. I would be interested in knowing the author's background.

 

[bigshot]

yup, you and I know the truth.

KeithEmo does too. He took my listening test and had the opportunity to listen to it under whatever normal listening conditions he wanted to. Before he took the test, he admitted to me that he probably wouldn't be able to discern high data rate lossy from lossless, and after he got the results of the test, he admitted he couldn't hear a difference trying as hard as he could. He knows as well as we do that high data rate lossy is audibly transparent. He just likes the attention that creating theoretical arguments bring to him. He isn't the only one. There are a several people who come to this forum to turn the spotlight on themselves and perform.

I'm interested in facts and experiments that will lead me to improvements in the sound quality of my home audio system. I'm not interested in assisting someone with their grandstanding. So I simply cut to the chase and ask for proof that someone can tell the difference and offer to help prove it by helping them do a controlled test. When I make that offer, all I get is crickets. They move on to arguing with someone who will engage with their little "what if" dog and pony show. That's fine with me as long as everyone enjoys that game. I can cheerfully skip past all the time wasting blather. But every so often, I'll pop back in with my put up or shut up offer, knowing full well that they'll do neither.

It's up to me to remind folks every so often that we're here in Sound Science to talk about ways to use science to find ways to improve the sound of recorded music in our homes, not put on clown shows to draw attention to ourselves.

 

[KeithEmo]

I absolutely agree.

However, if that fellow can run his sub-four-minute mile one fraction of a second faster wearing those shoes.....
- even if they only make the slightest difference for him and one or two of his closest competitors
- and even if they won't help you or I

Then the manufacturer is NOT lying, or being unreasonable, when they claim that "their shoes are superior because they help some people run faster".
Likewise, a Formula 1 racing car really is faster than my Nissan, regardless of whether you or I could actually safely drive it faster than my car's top speed or not.

And, if you're foolish enough to believe that purchasing a faster car will make you a better driver, to me that seems like an error in judgment on your part.
If you watch that TV commercial, you'll notice that they'll SHOW you some video of exceptional athletes wearing their shoes...
However, they won't promise that the shoes will make you run faster...
They'll leave it to you to exercise typically poor human judgment and subconsciously associate their shoes with your athletic performance.
(They'll probably say something like "our shoes will enable you to perform your best".)

Likewise...
- A lossless file will deliver all of the original information to you...
- A lossy file will omit some of the original information...

It is up to the individual user to decide whether they consider discarding some of the original information in return for saving storage space to be "acceptable" or not.
(And, clearly, many audiophiles do not consider it to be acceptable to do so, whether based on careful listening tests, or "just on the principle of the thing".)

I should note, however, that the terms "lossy" and "lossless" are not some weasel words thought up by someone's marketing department to mislead consumers.
Both terms have specific technical meanings in terms of data storage that completely predate digital audio...
And they are in fact being applied accurately according to their defined meanings...

You are welcome to decide to choose to use lossy compressed files "because you just don't like the idea of using extra storage space that won't make them sound any better to you".
However, in return, I expect you to respect my right to choose lossless compression "because I just don't like the idea of throwing away part of the music - whether I can hear the difference or not".

yes I understand that. Same goes for Kemo apparently.

Sure, there's humans who can run a sub-4 minute mile. But don't nobody better try and sell me a pair of shoes claiming they will make ME able to run a sub 4 mile...

 

[Sgt. Ear Ache]

You are welcome to decide to choose to use lossy compressed files "because you just don't like the idea of using extra storage space that won't make them sound any better to you".
However, in return, I expect you to respect my right to choose lossless compression "because I just don't like the idea of throwing away part of the music - whether I can hear the difference or not".

yeah, I don't have any big issue with that statement at all...

 

[KeithEmo]

Just to be accurate here....

I freely "admit" that, with many files, I probably will not be able to hear a difference.
However, I do not at all agree that I believe that would be true with ALL recordings.
I see this as an important distinction (perhaps you do not).

1) Not all files start out at equal quality.
2) All lossy compression schemes work better on some types of content and less well on others.
3) As with many people, I am much better at noticing differences on content I am extremely familiar with.)

And, yes, I do like drawing attention to the fact that overly enthusiastic generalizations can come back to bite you in the butt later...

KeithEmo does too. He took my listening test and had the opportunity to listen to it under whatever normal listening conditions he wanted to. Before he took the test, he admitted to me that he probably wouldn't be able to discern high data rate lossy from lossless, and after he got the results of the test, he admitted he couldn't hear a difference trying as hard as he could. He knows as well as we do that high data rate lossy is audibly transparent. He just likes the attention that creating theoretical arguments bring to him. He isn't the only one. There are a several people who come to this forum to turn the spotlight on themselves and perform.

I'm interested in facts and experiments that will lead me to improvements in the sound quality of my home audio system. I'm not interested in assisting someone with their grandstanding. So I simply cut to the chase and ask for proof that someone can tell the difference and offer to help prove it by helping them do a controlled test. When I make that offer, all I get is crickets. They move on to arguing with someone who will engage with their little "what if" dog and pony show. That's fine with me as long as everyone enjoys that game. I can cheerfully skip past all the time wasting blather. But every so often, I'll pop back in with my put up or shut up offer, knowing full well that they'll do neither.

It's up to me to remind folks every so often that we're here in Sound Science to talk about ways to use science to find ways to improve the sound of recorded music in our homes, not put on clown shows to draw attention to ourselves.

 

[Phronesis]

I understand the motivation to simplify things, reach a general conclusion, and move forward based on that conclusion without looking back and revisiting it. In practical decisions, that's often what we do, and need to do. Making things black and white can make life easier and more psychologically comfortable.

But if we're going to approach this topic scientifically, rather than pragmatically, we do need to delve into a lot of messy details, question a lot of things, and accept that general conclusions may amount to an oversimplification.

With my audio purchases, I mostly wear my pragmatic hat, but here in Sound Science I mostly wear my scientific (not engineering) hat. Given that reference to science is nearly banned in head-fi outside of Sound Science, I think we should vigorously push for a scientific approach here. Otherwise, it just devolves into the typical us versus them battle of objectivists versus subjectivists (I don't put myself in either category).

 

[bigshot]

Provide me with a lossless track that you think would reveal lossy compression and I’ll set up another test for you. The tracks I used had a full frequency response and massed choral voices, which are two giveaways for comprsssion artifacting. The tracks were selected by a golden eared audiophile who claimed that she could easily detect a difference with these tracks. Above 192, she couldn’t. I’ve conducted tests like this with a variety of different types of music. At 256, modern codecs become transparent. If you have a better test track, please provide it.

I don’t think you know much about this subject.