Jan 9, 2019 at 12:35 AM Post #11,896 of 19,075
Perhaps you have a short memory......

There was a time when "nobody could tell the difference between a cylinder recording and a live performer".
Then people insisted that vinyl "was so close to perfect that there was no point in looking for improvement".
Then we were told that most people couldn't tell "is it live or is it Memorex" (referring to cassettes).
Then there was a time when "most people were sure that 128k MP3 files were audibly perfect".
(Note that the developers of the MP3 compression process never made claims beyond that "most listeners" wouldn't notice a difference with "most music".)

I agree.... there's nothing to suggest that the methodology itself is flawed.
However, there really have NOT been "comprehensive widespread tests".
(It seems reasonable to suggest that, at least for now, no single group has both the resources and the inclination to perform those tests.)

I should also note something about human nature - which is that we learn and evolve in our ability to recognize things.
In one very early test, an audience was unable to tell the difference between a live performer and a cylinder recording.
HOWEVER, it is important to note that the audience who participated in that test had no experience whatsoever with recorded music... having only ever experienced live performances.
To them, that poor quality recording was 'the closest thing they'd ever heard to a live performance - other than a live performance".
A modern audience would have been quick to notice the surface noise, ticks and pops, and distortion of the cylinder recording as "obvious hints that it was a record".
In short, we have LEARNED that ticks, pops, and hiss are artifacts often associated with mechanical recordings like vinyl records.

This strongly suggests an interesting avenue of research.

After doing careful tests to determine whether listeners can detect differences between lossless and lossy compressed files (using a particular level and sort of compression).
We should take one group of listeners and "teach them the differences".
This would be accomplished by allowing them to listen to both versions of several different files - while pointing out the differences that exist "so they know what to listen for".
("Here's what those two files look like on an oscilloscope. Do you see the differences? Do you hear a difference that seems to correlate with the difference you see?")
We should then re-run the test, to find out whether our "taught" group has in fact LEARNED how to better notice and recognize the differences between the files.
We could then perform a double blind test to determine whether our "taught" group has actually LEARNED to be more accurate in distinguishing lossy files - or not.

We aren't born knowing how to tell a counterfeit painting from an original - doing so is a skill that we learn - and that some people have a particular aptitude for while others don't.
And, for those of us who lack that skill, the differences noticed by skilled experts are often invisible or very difficult to detect until they are pointed out to us.
Why would we assume that the ability to recognize the small differences caused by lossy compression shouldn't have a similar characteristic?


So many words and so little actual refutation. Wax cylinders, alledged testing from the 1920s with no references, and 70s advertising slogans - seriously?

Still waiting for you to present actual evidence rather than blindly lobbing grenades in the hopes of actually hitting something. It’s almost as if you have a financial stake in avoiding the data available from existing testing...
 
Jan 9, 2019 at 4:55 AM Post #11,897 of 19,075
[1] I can't propose a test better than a controlled blind test, but that doesn't mean that such a blind test is good enough to draw the kinds of sweeping conclusions which are sometimes asserted in Sound Science.
[2] There can be such a situation as "we're not sure, but leaning this way.
[3] I'm comfortable with saying that the blind tests conducted so far are probably sufficient to conclude that any audible differences between lossless vs somewhat lossy, DACs, amps, and cables are very likely to be very subtle at most for the vast majority of listeners. (The qualifiers I italicized make this sort of statement acceptable to me ...)

1. But we don't ONLY use "such a blind test" to draw the kinds of sweeping conclusions. Lossy compression algorithms have been tested extensively over a period of at least a couple of decades (and have been refined over that period) and not just with "such a blind test" but with countless and far more robust controlled double blind/ABX tests.

2. Yes, there can be but this isn't one of those situations!

3. You are of course entitled to your opinion but this is the sound science forum and what you personally are "comfortable with saying" or what "sort of statement is acceptable to you" is irrelevant. There is no evidence to support the assertion that the differences are "very likely to be very subtle at most", all the reliable evidence indicates that the differences are "very likely to be completely inaudible". If there were audible but very subtle differences we would see some evidence of that, we would see a small minority of test subjects who could reliably identify a difference and we would expect to see a somewhat higher percentage when the test subjects were trained listeners using the highest quality (most accurate) reproduction systems/environments but we don't see this, we don't see ANY percentage of test subjects who can reliably identify a difference, even among the most highly trained listeners with the best reproduction equipment. The actual situation, as far as the evidence is concerned, is therefore: "We cannot be absolutely sure about anything but we can be reasonably certain".

[1] But, for starters, you will have to test it under ALL "normal listening conditions".
[2] You should probably also include a few of the proprietary surround sound modes offered by the major manufacturers.
[3] These are all "normal listening conditions" used by large numbers of "typical listeners".

1. Clearly that statement is false. It's clearly impossible to test every combination of consumer equipment and listening environment, let alone test every consumer with every combination of equipment/environment. There can be no absolute proof, only a weight of evidence.

2. If a consumer wants to take a lossy (or lossless) recording and completely change the fidelity/purpose according to their own preferences, that's entirely up to them but it's not a lack of performance if it's used for a different purpose than it was designed for. Using your own analogy, a Formula 1 car does NOT have "better performance" than a Nissan Versa, if you want to go to a store (with your wife or kids, and buy something) a Formula 1 car has no performance at all, let alone "better performance"!

3. No they're not, they are abnormal listening conditions, not what lossy codecs were designed for. Additionally, it's not done "by large numbers" of "typical listeners", it's done by an extremely small minority.

[1] However, this leads us right around the circle, and back to a very basic question: WHICH claim is the one that we are supposed to reject without proof?
[1a] If we accept the criterion that "we shouldn't accept ANY claim without proof" then we simply have two unproven claims.
[1b] It is a logical fallacy to assume that either of those claims is some sort of "default assumption"; by your criterion there is no such thing as a default assumption.
[2] There was a time when "nobody could tell the difference between a cylinder recording and a live performer".
[2a] Then people insisted that vinyl "was so close to perfect that there was no point in looking for improvement".

1. It does indeed unfortunately lead us right around the circle again, and back to the very basic fallacy which you keep repeating! As there is and cannot be any absolute proof, the claim "we are supposed to reject" is the claim WHICH has no supporting reliable evidence in favour of the claim which has overwhelming supporting evidence.
1a. You can accept any criterion you choose but this is the Sound Science forum (not the "KeithEmo's Criterion" forum) and therefore we "accept the criterion" of science, which is that we accept the claim which is supported by reliable evidence and reject the claim which has none! For example, Evolution has not been proven, neither has Creationism but scientifically we do NOT "simply have two unproven claims". Scientifically, the claim of Evolution is accepted (without proof) because of it's weight of reliable evidence (and the lack of evidence for creationism).
1b. No it's a logical fallacy not to accept the "default assumption" of say Evolution.

2. No there wasn't! You keep contradicting yourself, you keep going on about proof and then make an assertion without even any supporting evidence, let alone proof. Why is that?
2a. What people? Clearly your statement does not apply to all people and specifically excludes entire groups of people (scientists and engineers for example). Why was the RIAA curve invented, why was digital audio invented, was no one able to measure anything 80 years ago?

And off we go again, round and round in circles!

G
 
Jan 9, 2019 at 6:37 AM Post #11,898 of 19,075
lol, I love the suggestion that its necessary to prove the difference is inaudible under every possible (imaginary) listening condition! Why? How bout if we just go with the simplest, by far most common listening situation first. A pair of headphones and a laptop or other suitable source? Then, if you want to try and design a situation under which you can reliably tell the difference between a high br lossy file and a lossless one simply by listening to them you go ahead and do that. Nobody is trying to argue that the files aren't different. We're saying that under the conditions that 99% of people are using them they are functionally indistinguishable and that - by extension - lossless files are by no means a necessity for audiophile enjoyment. In other words, selling gear on the basis that lossless files make a substantial (or really ANY) difference in terms of sound quality (again, compared to high bitrate lossy) is a lie.
 
Jan 9, 2019 at 8:18 AM Post #11,899 of 19,075
lol, I love the suggestion that its necessary to prove the difference is inaudible under every possible (imaginary) listening condition! Why? How bout if we just go with the simplest, by far most common listening situation first. A pair of headphones and a laptop or other suitable source? Then, if you want to try and design a situation under which you can reliably tell the difference between a high br lossy file and a lossless one simply by listening to them you go ahead and do that. Nobody is trying to argue that the files aren't different. We're saying that under the conditions that 99% of people are using them they are functionally indistinguishable and that - by extension - lossless files are by no means a necessity for audiophile enjoyment. In other words, selling gear on the basis that lossless files make a substantial (or really ANY) difference in terms of sound quality (again, compared to high bitrate lossy) is a lie.

Even if that conclusion is that correct (my guess is that it’s correct), I’m not aware of it having been established based on testing which meets scientific standards. We need to make the distinction between a belief and the support for the belief. We shouldn’t be unscientific in Sound Science.

If there are good studies on this which I’ve missed, perhaps someone could provide links.
 
Last edited:
Jan 9, 2019 at 8:36 AM Post #11,900 of 19,075
the thing that hasn't been established via scientific testing (by people making the claim that they CAN hear a difference) is that there IS any audible difference! LOL. The fact that it's incredibly tough (as in impossible) to hear a difference between high BR lossy and lossless has definitely been well-established and the suggestion that it hasn't is really pretty much just willful obfuscation at this point...
 
Jan 9, 2019 at 8:43 AM Post #11,901 of 19,075
the thing that hasn't been established via scientific testing (by people making the claim that they CAN hear a difference) is that there IS any audible difference! LOL. The fact that it's incredibly tough (as in impossible) to hear a difference between high BR lossy and lossless has definitely been well-established and the suggestion that it hasn't is really pretty much just willful obfuscation at this point...

If something hasn't been established by scientific testing either way, you don't make a default assumption, you say "I don't know." To the extent that some testing has been done, the quality and quantity of that testing can be evaluated and people can draw their own conclusions about what the testing suggests. People may draw different conclusions, there wouldn't necessarily be a consensus. So again, it would be helpful to gather up all the relevant links and post them here so that people can review the testing for themselves. Otherwise, we're just debating without reference to evidence and using tactics like asserting burden of proof, which is fine for law, but isn't really how science works.
 
Jan 9, 2019 at 8:46 AM Post #11,902 of 19,075
There is no evidence to support the assertion that the differences are "very likely to be very subtle at most", all the reliable evidence indicates that the differences are "very likely to be completely inaudible".

I was careful with my wording. "Very subtle at most" includes the possibility of "complete inaudible," while setting a tight upper bound which rules out "obvious" and "night and day" differences.
 
Jan 9, 2019 at 8:51 AM Post #11,904 of 19,075
When "some testing" (quite a lot actually) has been done that shows one result, and very little testing (essentially none) that shows an opposite result...we can surmise quite a lot. This is neither a courtroom, nor a science lab.

Please provide links and then we can all discuss.
 
Jan 9, 2019 at 9:11 AM Post #11,906 of 19,075
Indeed.

And since nobody in this forum, or anywhere else, has actually conducted a thorough, reasonably well conducted, and valid test - all we've got is words.
So I guess you'll have to live with that.

The thorough testing that we may wish existed HAS NEVER BEEN DONE.
The data from existing tests is VAGUE AND ONLY MARGINALLY RELEVANT.
Wishing it were otherwise will not make it so.

If you disagree.....

Then show me the test where a statistically useful number of participants, listening to a reasonably comprehensive selection of test samples, on a reasonably comprehensive cross section of all available equipment, under a reasonable fraction of the listening situations most listeners today use, has produced a result demonstrating that none of those listeners was able to tell the difference between lossy compressed files and the originals. Please provide the demographic data on your test subjects, a full list of the test samples you used, full documentation on the performance of the test equipment you used, and full justification why both were properly representative of "all the listeners and music available today". You must provide both statistical results showing that "a statistically significant number of listeners were unable to distinguish the difference, and proper confirmation that no few outliers were able to do so. When you have that, please tell me which encoder you used, and which settings you used, so we can have someone else duplicate your test and confirm the results.

After you do that, THEN we will "have something beyond lots of words on which to base our judgment".

NO REFUTATION IS REQUIRED BECAUSE THERE IS NOTHING TO REFUTE.
All we have are a few isolated, small scale, extremely limited tests - which produced results that failed to conclusively disprove the assertion that most people don't notice the difference.


So many words and so little actual refutation. Wax cylinders, alledged testing from the 1920s with no references, and 70s advertising slogans - seriously?

Still waiting for you to present actual evidence rather than blindly lobbing grenades in the hopes of actually hitting something. It’s almost as if you have a financial stake in avoiding the data available from existing testing...
 
Jan 9, 2019 at 9:22 AM Post #11,907 of 19,075
Actually, even though the test was rather limited, I thought their results were interesting.....

Out of a bunch of participants, they did in fact find one listener who was reliably able to recognize the difference....
So they DID prove that the assertion that "nobody can reliably hear the difference" is UNTRUE after all.
(And the iTunes sales and marketing department will be very interested in the statistical data that most users could not.)

If we consider the results of that test to be valid, then I guess now we know that at least some people can hear the difference after all, and we can move on.
It should also be noted that they managed to find that listener with only a limited number of tests and a limited number of participants.
(If it were to turn out that 1/500 listeners could hear a difference - that would be a quarter of a million who can in the USA alone.)

I know from the outset that anything I provide will not meet your high standards of scientific scrutiny of course, but here's one link...

https://cdvsmp3.wordpress.com/cd-vs-itunes-plus-blind-test-results/


now, if you can provide one link to a similar test that has an opposite result?
 
Jan 9, 2019 at 9:45 AM Post #11,908 of 19,075
I don't disagree at all - I agree with you that: "The majority of listeners who listen to stereo music in pure stereo will probably not notice any difference".
I wouldn't disagree at all with that assertion.
(However, you are incorrect in one thing; some people on this forum are trying to argue that "high quality lossy files are never audibly different".)

I also disagree with your guess about the percentages...
A significant percentages of the people these days listen to their music on various surround sound and home theater systems.
Likewise, many people with computers, and many separate headphone amps, include various "headphone processing" options of one sort or another.
(And many of those processors operate by acting on minor channel differences, or minor phase differences, which are altered by many lossy decoders.)


As I mentioned, here at Emotiva, we sell both stereo and surround sound equipment.
And, while we haven't kept count, I can tell you that, based on their support questions, a large percentage of our home theater customers listen to all their music in surround sound.
And I suspect Dolby Labs might dispute your suggestion that using their Dolby headphone plugin is NOT "a common listening situation".
(If you believe their counter - the plugin for "Dolby Atmos and Dolby Atmos Headphone for Windows" has been downloaded over 35,000 times from CNET alone.)
Or you might consider the lively debates about "which upmixer works better with stereo content" to suggest that in fact a lot of people use one or the other in that situation.

It might be interesting to do a survey on Head-Fi to find out how many headphone listeners use some sort of headphone enhancement or cross-mix plugin or gadget.
(And it might be equally interesting to actually test what effect various lossy compression methods and settings have on the AUDIBLE outputs of each.)

lol, I love the suggestion that its necessary to prove the difference is inaudible under every possible (imaginary) listening condition! Why? How bout if we just go with the simplest, by far most common listening situation first. A pair of headphones and a laptop or other suitable source? Then, if you want to try and design a situation under which you can reliably tell the difference between a high br lossy file and a lossless one simply by listening to them you go ahead and do that. Nobody is trying to argue that the files aren't different. We're saying that under the conditions that 99% of people are using them they are functionally indistinguishable and that - by extension - lossless files are by no means a necessity for audiophile enjoyment. In other words, selling gear on the basis that lossless files make a substantial (or really ANY) difference in terms of sound quality (again, compared to high bitrate lossy) is a lie.
 
Jan 9, 2019 at 10:02 AM Post #11,909 of 19,075
I might suggest that, if you consider "the big picture", most people have historically believed, and continue to believe, that various audio formats do sound different.
(So, in fact, it is the assertion that they don't sound audibly different that constitutes "lobbing the grenade" - and not the other way around.)

It would be VERY interesting to survey all of the customers of a major service that uses lossy compression - like iTunes.
I would be curious to know how many of their customers are actually convinced that the lossy compressed music they're listening to is "indistinguishable from CD quality"...
And how many would say that, even though they personally don't care much either way, they just assume, or are outright convinced, that the CD really is a little bit better...
And, for that matter, it would be interesting to find out how many assume that vinyl is still better than both...

Always remember that, statistically, the majority of listeners are not "convinced that the lossy compression used by someone like iTunes sounds just as good as a CD".
The majority of listeners simply don't care all that much and haven't given it much thought either way.

So many words and so little actual refutation. Wax cylinders, alledged testing from the 1920s with no references, and 70s advertising slogans - seriously?

Still waiting for you to present actual evidence rather than blindly lobbing grenades in the hopes of actually hitting something. It’s almost as if you have a financial stake in avoiding the data available from existing testing...
 
Jan 9, 2019 at 10:06 AM Post #11,910 of 19,075
Actually, even though the test was rather limited, I thought their results were interesting.....

Out of a bunch of participants, they did in fact find one listener who was reliably able to recognize the difference....
So they DID prove that the assertion that "nobody can reliably hear the difference" is UNTRUE after all.
(And the iTunes sales and marketing department will be very interested in the statistical data that most users could not.)

If we consider the results of that test to be valid, then I guess now we know that at least some people can hear the difference after all, and we can move on.
It should also be noted that they managed to find that listener with only a limited number of tests and a limited number of participants.
(If it were to turn out that 1/500 listeners could hear a difference - that would be a quarter of a million who can in the USA alone.)

Odd how you toss away every other test based on your absurd and unachivable criteria but seem more than willing to accept this one without applying the same.

Not that I’m surprised.
 

Users who are viewing this thread

  • Back
    Top