[ursdiego]

Hello Guys!
 
Is anyone aware of any real, high quality, representative (double-)-blind-tests done to evaluate whether High Resolution Audio does sound better, equal or worse than CD-standard?
 
PLEASE no speculative theoretical answers here - the internet is full with them all over, the same for good sciense based explications. However, real blind tests are rather rare.
 
The only one I am awaree of is this:
http://www.aes.org/e-lib/browse.cfm?elib=14195
"Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback"
 
The conclusion there is rather disillusioning: Not only that, according to this test, human do not seem to be able to determine a difference between CD-Standard and Hi-Res-Audio, but even worse, they did not even notice the digital-to-analog and back to digital and again back to analog - loop in between - something that must struggle us quite a bit.
 
Any other tests that can be cited? I would love to see one that proves that Hi-Res-Audio makes sense... (real evidence, representative, double-blind-tests, based on hi-quality-downsampled streams)
 
Thanks for any links! Cheers!

 

[wink]

This needs to go to the Sound Science forum.

 

[RRod]

It's pretty easy to answer the question of "can I detect a difference blind on my own playback chain?". Load up the foobar ABX comparator and see what you can hear.

 

[jcx]

there turned out to be some problems - Meyer-Moran relied on "Hi Res" source that was marketed as such - later examination of spectrographs showed some of their "Hi Res" source was upsampled from 44.1
 
naturally critics want to dismiss the whole study on the grounds of some of the source being mislabeled/marketed by fraudsters
 
others are amused that the naïve subjectivist audiophile reviewers hadn't pointed it out previously based on their uncontrolled listening tests with their golden ears that perhaps as much as half the early "Hi Res" releases didn't have any >20 kHz content
 
there have been a few "suggestive" studies - some by people pushing alternative digital reconstruction filtering - no agreement so far as I know
 
Bob Katz is a pro audio engineer that writes on mixing/mastering - his latest book has a bit on his personal and fellow pro's private listening based beliefs that 44.1 isn't quite adequate - but still no peer reviewed publications

 

[RRod]

There's a quote from Meyer on criticism here. One of the comments on that page sums it all up quite nicely:
 

 Having only superficial knowledge of the background of this debate / argument I can only say that Meyer’s comments above make sense; regardless of what they actually set out to prove originally or what people thought they set out to prove. A well recorded / mastered red book CD may only be differentiated from higher ‘resolution’ (how I hate this word) recordings / masterings by extremely subtle differences in HF and, under ideal conditions, by the noise floor. Which nicely fits with the theory. The rest is industry and ‘high-end’ press mumbo jumbo. I don’t think there’s much material disagreement here. Peace!

Quote:

Bob Katz is a pro audio engineer that writes on mixing/mastering - his latest book has a bit on his personal and fellow pro's private listening based beliefs that 44.1 isn't quite adequate - but still no peer reviewed publications

 
The late-great John Eargle had a reasonable view on this. A quote by him from an Audio Critic article back in '92:
 

  I'm very happy to say that most of the problems of digital have been dispatched—or essentially dispatched—in recent times. There are so many improvements in converters being made today that I really haven't got a problem with it. In a purely intellectual sense, I think we all wish that we had a higher sampling frequency, just for the sake of maybe that very, very tiny percentage of the population who can still hear a difference at 20 kHz. I don't really object to the 16-bit word length because we now have a very benign way of handling the bottom end of the scale. That's by dithering the signal and getting almost analog-like performance at the low end [of the dynamic range], where the signal can be heard to fade into a noise floor without any abrupt things going on, and no more of this nonsense that we heard earlier of the signal disappearing. You know, the old stuff about the reverberation disappearing when it got down to the least significant bit. I doubt that ever really happened anyhow because there's always been dither in the form of amplifier/preamp noise or even room [noise]: That's not ideal dither, but it can be effective enough to keep some of these things from happening. In any event, I feel that the quality of conversion today—the low-bit conversion techniques, noise-shaping methods—really do create a superior medium, one that I have absolutely no problems with and one which I'm confident—despite what some people say—can be cloned ad infinitum. As long as all the errors are detected and corrected, there's no reason cloning can't go on forever.

 

[ursdiego]

Thanks for your replies. However, there is no citation of a representative study that would include blind (or better double-blind) listening tests with good material (good means: native "hi res" - recordings on one hand, versus good material that was properly (!) down sampled to 44.1 or 48 khz, and all of that played on a chain that is as good for high resolution as it is for standard resolution).

Maybe there is no commercial interest in working on such a study… or in publishing its results...? I have paid quite some money for "hi-res" traks, but I start doubting if I should rather save my bucks and buy more music in lowever khz-rates... I have done some tests myself with original 24bit/96khz-tracks versus downsampled versions, but of course these were no proper blind tests, so I don't want to elaborate on my on subjective impressions here.
 
So the question remains open: Any GOOD study including blind listening tests showing in a representative manner whether "high resolution" really sounds better than CD-sample rates?

 

[RRod]

  Thanks for your replies. However, there is no citation of a representative study that would include blind (or better double-blind) listening tests with good material (good means: native "hi res" - recordings on one hand, versus good material that was properly (!) down sampled to 44.1 or 48 khz, and all of that played on a chain that is as good for high resolution as it is for standard resolution).

Maybe there is no commercial interest in working on such a study… or in publishing its results...? I have paid quite some money for "hi-res" traks, but I start doubting if I should rather save my bucks and buy more music in lowever khz-rates... I have done some tests myself with original 24bit/96khz-tracks versus downsampled versions, but of course these were no proper blind tests, so I don't want to elaborate on my on subjective impressions here.
 
So the question remains open: Any GOOD study including blind listening tests showing in a representative manner whether "high resolution" really sounds better than CD-sample rates?

 
No large-scale controlled study post-M&M exists that I know of. There is zero incentive for the hi-res seller's to show failed blind tests of their material versus a good 16/44.1 downsampling. I assume those in the AES without skin in the game just don't think it's worth their time.
 
Again I'll suggest using something like the foobar ABX plugin to try and do an actual controlled DBT of your ability to differentiate formats. That would settle the answer for the current combination of your ears and your equipment.

 

[ursdiego]

@RRod yes I did that. Using “Qobuz” “master recordings”, natively hi-res recorded hi-res tracks, and comparing the down-sampled ones to the originals. The tests were not really blind though, and I was trying hard to justify my investment into a decent hi-res DAC and hi-res tracks. But I was honest enough to realize that "hi-res" was at least far far away from what we are supposed to believe. My chain is certainly sufficiently high quality to take advantage of whatever advantage there may be. Not sure though if the bottleneck is at my ears, although they have been checked recently by a doctor and turned out to hear pretty much everything as the ears of an average 25 years old - even though I am quite some years older. So, I wonder if this was just my subjective impression, and this is, why I ask these questions here.
 
I do have a supposition... I believe that not all DACs react the same to lo-res and to hi-res material. I would even argue that it is often on low- or mid-fi chains, that hi-res may sound better, maybe because some DAC-chips (and other digital parts), that are made to support high frequencies, simply work better and more accurately at these higher frequencies. And up-sampling algorithms may be better implemented on expensive equipment. This would explain, why many mid-fi owners (eg here on the forum) are so keen to get hi-res tracks, while I don't really feel that hi-res makes a fundamental difference.
 
However, this is just speculation, and that is precisely, what I don't wish to see in this thread, but instead of this, hard, reliable evidence would be nice...

 

[castleofargh]

the thing is, you would usually need to be more like maybe 10 or younger to be able to discriminate stuff at 20khz. and even then you would need materials that really make use of those frequencies. when a recording engineer who certainly can't hear 20khz is mastering an album adding tracks to one another dealing with loudness differences for each, making sure to EQ some frequency for some instrument so that another one can be heard clearly etc,  how did he know if he did something ok in the ultrasounds?
 
and bit depth... well I don't think there is a debate over this, almost no song will use the dynamic of a CD, so using more is really just pushing the quantization noise lower. which is cool on principle. but then for it to have a meaning, we need a sound system that won't generate an even louder noise. and that's not a given. same with the noise in the room.
 
so yes highres has the potential to carry more data, is that extra data important to you to the point where you're ready to pay for it? IMO only you can answer that.
 
about papers on highres, there won't be something as global as cd vs highres, but maybe some papers on filters in DACs, or ultrasounds, I remember an old one on SACD vs DVD or something like that. stuff that can involve using a different resolutions one way or another. maybe if someone is an AES member he can tell what is available and if it's worth paying to get the paper?

 

[spruce music]

Actually there are two such tests done to better standards than Meyer-Moran.  I will see if I can find a place you can read about them and link them.  Both were done in the last few years.
 
In one, using superb gear (dcs gear I think), they recorded concurrently in 44/24 and 88/24 using minimal mikes.  Then played it back for university music students.  They also did downsampling.  The down-sampled files were perceptible at near 95% levels, but 44 was heard same as 88.  This leading one to think the downsampling was audible as the native 44 wasn't audibly different than native 88.  I don't remember which downsampler they used, but a few of these done in software recently are pretty much beyond reproach imo.  IMO some such downsampling algorithms are not blameless and you wonder why they are still in use.
 
The other test I have read was similar, and done with a large number of participants.  I think it was 96/24 vs 44/24.  Also used several types of music done as recordings concurrently.  There appeared to be some positive results for one of the simpler recordings (a jazz trio) for some participants.  I disagreed with the interpretation of the results however. They had 7 people of 109 choose at the 95% confidence level it wasn't chance.  They concluded some people hear a difference.  On the other hand you expect to randomly get a seeming positive result 5% of the time.  So of 109 test takers you would thik 5 or 6 might at random pass that level in their choices.  So I was less convinced than the writers of this report.
 
Both of these used high quality recording and playback gear.  The recordings were made without any processing.  And in one you get no real results in another little, or maybe on one particular piece of music.  At best high res is barely, maybe, potentionally sometimes slightly audible is how I would read it.  At best. With lesser systems, older ears, more processed recordings well.....you can almost surely ignore it altogether. 
 
In buying hires remastered material the difference is surely the mastering.  Even re-mastered CD can be a big improvement in some cases.
 
Now let me see if I can dig up some references to these two tests.
 
PS-I do some recording, and 48/24 is all we need best I can tell.  The 24 is just to give us headroom recording. 

 

[spruce music]

Well my memory wasn't too faulty on the details though I don't score an A+.  At least for the first one I found.  Now to find the other.
 
https://www.academia.edu/441305/Sampling_Rate_Discrimination_44.1_KHz_Vs._88.2_KHz

 

[spruce music]

Okay here is a portion of a test which was actually high rate PCM vs DSD which I believe my failing memory got mixed together.   Still might be of interest.
 
http://audiosciencereview.com/forum/index.php?threads/aes-paper-digest-dvd-audio-versus-sacd.150/
 
 
 
This thread on the same web forum has a link to a test I had not seen before now.
 
http://www.audiosciencereview.com/forum/index.php?threads/aes-paper-digest-differences-among-several-high-sampling-digital-recording-formats.151/

 

[sonitus mirus]

  Well my memory wasn't too faulty on the details though I don't score an A+.  At least for the first one I found.  Now to find the other.
 
https://www.academia.edu/441305/Sampling_Rate_Discrimination_44.1_KHz_Vs._88.2_KHz

 
If the 44 and 88 could not be identified natively, than it seems that the chosen downsampling process was fudging the files.  To remove the gear from potentially being the culprit, I normally upsample the lower rate back up to the higher format when I ABX with Foobar, just in case the DAC handles one rate differently from another that could possibly be audible.  But in this case with the native 44 and 88 being too difficult to consistently identify between the two, I don't see the point in the downsampling.

 

[ursdiego]

@ spruce music: Thanks for the references and your A-grade-summaries! These tests sound rather interesting, I will have a closer look tomorrow. And even if a difference is audible to some, it is not yet clear, which one sounds better or more close to the source. As @ sonitus mirus I believe that anyway, it has to be excluded, that differences between sampling rates are audible simply because the DAC handles them somehow differently.

 

[mevans]

The technical discussion is light years over my head.  However, if hobbyists decide to develop a project to conduct blind tests of recorded music, then, to beat a dead horse, maybe it would make sense to test amps (cables? converters?) as well.