[ferday]

i'm not going to put this in the now awful 24/16 bit thread
 
a couple months ago one of my favorite internet audio blogs (archimago) put some files out for testing, 24 bit vs. 16 bit.  i completed the test (rather exhaustively, and it was fun) and totally forgot about checking back for results.  thanks for the efforts archimago!
 
here they are, this should be no surprise to any regulars here in sound science
 
http://archimago.blogspot.de/2014/06/24-bit-vs-16-bit-audio-test-part-ii.html

 

[James-uk]

I personally found it difficult to remain objective in this test . I already knew the result. So to try and hear a difference that doesn't exist is a strange thing to try and do. I did pretend there might be a difference and I spent an exhausting amount of time casual listening / intensely focusing on every different aspect of sq and potential 'artifacts' etc . I even did several ABX tests using ABX tester for mac. They sounded identical to me and I just guessed a result as expected. The main thing I wanted to see from the results was a clear indication that the 'golden ears' that said they could hear a difference actually couldn't . Hopefully a few more people will realise that 16 bit is plenty good enough. Annoyingly a few will shout from the roof tops that somehow the test was flawed and it means nothing. I have no time for those people anymore , I do however feel sorry for them.

 

[ferday]

curious, how did you know which was which?  i'm assuming you only did the test after the results were posted?
 
i had no idea when i took it and couldn't tell a difference (literally 50/50 over about 100 trials)...but i did lie and say i had "fair confidence" as i knew admitting to guessing would likely get my results excluded from a few categories ha ha ha.  
 
i really appreciated his honest commentary on how the test could be considered flawed (to try and avoid the rooftop shouters) as well as pointing out the tests strengths (random internet group selection, asking about price range/audio experience, etc.)

 

[James-uk]

curious, how did you know which was which?  i'm assuming you only did the test after the results were posted?

i had no idea when i took it and couldn't tell a difference (literally 50/50 over about 100 trials)...but i did lie and say i had "fair confidence" as i knew admitting to guessing would likely get my results excluded from a few categories ha ha ha.  

i really appreciated his honest commentary on how the test could be considered flawed (to try and avoid the rooftop shouters) as well as pointing out the tests strengths (random internet group selection, asking about price range/audio experience, etc.)

Sorry that was misleading.i didn't know which file was which but I knew they would sound identical based on my understanding of 16 bit audio. I would of bet my house on the result . Just like I know the tooth fairy doesn't exists . I can't PROVE it but it's an accepted fact.

 

[James-uk]

I used hd800 with odac / O2 and hdvd800. I've had my hearing tested recently and it's excellent. I had it tested because I thought I had a touch of tinnitus but apparently it's just normal to hear background 'hiss' in a quiet room and I'm focusing on it to much ( the problem with being obsessed with audio and subsequently hearing I suppose!) the point being if there was a difference I should of heard it. Let's just say I'm very happy listening to spotify streaming at 320kbps. I've done numerous a/b checks between it and cd and it's the same to these 32 year old ears. It's saving me a fortune in CDs!!!

 

[cjl]

  curious, how did you know which was which?  i'm assuming you only did the test after the results were posted?

If you really felt inclined to cheat, you could always import the files into something like Audacity and look at the noise spectrum.

 

[ferday]

of course, and this was admitted by archimago...and why he's reluctant to do a similar test with sample rate

 

[ab initio]

Just to clear things up a bit. This test was not designed to determine whether or not people could tell a difference between 16 and 24 bit PCM files with 96kHz sample rates. This was a test of whether or not people could identify which of the files was which of the bit depths. This is a subtle but important difference. In this test, we can conclude that the listeners could not correctly identify a which of the audio tracks were which bit depth.
 
To determine if the listeners could reliably differentiate between 16 and 24 bit files, sufficient controlled ABX data would be required from the participants. Because this data was not collected, this study cannot conclude whether or not the listeners could tell the difference between the bit depths.
 
That said, this certainly is very strong evidence that anybody claiming to hear obvious improvements in audio quality with 24 bit PCM audio is either lying or falling victim to bias.
 
140 independent samples for 3 different tracks, all with ~ 50% correct is very strong evidence that people are guessing at which bit rate is which.
 
Cheers

 

[James-uk]

16/44 is more than enough for any human ears. the good thing about science is it's true whether you believe in it or not . My only concern for the last few years has been if good compression is 'transparent ' to my ears. I have concluded that it is in my own a/b tests. Now all I'm concerned with is good music, great masters and the best possible transducers .

 

[jonbernard]

Just to clear things up a bit. This test was not designed to determine whether or not people could tell a difference between 16 and 24 bit PCM files with 96kHz sample rates. This was a test of whether or not people could identify which of the files was which of the bit depths. This is a subtle but important difference. In this test, we can conclude that the listeners could not correctly identify a which of the audio tracks were which bit depth.

 
I asked Mark Waldrep (Dr. AIX), a proponent of 96K/24bit, what he thought of the test, and he responded
 
More of the same. If the samples that are played don’t exhibit any differences for the quality (in this case dynamic range) that you’re trying the test for, then how can you claim it’s a valid test? In this case, I downloaded the files and the largest dynamic range present was about 60 dB…just 10-12-bits of PCM would more than cover that.
 
That said, I'm still a big fan of Archimago's blog.

 

[Roly1650]

And does Dr. AIX know of any commercial recordings, in any format, that exceed 96dB dynamic range? If he doesn't then there will never be a valid test and his comment is both misleading and pointless.

 

[ieee754]

So what was the point of this test if it didn't prove anything? Most of these audio tests are a waste of time, unless someone is really convinced that he hears a difference and wants to prove it (I think that all audio equipment producers should prove that their stuff make a difference in some ABX testing).

 

[analogsurviver]

And does Dr. AIX know of any commercial recordings, in any format, that exceed 96dB dynamic range? If he doesn't then there will never be a valid test and his comment is both misleading and pointless.

To exceed the dynamic range of 96 dB is next to Mission Impossible. Even assuming the recording chain is capable of meeting and/or exceeding 96 dB dynamic range - it is highly unlikely a place with so low noise can be found. Stax test CDs have tracks partially recorded in anechoic chamber - and these are audibly quieter than recordings in any normal room used for music. Even on Stax test CD, there is a special warning for dynamic range that exceeds - 50 dB; only a couple of tracks out of 99 aciheve this, most notably percussion .
 
IMO more relevant is the frequency response - limited to redbook CD or allowed to go beyond; here, it is more difficult to make such a test, as it is possible to analyze the spectrum with some software and therefore learn which format it is an thus cheat.  Foobar2000 will also reveal the switching from PCM to DSD or vice versa by faint clicks - which at least gives away the switching from one to another took place. "Locking" the test file against any spectrum analyzing software and having so powerful software/processing that Foobar2000 (or any other ABX computer based comparator ) does change different file formats in total silence exceeds present capabilities - at least of most interested people. 

 

[Roly1650]

To exceed the dynamic range of 96 dB is next to Mission Impossible. Even assuming the recording chain is capable of meeting and/or exceeding 96 dB dynamic range - it is highly unlikely a place with so low noise can be found. Stax test CDs have tracks partially recorded in anechoic chamber - and these are audibly quieter than recordings in any normal room used for music. Even on Stax test CD, there is a special warning for dynamic range that exceeds - 50 dB; only a couple of tracks out of 99 aciheve this, most notably percussion .

Thanks for that, but I did know the answer before I asked the question.

 

[castleofargh]

 

Just to clear things up a bit. This test was not designed to determine whether or not people could tell a difference between 16 and 24 bit PCM files with 96kHz sample rates. This was a test of whether or not people could identify which of the files was which of the bit depths. This is a subtle but important difference. In this test, we can conclude that the listeners could not correctly identify a which of the audio tracks were which bit depth.

 
I asked Mark Waldrep (Dr. AIX), a proponent of 96K/24bit, what he thought of the test, and he responded
 
More of the same. If the samples that are played don’t exhibit any differences for the quality (in this case dynamic range) that you’re trying the test for, then how can you claim it’s a valid test? In this case, I downloaded the files and the largest dynamic range present was about 60 dB…just 10-12-bits of PCM would more than cover that.
 
That said, I'm still a big fan of Archimago's blog.

but that's exactly the point here right? before asking if some people could actually hear the benefits of sounds at least 96db under the loudest recorded instrument, let's start with the fact that albums don't even use that much dynamic. what is the point of trying to demonstrate something by using music that isn't what's available to public? I'm not recording a whale, I'm listening to my favorite albums.
in my collection the good stuff I've found was around 60-70db of dynamic, so it does look like archimago tried to use some of the most dynamic samples he owned.
what was done in this test was overall the right call IMO. nothing new, that's right, but multiplying those tests is obviously a necessity when so many people keep being purposedly misled. he probably started this himself because he was bored of reading false claims(something I can really relate to).
 
I'm not really against 24bit, at least not as much as I am against 96khz or more. because at least 24bit doesn't hurt the sound or forces to use architectures that are best for wide frequency range instead of using what's best for the actual audible range. but even to use the volume of foobar at -60db like a crazy person, I just need to tell foobar to output 24bit so my music stays safe. no need for an original 24bit album right?
 
 
I liked the article a lot, lot of information but not at all boring, I see this as an actual success even if the conclusion was indeed obvious from the start. a bigger number of people would always be better for specific readings but it's hard to blame him for that ^_^. it was a good call and I enjoyed reading it thanks.