[RRod]

  Consider me duly riled.  It wasn't just you.  Think about it: The whole world is on the cusp of going 24-bit crazy (well the audiophile world anyway).  Someone comes along and tries to do some honest ABX testing.  And within nanoseconds I'm accused of constructing some kind of devious Kobayashi Maru no-win test (because of course everyone _knows_ what the right answer is) and if I somehow pass I must have rigged it.  Really.  
 
Anyway back to the testing - so you got 15/25 on the Linn test192 clarinet, right? Do you think you heard anything specific?
 
I just did another run, this time with Cassandra Wilson from HD Tracks sampler.  Same Sox -b 16 not dither no upsampling.  This time however I unchecked "look at the results".  I also discovered that when you click that it doesn't tell you how many trials (?) and then I got interrupted. I definitely intended to do 10 or more.
 
The biggest change was on this I just played A and B a couple times each, then played X then Y from the start and normal med-low volume extended listening conditions.  I just closed my eyes and played about 16-24 bars from time 0 until I had the feel taking in the whole groove & rythm and not trying to focus on anything specific.  Played X once, Y once, whichever I liked better I picked.  Frankly I'm surprised how well I did.  I got interrupted at this point, and I know it is not enough trials.  But still - interesting.  Totally different method than the previous "successes".   This method I would say is much more akin to "natural" listening.  Hmmm.  
 
The ABX is not co-operating.  It would be much more convenient just to get results all over the place and conclude it's a wash and go back to fighting clingons.
 
foo_abx 1.3.4 report
foobar2000 v1.3.6
2014/12/16 17:00:38
File A: C:\Users\Public\Music\HDtracks\Various Artists HDtracks Sampler\HDtracks 2014 Sampler\08-Another Country.flac
File B: C:\Users\Public\Music\HDtracks\Various Artists HDtracks Sampler\HDtracks 2014 Sampler\cassandra16.flac
17:00:38 : Test started.
17:03:43 : 01/01  50.0%
17:04:33 : 02/02  25.0%
17:05:41 : 03/03  12.5%
17:06:34 : 04/04  6.3%
17:07:21 : 04/05  18.8%
17:08:42 : 05/06  10.9%
17:09:34 : 05/07  22.7%
17:09:43 : Test finished.
 ---------- 
Total: 5/7 (22.7%)

 
Here was my method:
1) Do the conversion, don't look at anything
2) Set everything to 24/192k
3) Do the trials
4) Look at spectrograms and see if everything looks right
 
For 3), I marked a number of passages (breathiness of clarinet on first long note, quiet piano parts, fade out, etc.), and tried my best to hear a difference in a couple of marks for each sample. I really couldn't latch on to anything, and the difference spectrogram concurs: the only difference is some noise, and that's already at -100dB or so and that's in the noise-shaping region. I went ahead and downloaded Linn's recording of the Poulenc organ concerto, and it's the same thing: just a bit of noise is the difference.

 

[Krutsch]

  Only if you can hear frequencies humans can't hear.

 
And that's because the increased sample rate contributes to the range of frequency response (e.g. 96 kHz for a sample rate of 192 kHz), as opposed to adding greater resolution to the audible Redbook range (i.e. 20 - 20 kHz), correct?
 

It's pretty safe to say that if you hear a difference between redbook and high bit/sampling rate, it's the difference in the way your equipment is playing the file that makes the difference, not the sound in the recording. 

 
Ignoring the possibility that the high bit/sampled version is from a different master than the Redbook version, of course; I might expect the higher bit-rate/depth tracks to place additional load on the playback system (i.e. the files are much larger, requiring more of everything to play them back: CPU load to decode and convert, network throughput, HDD activity).
 
So, what's the motivation for standards like DTS-HD and Dolby True HD, et al., that support 24-bit depth and high sample rates (outside of additional surround channels and volume leveling)?
 
I can see recording / processing sound tracks that way to mitigate rounding errors (big difference in this regard between 24 and 16 bit, from a numerical analysis standpoint). But when presented to the listener, it's all marketing spin (i.e. "as the artist intended" or as the "director originally recorded" for ease of post-production)?  I suppose, at the point, why not just release as-is and use it to push a new format?
 
Anyway... thanks.

 

[Greenears]

 
For 3), I marked a number of passages (breathiness of clarinet on first long note, quiet piano parts, fade out, etc.), and tried my best to hear a difference in a couple of marks for each sample. I really couldn't latch on to anything, and the difference spectrogram concurs: the only difference is some noise, and that's already at -100dB or so and that's in the noise-shaping region. I went ahead and downloaded Linn's recording of the Poulenc organ concerto, and it's the same thing: just a bit of noise is the difference.

Yup, I tried "breathiness" of first clarinet note as well.  At first my mind was sure it sounded more resolved or "real" on the 24 but when I tried to use that to ABX I did horrible.  So I abandoned that pretty quickly.  
 
I don't consider white noise (or even slightly pink noise) to be an issue with 16.  Humans have an amazing ability to tune out random backgrounds, go back and listen to tapes from the 80s compared to CDs.  I never hear or notice noise on redbook, and haven't heard on any testing.  It is too far down.  So I don't consider it valid to find a short gap, mark that and amplify the heck out of it until you can hear noise and ABX that to pass.  Yes I may be Kirk, but even he wouldn't do that.

 

[bigshot]

  And that's because the increased sample rate contributes to the range of frequency response (e.g. 96 kHz for a sample rate of 192 kHz), as opposed to adding greater resolution to the audible Redbook range (i.e. 20 - 20 kHz), correct?

 
Yes. Sampling rate is frequency response. Bit depth is noise floor. If you know how digital audio works, you know what to expect. Inaudible is inaudible. If someone comes up with a way to hear the unbearable, you give them the benefit of the doubt and figure that they are doing something wrong without realizing it. The truth is that none of it makes a lick of difference. If you have to sweat and strain and fudge your numbers to be able to hear something, it just doesn't matter anyway.
 

  So, what's the motivation for standards like DTS-HD and Dolby True HD, et al., that support 24-bit depth and high sample rates (outside of additional surround channels and volume leveling)?

 
Marketing?

 

[castleofargh]

 

 
For 3), I marked a number of passages (breathiness of clarinet on first long note, quiet piano parts, fade out, etc.), and tried my best to hear a difference in a couple of marks for each sample. I really couldn't latch on to anything, and the difference spectrogram concurs: the only difference is some noise, and that's already at -100dB or so and that's in the noise-shaping region. I went ahead and downloaded Linn's recording of the Poulenc organ concerto, and it's the same thing: just a bit of noise is the difference.

Yup, I tried "breathiness" of first clarinet note as well.  At first my mind was sure it sounded more resolved or "real" on the 24 but when I tried to use that to ABX I did horrible.  So I abandoned that pretty quickly.  
 
I don't consider white noise (or even slightly pink noise) to be an issue with 16.  Humans have an amazing ability to tune out random backgrounds, go back and listen to tapes from the 80s compared to CDs.  I never hear or notice noise on redbook, and haven't heard on any testing.  It is too far down.  So I don't consider it valid to find a short gap, mark that and amplify the heck out of it until you can hear noise and ABX that to pass.  Yes I may be Kirk, but even he wouldn't do that.

but noise floor is or at least should be the only difference from changing the bit depth. so by dismissing the noise floor difference, I see it like dismissing the factual difference to look for something that shouldn't exist. I never said you were not hearing something, but you have to admit that you're mindset for going at it is strange.
it's always possible that the conversion somehow changed something it shouldn't, then the abx prog will turn the 2 files into 32bit pcm tracks for the test, those 2 pcm streams will then be turned down to whatever you have set on your computer's output(foobar is in 24bit? are you using direct input? ...). it's all a matter of adding zeros and then cut them out, it shouldn't change the value of the music sample themselves as they are well above 16bit in value. but still that's a lot of ups and downs for a file and maybe somehow somewhere, something goes wrong for one of them? but even if that happens, how can you attribute the difference to the track being in 16bit, or even to the first conversion to 16bit?
by turning the 16bit back to 24bit, you add yet another change, but at least you know that the computer will treat both files the same way. that's why even though that should not change a thing, most of us will do that and put the files we want to test back into a common resolution/file format. just to be sure we're testing the track and not the computer or the DAC.
but when we make suggestions, apparently if it's not RRod you decide you know better. (yeah I'm jealous

).
same for dithering, if the noise floor isn't audible why should that matter to add dither? CDs are dithered, so it would seem like a more honest comparison.
 
as for us knowing the result in advance and trying to see you fail (or whatever), there had been quite a few tests done before you, a few AES papers,and of course tests we did for ourselves. this isn't really cutting edge experiment and anybody can do it with free and easy to use tools. so yeah we tend to feel like we already know the end of the movie. I really don't see what's wrong with that? people getting positive results are marginal, and usually when they don't run away insulting us for doubting them, it ends up that the files were different, one way or another. you can feel offended when we suspect something done wrong in your trials if it pleases you, but it's not because we secretly hate you or that we're all members of the 16bit lobbying cult. it's because you offer us an unlikely result that goes against what is mostly recognized(by science and engineers, maybe not so much by "audiophiles"

) for abx at normal listening levels.
 
if you come telling me that you can abx a flac from a mp3@96kb I will not try to find a reason why you succeed, because it is expected for you to do so. it has nothing to do with egos or knowing better, it's about you saying that you can identify 16bit from 24bit when pretty much any controlled tests resulted in people unable to tell 16/44 from any superior resolutions whatever the file format or the resolution. DVD, DSD, PCM they all failed to show audible differences one after the other. and that's why we look for a reason for you more than guessing results that isn't the track being 16bit. maybe RRod can send you his converted file or you send yours(short sample else copyright police will strike us all dead) so we can start by making sure the conversion went ok? that would be one less possible bias in the way.

 

[RRod]

Don't be jelly, brah! 

 

[Danz03]

Signal to noise ratio for 24bit is 144dB, signal to noise ratio for 16bit is 96dB. One of the best analogue 2 track master recorders was the Studer A820, with 77dB signal to noise ratio at 1/2" 30ips and crosstalk of 65dB at 1kHz, most digital recordings in the 80s and 90s were mastered in 16bit; so unless the 24bit file was recorded within the last 15 years, the 2 track master that it was mastered from would unlikely to have signal to noise ratio better than 96dB anyway. And if it was remastered using some kind of denoisng plugins, why would anyone need to pay for the privilege? Anyone could have done it himself with a denoiser plugin.
 
Quote:

 
but noise floor is or at least should be the only difference from changing the bit depth. so by dismissing the noise floor difference, I see it like dismissing the factual difference to look for something that shouldn't exist. I never said you were not hearing something, but you have to admit that you're mindset for going at it is strange.

 

[castleofargh]

  Don't be jelly, brah! 

I know it's because I'm bald.

 

[Krutsch]

Originally Posted by castleofargh /img/forum/go_quote.gif
 
<snip, snip>
 
...it's about you saying that you can identify 16bit from 24bit when pretty much any controlled tests resulted in people unable to tell 16/44 from any superior resolutions whatever the file format or the resolution. DVD, DSD, PCM they all failed to show audible differences one after the other. 
 
<snip, snip>

 
Do you have a link you can share? I've read examples of controlled tests for audibility of various bit rates of compressed music, but I haven't seen examples of the above. I can Google, too, but if you have links handy, I would like to read/learn more. I'm more interested in the testing methodology than the actual results, to be frank.

 

[RRod]

   
Do you have a link you can share? I've read examples of controlled tests for audibility of various bit rates of compressed music, but I haven't seen examples of the above. I can Google, too, but if you have links handy, I would like to read/learn more. I'm more interested in the testing methodology than the actual results, to be frank.

 
Here's one:
http://www.drewdaniels.com/audible.pdf

 

[castleofargh]

 

Originally Posted by castleofargh /img/forum/go_quote.gif
 
<snip, snip>
 
...it's about you saying that you can identify 16bit from 24bit when pretty much any controlled tests resulted in people unable to tell 16/44 from any superior resolutions whatever the file format or the resolution. DVD, DSD, PCM they all failed to show audible differences one after the other. 
 
<snip, snip>

 
Do you have a link you can share? I've read examples of controlled tests for audibility of various bit rates of compressed music, but I haven't seen examples of the above. I can Google, too, but if you have links handy, I would like to read/learn more. I'm more interested in the testing methodology than the actual results, to be frank.

wow not having access to AES I'm probably not the most reliable source. I just come across one from time to time, but I can't say I'm organized enough to even pretend I have a list ^_^.
the first that comes to mind is indeed moran&meyer linked by RRod. I think it's the one that was both recognized and meaningful. I think it closed the book for many people about the purpose of hires. as a listener at least, because we all understand why sony would want to get new support patents from time to time.
one I've posted myself not long ago https://www.gearslutz.com/board/attachments/high-end/6491d1114045260-why-didnt-dsd-catch-reshaping_digital_-audio.pdf  with a relatively small number of participants so not really conclusive but following the general idea.
a good deal of websites or groups of audiophiles have at some point conducted some sort of test like that with more or less controls. the latest being probably at archimago's bog. but that's totally uncontrolled, the results are based on trusting people not to open the files in an audio editor.
 
if it's only for methodology of tests, then I guess you can also look for hires vs hires.

 

[Krutsch]

^^^ Thanks for the suggestions.

 

[limpidglitch]

If you're interested in how they do it within the industry, this paper on testing codecs for use in DAB might be interesting.
The testing standards discussed can be found here, and here.

 

[Greenears]

   
Here's one:
http://www.drewdaniels.com/audible.pdf

Thank you RRod for actually providing a link, as compared to Castle's long-winded hand-waving at mountains of evidence.....  Surely I jest Castle, your input has been valuable RRod just has a certain way with posts....    
 
I had seen reference to this paper but never were able to see the content of it until now.  I read it with apt attention.
 
My comments:  I like the general test setup.  I think they did a good job of proving for their listening setup (state of the art at the time) and for the SACD/DVDA programming available at the time of testing in 2007 that the material could just as well been recorded on Redbook since it was not distinguishable by a large number of test subjects.
 
The shortcomings I think are that this needs to be updated for 2014 to test 192/24 FLAC, and I would have liked to seen a list of the material tested.  There have been advances in DAC design in the last 7 years, with relatively inexpensive 24 bit DACs multi-segment SD architectures with amazing 130 dB performance.  I doubt this would have been in the equipment tested at the time.
 
To me there were two very interesting notes in the paper.  In section 1 they note there was no published paper testing SACD vs Redbook up until then.  Which is amazing considering that high res formats had already been going for a decade.  This is my chief complaint: Lots of people today claim "there are mountains of testing" but I never get to see the notes of the actual test.  I have looked.  The second even more interesting thing is in section 4.  They point out that most of the SACD/DVDA "sounded better" than a CD, even through the CD loop.  They actually say these recordings should be released on Redbook!  This is something I've already posted about, that it is all in the mastering. But it is very hard to get information on the mastering and mixing of any tracks let alone 24 bit.

 

[bigshot]

If it's transparent at 44.1/16 it doesn't matter how high you go, it's still going to be transparent. No point reinventing wheels. Better to focus on things that actually make sound better.