[limpidglitch]

 
 
I'm really really sorry HDtracks doesn't support Linux.  But if you find another track we can both download legitimately I'm game.  I was suggested to use HD Tracks 24 bit Random Access Memories but I thought Vivaldi was a better start point.  I can't get into Pono store yet, the European place only starts in January.  Where else?
 

 
People have been using free download samples from 2L in the past. Haven't listened to any of them myself, so I can't comment on recording and musical qualities.

 

[Greenears]

I understand the padding with zeros. It is just weird with dither. I'm going to first try get to 5% then I'll try to reverse that by converting to 24,then dither 16 then dither 24.

I'm not sure how familiar you are with the details of the foobar abxy plugin current version? I used the default setting.

Forgot about linn. Good idea.

I think we agree to try high dr tracks listening to quiet section. Best chance of many leading zero where 24 shines. Anyway to measure that with sox?

 

[castleofargh]

  Thanks all for interesting responses.  I'd like to get back to the meat of the testing, so let me address them collectively:
 
Maybe I should back up first and let you know the purpose of me posting as I am going along is to (a) get tips for improvement (b) encourage others to reproduce my results.  Maybe with your ears and/or equipment you can do better.  I will address a few objections as a courtesy, but I don't want to get too sidetracked from (a) and (b).  I would encourage more ideas on whether to listen to short clips or long, what to look for, better tracks to try (as long as normally accessible in US over regular network), etc etc etc.  
 
1. Number of trials and looking at the results
 
So I hear your input, and I can see where you are coming from.  Actually I never put any thought into this it was the default setting and off I went.  Plus it was late and I wanted success quicker rather than slower, if it was possible to discern.  However, I have taken a couple of 2nd year courses in probability enough to know the binomial distribution and its application.  It seems the ABX plugin is doing a straight binomial distribution calculation on the success/trials ratio.  It doesn't care what you are thinking or doing or intent or whether you are looking at the result or not.  That's the beauty of ABX testing.  The only thing that would be invalid is to throw out failing trials but the tool won't let you do that .... you can start again from the beginning or continue but that's it.  Note that 3/4 yields a very different % than 30/40 - that is baked into binomial. The % is valid no matter what you do or when you quit.  I promise you.  However, note that 10% means exactly that - 1 in 10 chance I was flipping coins and using that to decide. 
 
To humor everyone if I get what I consider solid results (5% or better) I'll redo it both ways and post the logs.  See how nice I am?  But I did want to set the record straight for our dear readers.
 
2. Dynamic range, and whether this Vivaldi is a good track to get a positive result
 
Firstly, people often confuse available dynamic range (eg. 96 dB for redbook CD) vs actual dynamic range of a section of music (ratio of loudest to softest part).  I was trying to select a piece of music with a high value for the second. I didn't measure it (someone provide me a SoX incantation and I'll gladly do it).  Whether Spring can have a dynamic range depends on the music to an extent, yes I agree, but also how it was recorded.  With a sensitive mic really close to a violin and if the player plays very softly and very loud, you could get a range.  The rest is the mix and how much dynamic range compression is applied.  My expectation is that at the most this is 50 dB, more likely less  Compared to 15 db for a lot of popular music I'm told.  I heard one engineer claim 60 dB on a big orchestra, that is the highest claim I've ever read.  The reason I want high range is to look for softer passages, were the relative delta of each step of quantization is highest.  This is where 24 bit might shine.
 
3. Difficulty downloading to reproduce
 
I'm really really sorry HDtracks doesn't support Linux.  But if you find another track we can both download legitimately I'm game.  I was suggested to use HD Tracks 24 bit Random Access Memories but I thought Vivaldi was a better start point.  I can't get into Pono store yet, the European place only starts in January.  Where else?
 
4. Dither and reconverting to 24 bit
 
I intentionally started without dither.  I want the best chance for success first, then I'll add dither and see if it makes me fail.
 
>sox -V4 24to16bit.flac -b 24 16to24.flac  -- interesting suggestion, but I don't think think upconverting is legitimate is it?  The dither will be on the 8th LSB and not 0 LSB when you get back to 24.  The DAC natively handles 16 and 24 and converts to multi-segment Sig-Delt anyway so I think my method is legit. This incantation was suggested after some discussion.  
 
One word about success: I am an admitted skeptic about 24 bit.  My intellectual bias says that 16 bit redbook may be the be-all and end-all of music, all we need is better reproduction hardware.  But, I'm not 100% sure of my position.  So why do I want to succeed in telling them apart?  The weakness I find in ABX testing is when someone does a whole series of comparison and they all come up negative.  It is easily attacked, and hard to prove the negative (ie if you had just done X you would have passed).  But if you pass one and gradually change parameters till you fail, it shows where the knee in the curve is for your ears and setup. IMHO.

1/ if you do more trials, you just have to add the previous results to them. that way you get rapidly a much better statistical relevance and avoid the problems of seeing the results live, or simply picking the one test that looked the best out of several. it's not a problem for you and it's very much reassuring for us ^_^.
 
2/

My expectation is that at the most this is 50 dB, more likely less  Compared to 15 db for a lot of popular music I'm told.  I heard one engineer claim 60 dB on a big orchestra, that is the highest claim I've ever read.  The reason I want high range is to look for softer passages, were the relative delta of each step of quantization is highest.  This is where 24 bit might shine.

agreed with about 60db, the max we encounter on albums is in that ballpark.
about the quantization delta, maybe I don't get what you're saying again, but to me it's nope. with the lowest signal let's say at -70db(if the audio wasn't topped at 0db but at -10instead), even then you would still have the quantization noise much lower, so no masking, no nothing from 16bit IMO.
and of course the values of each samples are strictly the same with the same precision in 16 and 24bit, only the added zeros unused, and the overall noise floor will change on 24bit.
 
3/ the track that we pick doesn't mean a lot. obviously for the purpose of the test it would seem logical to get the most dynamic piece we can, but as we all agree, the most dynamic track is still far from reaching 16bit limits. the track dynamic will not change, no part of the music will be truncated of compressed, so all in all if you want to abx 16 and 24bit of a metallica track, it would give pretty much the same results I think. because if there is an audible difference between the 2 bit depth, the tracks dynamic won't really have much to do with it.
at least that's how I see it.
 
4/dither or no dither, well it doesn't really matter much. just know that almost any 16bit album is dithered, so it's not such an unfair game to dither you track.  my own trials have shown that unless I push the volume to the point where I can hear the noise(not safe at all!), I can't seem to distinguish dither or no dither, or even different dithering methods.
 
and of course converting back to 24bit is just a harmless way to make sure you don't get external hints. on a computer I wouldn't think so, and with foobar abx I believe both files are turned into 32bit wave or something before being played.  so it doesn't matter.

 

[stv014]

Originally Posted by Greenears /img/forum/go_quote.gif
 
1. Number of trials and looking at the results
 
So I hear your input, and I can see where you are coming from.  Actually I never put any thought into this it was the default setting and off I went.  Plus it was late and I wanted success quicker rather than slower, if it was possible to discern.  However, I have taken a couple of 2nd year courses in probability enough to know the binomial distribution and its application.  It seems the ABX plugin is doing a straight binomial distribution calculation on the success/trials ratio.  It doesn't care what you are thinking or doing or intent or whether you are looking at the result or not.  That's the beauty of ABX testing.  The only thing that would be invalid is to throw out failing trials but the tool won't let you do that .... you can start again from the beginning or continue but that's it.  Note that 3/4 yields a very different % than 30/40 - that is baked into binomial. The % is valid no matter what you do or when you quit.  I promise you.  However, note that 10% means exactly that - 1 in 10 chance I was flipping coins and using that to decide. 
 
To humor everyone if I get what I consider solid results (5% or better) I'll redo it both ways and post the logs.  See how nice I am?  But I did want to set the record straight for our dear readers.

 
Restarting the test when the result starts to look bad (discarding the previous trials as if they never happened) is definitely a "cheating" method, as is doing multiple runs and choosing the best score while ignoring the rest. In general, deciding whether a trial is to be included in the "final" statistics after knowing if it was a correct guess introduces bias. That is why I suggest computing an overall score from all trials ever done, except those that were chosen to be excluded in advance before doing them (e.g. for "training" purposes). Looking at the score after every trial, and stopping when it is "good enough" is also a problem, particularly when only looking for a relatively high p-value like the usual 0.05. In any case, with biased methods, the real chance of guessing is higher, for example one 5% best result cherry picked from 10 runs can actually be achieved with ~40% chance just randomly guessing. The various biases can be compensated by requiring a much lower p-value, like 0.0001 (0.01% chance of guessing).

 

[RRod]

I understand the padding with zeros. It is just weird with dither. I'm going to first try get to 5% then I'll try to reverse that by converting to 24,then dither 16 then dither 24.

I'm not sure how familiar you are with the details of the foobar abxy plugin current version? I used the default setting.

Forgot about linn. Good idea.

I think we agree to try high dr tracks listening to quiet section. Best chance of many leading zero where 24 shines. Anyway to measure that with sox?

 
I'm on Linux, so I can't advise on foobar. One other thing to check for is that you're not getting messages about clipping from sox (don't know if this happens with non-dithered bit-depth conversions). My first step is typically to normalize the volume of the 24-bit track to -3 or -6dB, which is legit because even the best audio equipment doesn't actually get 24 bits above noise, so 23-23.5bits isn't losing anything. But this gives you headroom for the calculations sox has to do. Also, since there is 0% chance that any real music track is taking up all 24bits, normalization brings up the soft parts exactly so they aren't lost in any bit truncation.
 
The incantation:
sox 24bitfile.flac normed.flac gain -n -3

 

[vid]

 
I'm on Linux, so I can't advise on foobar.

 
Works fine on Linux through Wine, for the record.

 

[RRod]

   
Works fine on Linux through Wine, for the record.

 
No doubt, but I save Wine for things that have no legit equivalent on Linux.

 

[Greenears]

 
1/ if you do more trials, you just have to add the previous results to them. that way you get rapidly a much better statistical relevance and avoid the problems of seeing the results live, or simply picking the one test that looked the best out of several. it's not a problem for you and it's very much reassuring for us ^_^.
 
2/
agreed with about 60db, the max we encounter on albums is in that ballpark.
about the quantization delta, maybe I don't get what you're saying again, but to me it's nope. with the lowest signal let's say at -70db(if the audio wasn't topped at 0db but at -10instead), even then you would still have the quantization noise much lower, so no masking, no nothing from 16bit IMO.
and of course the values of each samples are strictly the same with the same precision in 16 and 24bit, only the added zeros unused, and the overall noise floor will change on 24bit.
 
3/ the track that we pick doesn't mean a lot. obviously for the purpose of the test it would seem logical to get the most dynamic piece we can, but as we all agree, the most dynamic track is still far from reaching 16bit limits. the track dynamic will not change, no part of the music will be truncated of compressed, so all in all if you want to abx 16 and 24bit of a metallica track, it would give pretty much the same results I think. because if there is an audible difference between the 2 bit depth, the tracks dynamic won't really have much to do with it.
at least that's how I see it.
 
4/dither or no dither, well it doesn't really matter much. just know that almost any 16bit album is dithered, so it's not such an unfair game to dither you track.  my own trials have shown that unless I push the volume to the point where I can hear the noise(not safe at all!), I can't seem to distinguish dither or no dither, or even different dithering methods.
 
and of course converting back to 24bit is just a harmless way to make sure you don't get external hints. on a computer I wouldn't think so, and with foobar abx I believe both files are turned into 32bit wave or something before being played.  so it doesn't matter.

This maybe difficult for those without background in signal theory to understand, I'll try a different way.  Think of quantization as levels, or rungs on a ladder.  Let's say there are 100 equal space levels to use round numbers.  If the loudest peak is at level 90, the lowest peak at level 40, and the combined noise floor (recording noise plus quantization) is at 10,  then there are 80 levels between the loud parts and floor, but only 30 levels between the soft parts and the floor.  You can see that the level gaps are twice as big relative to the amplitude of the soft part compared to the loud part.   
 
Yes I've read most CD are mastered with peaks at -10 to -15 dBFS.   So let's say this one peaks at -10 dBFS and with 50dB DR then the bottoms are at -60dBFS and let's say the noise is at -90.  You only have 30 dB SINAD for the soft parts compared to loud parts 80 dB.  It's exactly the same thing I just said before, expressed in dB.  Since I expect 80 dB is not audible, I focus on the 30. 
 
Note though I started just trying to get a "Feel" for the opening loud passage.  My score improved when I focused on a narrow medium soft passage.  That is why I think there is something to it.

 

[Greenears]

FWIW I did another run.  Same files same procedure, nothing touched since the other day.  Still no dither, no upconvert (I'll get to that later if I can get to 5% or better). I got the first trial wrong, but then had a streak of 4 in a row to get me to 5/6 and about 10% or so.  Then I kept going and ended up with 5/10 62.3%.  Interestingly I felt I was starting to fatigue into the 2nd half and I was also distracted maybe somewhere around trial 6 I had to get up for a moment.
 
Other than that I just focused on that one passage from 1:10-1:21.1 at volume 48.  I listened to A & B for a few minutes to get warmed up, but then only XY for the first 5 trials .... I checked back with A & B a little at maybe trial 7 but it didn't help.   I also felt less certain about my choices later.
 
I'm going to try again a longer run when I'm better rested and no distractions.  I'm prettty confident in what I'm listening for, but you play this passage 5 or so times each trial, so after play 50 your brain doesn't want to focus.   Another tactic may be to do 5 trials, take a break and come back for another 5 (on the same test panel) and so on.
 
If someone has a passage were it is way easier to pick it out ... I'm all ears please suggest.  I would prefer a short passage, 10 seconds or less.
 
So far I have never seen less than 10%, not even for a fleeting moment.

 

[RRod]

  FWIW I did another run.  Same files same procedure, nothing touched since the other day.  Still no dither, no upconvert (I'll get to that later if I can get to 5% or better). I got the first trial wrong, but then had a streak of 4 in a row to get me to 5/6 and about 10% or so.  Then I kept going and ended up with 5/10 62.3%.  Interestingly I felt I was starting to fatigue into the 2nd half and I was also distracted maybe somewhere around trial 6 I had to get up for a moment.
 
Other than that I just focused on that one passage from 1:10-1:21.1 at volume 48.  I listened to A & B for a few minutes to get warmed up, but then only XY for the first 5 trials .... I checked back with A & B a little at maybe trial 7 but it didn't help.   I also felt less certain about my choices later.
 
I'm going to try again a longer run when I'm better rested and no distractions.  I'm prettty confident in what I'm listening for, but you play this passage 5 or so times each trial, so after play 50 your brain doesn't want to focus.   Another tactic may be to do 5 trials, take a break and come back for another 5 (on the same test panel) and so on.
 
If someone has a passage were it is way easier to pick it out ... I'm all ears please suggest.  I would prefer a short passage, 10 seconds or less.
 
So far I have never seen less than 10%, not even for a fleeting moment.

 
See my comment on normalization above, as that will affect results across the board. You can save interim results to a file, then compile all the results at the end if you want to spread your testing out over several days to avoid fatigue. Just ignore the file results and hold yourself to the preset number of trials.

 

[castleofargh]

 

 
1/ if you do more trials, you just have to add the previous results to them. that way you get rapidly a much better statistical relevance and avoid the problems of seeing the results live, or simply picking the one test that looked the best out of several. it's not a problem for you and it's very much reassuring for us ^_^.
 
2/
agreed with about 60db, the max we encounter on albums is in that ballpark.
about the quantization delta, maybe I don't get what you're saying again, but to me it's nope. with the lowest signal let's say at -70db(if the audio wasn't topped at 0db but at -10instead), even then you would still have the quantization noise much lower, so no masking, no nothing from 16bit IMO.
and of course the values of each samples are strictly the same with the same precision in 16 and 24bit, only the added zeros unused, and the overall noise floor will change on 24bit.
 
3/ the track that we pick doesn't mean a lot. obviously for the purpose of the test it would seem logical to get the most dynamic piece we can, but as we all agree, the most dynamic track is still far from reaching 16bit limits. the track dynamic will not change, no part of the music will be truncated of compressed, so all in all if you want to abx 16 and 24bit of a metallica track, it would give pretty much the same results I think. because if there is an audible difference between the 2 bit depth, the tracks dynamic won't really have much to do with it.
at least that's how I see it.
 
4/dither or no dither, well it doesn't really matter much. just know that almost any 16bit album is dithered, so it's not such an unfair game to dither you track.  my own trials have shown that unless I push the volume to the point where I can hear the noise(not safe at all!), I can't seem to distinguish dither or no dither, or even different dithering methods.
 
and of course converting back to 24bit is just a harmless way to make sure you don't get external hints. on a computer I wouldn't think so, and with foobar abx I believe both files are turned into 32bit wave or something before being played.  so it doesn't matter.

This maybe difficult for those without background in signal theory to understand, I'll try a different way.  Think of quantization as levels, or rungs on a ladder.  Let's say there are 100 equal space levels to use round numbers.  If the loudest peak is at level 90, the lowest peak at level 40, and the combined noise floor (recording noise plus quantization) is at 10,  then there are 80 levels between the loud parts and floor, but only 30 levels between the soft parts and the floor.  You can see that the level gaps are twice as big relative to the amplitude of the soft part compared to the loud part.   
 
Yes I've read most CD are mastered with peaks at -10 to -15 dBFS.   So let's say this one peaks at -10 dBFS and with 50dB DR then the bottoms are at -60dBFS and let's say the noise is at -90.  You only have 30 dB SINAD for the soft parts compared to loud parts 80 dB.  It's exactly the same thing I just said before, expressed in dB.  Since I expect 80 dB is not audible, I focus on the 30. 
 
Note though I started just trying to get a "Feel" for the opening loud passage.  My score improved when I focused on a narrow medium soft passage.  That is why I think there is something to it.

ok so I understood your previous post right and just don't get why that would make any difference. unless you're meaning to say that you're hearing the noise floor?
 
also it's kind of funny that you're thinking I can't follow digital theory when I started writting to you because I thought you didn't understand the basics ^_^.

 

[The Walrus]

And of course you've checked that the HD Tracks file is genuine 24/96 and not upsampled 16/44 right? Being upsampled wouldn't be a first for that outfit.

This is the response to my query, by the genius at HDTracks tech support: 
 
"I'm going to try and clear things up for you. To understand the bit depth (also referred to as the bit rate) of an audio file, you need to understand sample rates and the process by which audio is captured digitally using modern recording techniques. With Digital Audio Workstations (DAW) audio is captured at a certain sample rate, usually being 44.1k, 48k, 88k, 96k, 176k, or 192k. This represents the amount of "samples," or digital snapshots, taken per second of the analog audio source. So obviously 192,000 samples per second will be much greater audio quality than 44,100 samples per second because more audio data is being recorded. 
 
Now onto bit depth. Bit depth is the quality of each individual sample. This means that a 24 bit sample is greater quality than a 16 bit sample. You can liken this to megapixels on a camera. The more megapixels your camera has, the greater the image quality.
 
If you would like to learn more about sample rates and bit depth, there are countless articles online. Please note that not everybody can hear the difference between hi-res audio and a regular 44/16 physical CD, which is why we recommend that everyone try our hi-res free sampler to find out.
 
I hope this information is helpful."


Go to their website and check the FAQ page. There they say that the tracks that they sell is CD QUALITY. Nowhere on their website you can find a single word about the benefits of HD audio. Isn't it strange that these people aren't promoting the very product they are selling?

 

[castleofargh]

 

And of course you've checked that the HD Tracks file is genuine 24/96 and not upsampled 16/44 right? Being upsampled wouldn't be a first for that outfit.

This is the response to my query, by the genius at HDTracks tech support: 
 
"I'm going to try and clear things up for you. To understand the bit depth (also referred to as the bit rate) of an audio file, you need to understand sample rates and the process by which audio is captured digitally using modern recording techniques. With Digital Audio Workstations (DAW) audio is captured at a certain sample rate, usually being 44.1k, 48k, 88k, 96k, 176k, or 192k. This represents the amount of "samples," or digital snapshots, taken per second of the analog audio source. So obviously 192,000 samples per second will be much greater audio quality than 44,100 samples per second because more audio data is being recorded. 
 
Now onto bit depth. Bit depth is the quality of each individual sample. This means that a 24 bit sample is greater quality than a 16 bit sample. You can liken this to megapixels on a camera. The more megapixels your camera has, the greater the image quality.
 
If you would like to learn more about sample rates and bit depth, there are countless articles online. Please note that not everybody can hear the difference between hi-res audio and a regular 44/16 physical CD, which is why we recommend that everyone try our hi-res free sampler to find out.
 
I hope this information is helpful."


Go to their website and check the FAQ page. There they say that the tracks that they sell is CD QUALITY. Nowhere on their website you can find a single word about the benefits of HD audio. Isn't it strange that these people aren't promoting the very product they are selling?

what's nice is "To understand the bit depth (also referred to as the bit rate) of an audio file"

yeah bit rate, expressed in bit per second is obviously the same as bit depth... it's probably just an attention mistake from the guy, but that certainly doesn't look pro.
I would bet that most hirez providers have started to become expert in the art of not clearly saying that hirez is always hirez quality. because else they get in trouble each time the track is actually 16/44 turned into 24/96. some put on a warning like "we take the files as given by the studio, so don't sue us, we're innocent and we don't have the money to buy a free copy of audacity and check ourselves", others just avoid the question altogether.

 

[Greenears]

   
See my comment on normalization above, as that will affect results across the board. You can save interim results to a file, then compile all the results at the end if you want to spread your testing out over several days to avoid fatigue. Just ignore the file results and hold yourself to the preset number of trials.

So I mucked around for 10 minutes trying to find out where the log files are, looking online, searching my drives.  Finally I figured it out by accident, it offers you a log when you exit.  But it wasn't obvious that was the log.  Next time I'll post a log.  I'll have to figure out how to join logged sessions.
 
I hear you on normalization, and I guess if I keep failing I'll try it later. The problem is once you manipulate the file in any way beyond chopping the 8 bits from 24 to 16 I think you open yourself to criticism (beyond leveling the volume between the tracks which is of course required).  By turning up the volume on a soft section I'm sort of doing the same thing. 

 

[Greenears]

 
ok so I understood your previous post right and just don't get why that would make any difference. unless you're meaning to say that you're hearing the noise floor?
 
also it's kind of funny that you're thinking I can't follow digital theory when I started writting to you because I thought you didn't understand the basics ^_^.

I don't understand why you don't understand!  What is the air-speed velocity of an unladen swallow!  Argh.
 
If you take my ladder analogy to the extreme, you can make the signal so small that there are only two rungs.  At that point you won't hear the signal.  So yes obviously amplitude matters and makes a difference.  The word "noise" means many things since there are many different types.  Quantization noise could be called quantization artifacts or whatever you want.  I am simply saying on soft passages I think you have a better chance of hearing Q artifacts.  However, the q noise may be below the background noise of the recording room, in which case you can't hear it whatever you do and I'll fail the ABX testing.  Unless there are other artifacts at bigger amplitudes, such as subtle phase shifts (frequency shifts) that you can hear.  I don't know.