24bit vs 16bit, the myth exploded!

[bigshot]

You're a lot more likely to reach the noise floor of the recording venue or your amp by turning up the volume, than you are to reach the noise floor of a CD.

 

[RRod]

  So I mucked around for 10 minutes trying to find out where the log files are, looking online, searching my drives.  Finally I figured it out by accident, it offers you a log when you exit.  But it wasn't obvious that was the log.  Next time I'll post a log.  I'll have to figure out how to join logged sessions.
 
I hear you on normalization, and I guess if I keep failing I'll try it later. The problem is once you manipulate the file in any way beyond chopping the 8 bits from 24 to 16 I think you open yourself to criticism (beyond leveling the volume between the tracks which is of course required).  By turning up the volume on a soft section I'm sort of doing the same thing. 

 
I wouldn't call it a manipulation, it's simply the business of mapping the 24bits into 16bits. If you can agree with me that the musical content of the track doesn't actually use all 24bits, then there are 2 extreme cases:
1) The recording is set to have a peak near 0dB. In this case, the softest musical material will be well above the noise floor and near 16bits anyway. Here is where truncation doesn't hurt much, as we're not throwing away that much musical content in the lower sig figs.
 
2) The recording is set to have the softest musical content just above the noise floor. In this case, the peaks won't get anywhere near 0dB, but since we have musical content near the floor, truncation will throw away lots of good stuff. This is where normalization comes in; you raise up the peaks to near 0dB to get the music away from the floor.
 
Normalization doesn't just turn up the soft parts, it turns up all track to get the peaks to a certain level. So it's a straight up proportional change, no tricks.
 
p.s. It's quite true that there comes a signal level where we only have 1 bit with which to encode it. But the thing is that you can still recover dynamics from this via dithering and noise shaping, which is exactly why DSD can work at all.

 

[Greenears]

   
I wouldn't call it a manipulation, it's simply the business of mapping the 24bits into 16bits. If you can agree with me that the musical content of the track doesn't actually use all 24bits, then there are 2 extreme cases:
1) The recording is set to have a peak near 0dB. In this case, the softest musical material will be well above the noise floor and near 16bits anyway. Here is where truncation doesn't hurt much, as we're not throwing away that much musical content in the lower sig figs.
 
2) The recording is set to have the softest musical content just above the noise floor. In this case, the peaks won't get anywhere near 0dB, but since we have musical content near the floor, truncation will throw away lots of good stuff. This is where normalization comes in; you raise up the peaks to near 0dB to get the music away from the floor.
 
Normalization doesn't just turn up the soft parts, it turns up all track to get the peaks to a certain level. So it's a straight up proportional change, no tricks.

Hrmmmphhh mmm arrr ergh. ... which is technical jargon for "I'm still mulling it over".  I understood what you meant right at the word "normalization" but it's still a subtle point.  The problem is the gain also amplifies the noise.  The ratio (dB) stays the same .... I'm actually no longer sure if it will help me pass or fail or make no difference.  To make the math simple, just think about a gain example as multiplying by power of 2 (shift left, pad with trailing zero, ~6dB).  When you truncate to 16 bits you only lose 7 bits of information vs 8 previously due to the trailing zero.  I know what you're saying but I think the volume control does the same thing.  Any noise wiggling around on the 7th bit will still be there in the same ratio.  
 
Do yo really think doing this will help me pass? I want to pass first.  After I get to 5% I'll dial it back and try more options. So far the best instantaneous number I've seen is 10%.  Today I mucked with the Linn tracks in poor listening conditions, my results were waay worse clearly guessing land.  Need to try more from HD tracks but so far that one passage from Spring is my best candidate since I've had 5+ streaks more than once. 
 
It just occurs to me there is a guy I know that I can ask some of these DSP questions.  He happens to be an awesome jazz musician (keyboards) and a wizard with FFTs.  Next week.

 

[limpidglitch]

 
Do yo really think doing this will help me pass? I want to pass first.  After I get to 5% I'll dial it back and try more options. So far the best instantaneous number I've seen is 10%.  Today I mucked with the Linn tracks in poor listening conditions, my results were waay worse clearly guessing land.  Need to try more from HD tracks but so far that one passage from Spring is my best candidate since I've had 5+ streaks more than once. 
 

 
If you continue doing it like this, you'll have to start applying Bonferroni corrections. But as you hove some basic statistics knowledge, you probably already know of this pitfall.

 

[tdog]

RRod et al. The only decent way to turn 24-bit audio into 16-bit is to dither it properly. For information on this see:
 
http://xiph.org/video/vid2.shtml
https://www.youtube.com/watch?v=cIQ9IXSUzuM - same as above if you don't have the codec.
https://www.youtube.com/watch?v=PydUQrka4pc
 
You could also try these (in rough order of sophistication):
http://en.wikipedia.org/wiki/Dither
http://www.audiocheck.net/audiotests_dithering.php
http://productionadvice.co.uk/no-stair-steps-in-digital-audio/
http://www.ece.rochester.edu/courses/ECE472/resources/Papers/Lip*s*h*i*tz_1992.pdf - I think I fixed the underlying link, if not remove the *s manually - it's well worth reading.
http://drewdaniels.com/dither.pdf
 
Turning 24-bit audio into 16-bit by truncating the last 8 bits is possible but not recommended, you will lose resolution as shown above..
 
Normalisation is the process of linearly multiplying all samples by the same amount to shift the peak value to some pre-determined (usually high) value. An unnormalised file might peak at -4.5dB, normalising it will bring it up to whatever value you choose, usually something close to 0dB, for example -0.05dB. This does not change the resolution at all, it does change your peak and average levels.
 
Hope that helps.

 

[tdog]

Oh the STOOPID editor has starred out Stanley Lip-shi-tz name in the link above because it has crap in it - GROAN!
 
Here is the link http://www.ece.rochester.edu/courses/ECE472/resources/Papers/Lip*s*h*i*tz_1992.pdf - remove the *s manually...

 

[limpidglitch]

  Oh the STOOPID editor has starred out Stanley Lip-shi-tz name in the link above because it has crap in it - GROAN!
 
Here is the link http://www.ece.rochester.edu/courses/ECE472/resources/Papers/Lip*s*h*i*tz_1992.pdf - remove the *s manually...

 
Or you can: http://bit.ly/1w4R1ii

 

[bigshot]

The goal is not necessarily to pass. It's to find out the truth.

 

[RRod]

  Hrmmmphhh mmm arrr ergh. ... which is technical jargon for "I'm still mulling it over".  I understood what you meant right at the word "normalization" but it's still a subtle point.  The problem is the gain also amplifies the noise.  The ratio (dB) stays the same .... I'm actually no longer sure if it will help me pass or fail or make no difference.  To make the math simple, just think about a gain example as multiplying by power of 2 (shift left, pad with trailing zero, ~6dB).  When you truncate to 16 bits you only lose 7 bits of information vs 8 previously due to the trailing zero.  I know what you're saying but I think the volume control does the same thing.  Any noise wiggling around on the 7th bit will still be there in the same ratio.  
 
Do yo really think doing this will help me pass? I want to pass first.  After I get to 5% I'll dial it back and try more options. So far the best instantaneous number I've seen is 10%.  Today I mucked with the Linn tracks in poor listening conditions, my results were waay worse clearly guessing land.  Need to try more from HD tracks but so far that one passage from Spring is my best candidate since I've had 5+ streaks more than once. 
 
It just occurs to me there is a guy I know that I can ask some of these DSP questions.  He happens to be an awesome jazz musician (keyboards) and a wizard with FFTs.  Next week.

Oh, no way it will help ^_^ Sorry, didn't know you wanted to bias towards that. This would only make it harder. You're right in that the noise moves up too, but then it gets hit by truncation and replaced by new artifacts from that.
 

  RRod et al. The only decent way to turn 24-bit audio into 16-bit is to dither it properly. For information on this see:
 
…
Turning 24-bit audio into 16-bit by truncating the last 8 bits is possible but not recommended, you will lose resolution as shown above..
 
Normalisation is the process of linearly multiplying all samples by the same amount to shift the peak value to some pre-determined (usually high) value. An unnormalised file might peak at -4.5dB, normalising it will bring it up to whatever value you choose, usually something close to 0dB, for example -0.05dB. This does not change the resolution at all, it does change your peak and average levels.
 
Hope that helps.

Normalization helps to prevent the problems of truncation, and is a separate issue from dithering. But yes, if you need every last inch of the Redbook dynamic range, you'll need dither, though in any non-high-range-dynamic setting, it won't really matter.

 

[bigshot]

  But yes, if you need every last inch of the Redbook dynamic range, you'll need dither

 
Redbook is overkill.

 

[RRod]

   
Redbook is overkill.

 
Yes, whence the if.

 

[tdog]

 
Normalization helps to prevent the problems of truncation, and is a separate issue from dithering. But yes, if you need every last inch of the Redbook dynamic range, you'll need dither, though in any non-high-range-dynamic setting, it won't really matter.

 
The "problems of truncation" are completely solved by dither - learn the theory and put it into practice properly, as per the papers I linked to.
 
I did not confuse normalisation and dithering just because I put them in the same post, as you have suggested. Truncating 24-bit audio to 16-bit is something that no one should do these days, even for relatively compressed and low dynamic range pop music - the difference is audible on good playback systems, as per the links in my previous post.

 

[RRod]

   
The "problems of truncation" are completely solved by dither - learn the theory and put it into practice properly, as per the papers I linked to.
 
I did not confuse normalisation and dithering just because I put them in the same post, as you have suggested. Truncating 24-bit audio to 16-bit is something that no one should do these days, even for relatively compressed and low dynamic range pop music - the difference is audible on good playback systems, as per the links in my previous post.

 
I didn't say you confused them, I said they were two different things, and the simple fact is that the default bit-depth down-conversion in SoX is truncation, and that's what we're talking about. Give it a try yourself if you want. Make a 24bit sine wave down around -100dB, tell SoX to convert to 16bit, and you'll get nothing but zeros in the output.
 
FWIW, here's my usual resampling script for doing ABX. Where would you like me to improve it?
 
#!/bin/bash
#1 = file
#2 = lo bit
#3 = lo rate
#find bits, rate, and # of samples
bits=$(sox --i "$1" 2>&1 | grep "Precision" | cut -d " " -f 8 | tr --delete "\-bit")
rate=$(sox --i "$1" 2>&1 | grep "Sample Rate" | cut -d " " -f 7)
samps=$(echo $(sox "$1" -n stat 2>&1| grep read |cut -d " " -f 11)/2 | bc)
#normalize xhi-res
peak=$(sox "$1" -n stats 2>&1 | grep "Pk lev" | sed 's/[ ]\+/ /g' | cut -d ' ' -f 4)
gain=$(echo "(("$peak")>-3)*(-3-("$peak"))"|bc)
sox -V4 "$1" xhi.wav gain -n "$gain"
sox -V4 xhi.wav -b "$2" xres.wav rate "$3" dither #-s
sox -V4 xres.wav -b "$bits" xlo.wav rate -v "$rate" trim 0s "$samps"s
sox -m xhi.wav -v -1 xlo.wav xdiff.wav
sox xhi.wav -n spectrogram -x 1920 -Y 1080 -o xphi.png
sox xlo.wav -n spectrogram -x 1920 -Y 1080 -o xplo.png
sox xdiff.wav -n spectrogram -x 1920 -Y 1080 -o xpdiff.png
echo "Read "$samps" samples"
echo "Processed as "$bits"-bit/"$rate"Hz"
echo "Applied "$gain" gain"

 

[tdog]

If this was a SOX list, that might be meaningful. A software package's default action is usually the easiest, not the best. I have posted enough links to verified research - where would you like to challenge that?

 

[RRod]

  If this was a SOX list, that might be meaningful. A software package's default action is usually the easiest, not the best. I have posted enough links to verified research - where would you like to challenge that?

 
I'm asking where in my workflow am I going against the principles in your research. And I am deliberately not using the default action of the software, precisely to get better results. And it's meaningful because it's the software being used to do the conversion that is currently under examination.
 
It's also interesting that your first link is to a xiph video, when Monty himself at one point says in the other video "no one ever ruined a recording by not dithering the final master."