24bit vs 16bit, the myth exploded!

[SoundAndMotion]

At 4-bit there is about 24 dB of dynamic range,

More if noise shaping and dither are used, which I assume @gregorio would request.

So, what are these important features?

Some clue about my signal... any evidence that it has been perfectly reproduced... see below.

However, it's perfect signal + really loud noise.

That sounds like doing evoked potential recording in the brain, which I've done. You do signal averaging. Your stimulus produces a tiny signal deeply buried in a lot of noise, so you repeat the process many times. We recorded from the brain stem, with surface (skin) electrodes, and got pretty good results with 1000 repetitions. The noise averages lower and lower and the tiny signal pops out. The larger the noise compared to the signal, the more repetitions required.

 

[gregorio]

I'm not contesting Shannon ... "Complete determination" plus a bunch of noise destroys the "complete determination", so it's wrong. It is not perfect.

This is contradictory! If "complete determination" is "wrong" or "not perfect", then YOU ARE contesting Shannon! Shannon's statement DOES NOT require some addendum or qualifications to be true, it is proven true as it stands.

You tell me exactly how you want me to prepare it, consistent with a simple test of your contention that bit depth doesn't alter the ability for "complete determination". Anything you want, ending with me sending you a file that is encoded at 4-bits and sampled at 44.1kHz. I will document each step with an intermediate, modified data file.

You don't appear to understand what noise-shaped dither is!

How you would need to prepare the signal would be: An appropriate noise-shaped dither applied at the point of quantisation, to redistribute the quantisation noise/error outside the range of human hearing (thereby revealing the "complete determination"). "Appropriate noise-shaped dither" would mean an algorithm designed with a suitable amount of "shaping" aggressiveness specifically for 4bit quantisation and a sample rate which allows a large enough audio bandwidth to accommodate the redistribution of such a large amount of noise. With SACD we obviously have an even larger amount of noise to redistribute but with a sample rate of 2.8mHz there's plenty of (inaudible) audio bandwidth within which to redistribute that noise. Even so, Sony required SACD players to implement an analogue LPF (@ 50kHz) post reconstruction, to remove most of that HF noise energy, as it was easily enough to cause IMD in downstream equipment.

G

 

[SoundAndMotion]

I really apologize, but I've grown very tired of this. Perhaps some sleep and some coffee will help... another time.
Some quickies:

1. So you're saying that "amplitude vs time" does not contain frequency information?

I'm saying "amplitude vs. time" is signal information in the time domain and "amplitude(+phase) vs. frequency" is the same signal information in the frequency domain. You must do a mathematical transform to go back and forth. In a digital audio file, e.g. .wav, there is no coding of the frequency content of the signal, just amplitude vs. time. Saying frequency domain is the same as time domain is like saying flour and water IS bread. It's not, unless you transform it. Look at this data:

Give me some idea about the frequencies**. How many are there? It's a small number. What are the frequencies? They are simple, round numbers. If I sent you this file, you could easily TRANSFORM it and answer. Otherwise, to answer your question above, yes, there is no frequency information in the graph, WITHOUT A TRANSFORM of the data.

It IS effectively perfect but with just one or two bits it would "really suck" because you'd barely be able to hear any of that perfect signal buried in the huge amount of quantisation error/noise. Of course, if you applied noise-shaped dither during the quantisation process then more of our perfect ("completely determined") signal would be exposed, as the quantisation noise is redistributed away from our range of hearing. This is the reason why SACD does not "really suck"!

I'm getting tired and bored with your "it's perfect, but with imperfections" arguments. SACD and DSD are 1-bit "delta-sigma"!!!! There are more than 2 output values, there are 2 delta values and many sigma values. It is also sampled at a dramatically higher sample rate. It is not the same as 1 or 2 bit LPCM.

** If you want to cheat, I'll tell you exactly how:
See if NIH Image still exists, or use your own image processing program.
Read in the image and kill everything not blue.
Do an AND with a 1-pixel wide black and white vertical stripe pattern.
Get the program (NIH Image will) to spit out the coordinates of each point.
Stick the data in MATLAB, LabVIEW, Mathematica, Maple, Octave... heck even MS Excel.
Do a Fourier TRANSFORM, careful with the time scale!
Post the answer.

 

[SoundAndMotion]

You don't appear to understand what noise-shaped dither is!

Not an experienced expert, but I understand it pretty well. I would use this algorithm:

algorithm noise_shape {
/*Input:
b_orig, the original bit depth
b_new, the new bit depth to which samples are to be quantized
F_in, an array of N digital audio samples that are to be
quantized, dithered, and noise shaped. It’s assumed that these are read
in from a RAW file and are values between
–2^b_orig-1 and (2^b_orig-1)-1.
c, a scaling factor for the noise shaping
Output:
F_out, an array of N digital audio samples quantized to bit
depth b_new using dither and noise shaping*/

s = (2^b_orig)/(2^b_new);
c = 0.8; //Other scaling factors can be tried.*/
e = 0;
for (i = 0; i < N; i++) {
/*Get a random number between −1 and 1 from some probability density function*/
d = pdf();
F_scaled = F_in / s; //Integer division, discarding remainder
F_scaled_plus_dith_and_error = F_scaled + d + c*e;
F_out = floor(F_scaled_plus_dith_and_error);
e = F_scaled – F_out;
}
}

If you have a better one, that's fine. I can upsample, apply TPDF (not TDPF), do noise shaping... whatever you want. BUT, in accordance with sampling theory, I'd only send 4-bit 44.1kHz, and I'd expect to see my signal in some form, not pure noise.

If you want to use the signal averaging I mentioned above. I could rerandomize the TPDF and send multiple files, to help you pull out my signal.

 

[gregorio]

Not an experienced expert, but I understand it pretty well. ... BUT, in accordance with sampling theory, I'd only send 4-bit 44.1kHz, and I'd expect to see my signal in some form, not pure noise.

Again, you are contradicting yourself. If you "understand it pretty well", then you would realise that "in accordance with sampling theory" 4bit 44.1kS/s (or did you really mean 44.1kHz? See, two can play at that game!) is insufficient. If you want to use such a few number of bits, the quantisation noise obscuring the perfect (completely determined) signal is so great you need to extend the sample rate to provide extended audio bandwidth to redistribute that noise. I'm not sure how much use this will be as you're apparently trying to disprove even the very foundation of sampling theory (the Nyquist-Shannon Theorem) but just in case, you should look up the Gerzon-Craven Noise-Shaping Theory. As I haven't the time to go into detail, a good starting place would be the introductory Lipshitz and Vanderkooy extract "Pulse-Code Modulation - An Overview" - particularly sections two and three, which deal with dither/quantisation and noise-shaping respectively.

G

BTW thanks, amplitude vs time contains no information in respect of frequency and bread contains no flour or water, got it!

 

[SoundAndMotion]

Again, you are contradicting yourself.

Karl Rove's Tactic #3? (Maybe you should switch to politics)
Let's see...

1. Shannon stated: "If a function x(t) contains no frequencies higher than B hertz, it is completely determined by giving its ordinates at a series of points spaced 1/(2B) seconds apart."
... (snip)
Shannon's statement does not require any additional conditions in order to be true, it is true, period!! For example, note that it does not mention bit depth, that's because it does not need to, Shannon's statement is correct as it stands, period (IE. Regardless of bit depth).

If you want to use such a few number of bits, the quantisation noise obscuring the perfect (completely determined) signal is so great you need to extend the sample rate to provide extended audio bandwidth to redistribute that noise.

(My bold) Hmmm... You two should get your story straight.

If you "understand it pretty well", then you would realise that "in accordance with sampling theory" 4bit 44.1kS/s (or did you really mean 44.1kHz? See, two can play at that game!) is insufficient.

Yes, I did really mean 44.1kHz sample rate, like RBCD. That gives 44,100 samples per second, so 44.1kS/s. Saying it's insufficient contradicts your previous quote about Shannon. I'm tired of the game you two are playing.

BTW thanks, amplitude vs time contains no information in respect of frequency and bread contains no flour or water, got it!

Actually, you didn't "got it". I think all caps threw you off... "TRANSFORM" is the same as "transform". Now go back and read it and you'll get what I said, and you can stop misquoting me.

Gregorio, you seem pretty smart and a lot of the stuff you write probably helps a lot of people. But you have to understand the limits of your knowledge, acknowledge and correct when you are mistaken (don't change your view and act as though it was always so), and quit saying so often that your take is "the truth and incontrovertible".
We all make mistakes.
But our exchange on the last few pages is going nowhere.

 

[bigshot]

But our exchange on the last few pages is going nowhere.

You only JUST noticed that?! Why do you ask questions you don't want answers to?

 

[SoundAndMotion]

Why do you ask questions you don't want answers to?

I do want the answers, but given the responses I got, I don't expect them.

 

[SoundAndMotion]

You only JUST noticed that?!

No...
If you READ the posts, you'll see my rising level of frustration with the responses. Been going on a while...

Anything meaningful to contribute?

 

[rule42]

Maybe someone could define a bit clearer as to what they mean by frequency domain. I get what Gregorio says about a simple relation between time and frequency but my understanding of signal processing in general seems to think that the frequency domain is defined as the set of data after a Fourier Transform is done, that involves complex numbers, negative frequencies and a whole bunch of maths way over my head, I think most image manipulation routines work on data in frequency domain like this then do an inverse with the signal still as was. Don't know so much about audio but have assumed a lot of the filters and the like work the same way.

 

[SoundAndMotion]

..my understanding of signal processing in general seems to think that the frequency domain is defined as the set of data after a Fourier Transform is done, that involves complex numbers, negative frequencies and a whole bunch of maths way over my head,

my bold

Yep, that's basically it. In the context of audio, you start with a data series of amplitude vs. time (time domain), do a FT and you do get complex values with negative frequencies, but you can fairly easily make it useful to you.

You discard the last (N/2 - 1) points (negative frequencies, N is number of points) and convert the complex values to polar form (most numerical packages have a simple command). If your time domain data is real (not complex, and in this case it is), then the negative frequencies duplicate all but 2 of the positive frequencies (well the complex conjugates, and counting zero as positive).

Then for each frequency, except the first (zero) and last (fs/2)(fs is the sampling frequency), you get the magnitude (modulus of complex value) and phase (argument of complex value). The first and last don't have phase. You must scale all values based on the number of data points.
You can plot this (it's called a Bode plot, and there are 2 graphs) and see the magnitude vs. frequency (commonly called spectrum or frequency response) and the phase vs. frequency. This is the frequency domain.

It is a lossless process (other than numerical roundoff errors of your computer) and can be reversed.
Freq. domain data --> Inverse Fourier transform --> Time domain data (if you set up the data correctly)

A lot of analysis and manipulation (like filtering) can be done in either domain, but yes filtering is computationally and conceptually easier in the frequency domain.
Does this help, or was I too complex ?


EDIT: many numerical packages do everything I just said for you and go straight from time domain data to frequency domain Bode plots!
EDIT 2: Audacity will give you a spectrum plot from time domain data.

 

[gregorio]

[1] You two should get your story straight.
[1a] Yes, I did really mean 44.1kHz sample rate, like RBCD. That gives 44,100 samples per second, so 44.1kS/s. Saying it's insufficient contradicts your previous quote about Shannon.
[2] I'm tired of the game you two are playing.
[3] We all make mistakes. ... [3a] But you have to understand the limits of your knowledge, acknowledge and correct when you are mistaken ... [3b] and quit saying so often that your take is "the truth and incontrovertible".
[3c] But our exchange on the last few pages is going nowhere.

[1] Who "two"? Me and Claude Shannon, Nyquist and Shannon? Don't forget Stephen Hawking and the there's Wittaker, Kotelnikov and others who contributed or discovered the theorem independently. As for getting our "story straight": You are confusing YOUR inability to understand the story with the story not being straight. The "story" is (again!): The signal IS "completely determined" plus there is ALWAYS some amount of noise; in theory zero noise would require an infinite number of bits and in practise even with an infinite number of bits there is still some amount of noise, due to the laws of physics (Thermal noise for example)! The real issue then is one of how much fidelity: How much of that "completely determined" signal do we want to recover or conversely, how little noise do we want obscuring that "complete determination"? You want to recover a significant amount of the "complete determination" AND use a bit depth which introduces a great deal of quantisation noise. Digital Audio/Sampling Theory allows for this scenario, because quantisation noise is effectively separate from the "complete determination" (perfect signal). The (band limited) "complete determination" can therefore be recovered/exposed (to any arbitrary level above thermal noise limits), by moving/redistributing the quantisation noise away from the frequency band occupied by the "complete determination" (as defined by the Gerzon-Craven Noise-Shaping Theorem). It should be obvious therefore, that you need to provide enough audio bandwidth to accommodate both the band of frequencies occupied by the "completely determined" signal + the band of frequencies occupied by the (redistributed) quantisation noise. The large amount of noise introduced by 4bit quantisation would need to be redistributed over a large band of frequencies, the vast majority of which would need to be in the ultrasonic range and therefore a far higher sample rate would be required to accommodate that audio frequency band.

Why don't you read the EVIDENCE ALREADY PRESENTED? Specifically, page 13 of the Lipsh*tz-Vanderkooy extract linked previously where two examples are given, an example with just 1 bit and another with 8 bits. The latter for instance demonstrates the perfect ("completely artefact free") recovery of the "complete determination" of any signal within an audio band of 0-20kHz with a SNR of 120.4dB using just 8bits, plus a sample rate of 176.4kS/s to redistribute the quantisation/dither noise (to above 20kHz).

1a. Saying it is insufficient does NOT contradict my previous quote of Shannon, it ENTIRELY agrees with it!

2. The game us two are playing is called Science, if you're "tired" and want to play a different game you're in the wrong forum!

3. By "we all" do you mean everyone except you?
3a. But you don't have to?
3b. No, YOU quit saying it's "your take", it is NOT my "take", it's Shannon's "take", I'm quoting him exactly and directly, with nothing added or taken away.
3c. I did not say (Shannon's "take") was incontrovertible, I said it was "logically incontrovertible". The reason our exchange is going nowhere is because you are effectively illogically controverting it! You've presented no evidence al all, only an example which you don't believe is possible but which the evidence I've presented (Lipsh*tz-Vanderkooy extract and the Gerzon-Craven Theorem) indicates it's entirely possible, plus of course, SACD actually demonstrates it!

G

 

[gregorio]

I get what Gregorio says about a simple relation between time and frequency but my understanding of signal processing in general seems to think that the frequency domain is defined as the set of data after a Fourier Transform is done, that involves complex numbers, negative frequencies and a whole bunch of maths way over my head

Yep, it is pretty simple. Here is a graph of amplitude vs time:

As frequency is just cycles per second (CPS or more commonly Hz), then "amplitude vs time" inherently contains the information for the frequency. In the above example, we can follow the amplitude over time and see that a "cycle" completes in 1 millisecond, it therefore has a frequency of 1,000 cycles per second (or 1kHz). We could have done it the other way around; not given you the time (not labelled the x-axis) but informed you it was a 1kHz signal and then you could have followed the amplitude to where the cycle completes and inserted the time yourself (1 millisecond).

From the above we can see that "frequency" and "amplitude vs time" are interchangeable. However, in real life we're rarely dealing with a single frequency, we're usually dealing with a whole bunch of simultaneous frequencies. All those frequencies combine (sum) to form a single "complex waveform", rather than the simple sine wave above. All the frequency information is still there but it needs to be calculated, split out into it's constituent frequencies. This calculation is done by means of a "Fourier Transform" (FT) and the result is called the "Fourier Series", after this calculation (FT) we are in the "Frequency Domain". Below we see a complex wave form but after a Fourier Transform we see that it's actually still relatively simple, just two frequencies of different amplitudes summed together (500Hz and 800Hz).

You seem to already know/understand the above but it might be useful for others.

G

 

[rule42]

@SoundAndMotion and @gregorio . Thank you both for the replies. All nice and clear. No, about the right level of complexity for now before I delve into it a bit deeper. I like trying to follow examples through as a way of learning, so do you think that Octave would be the best free tool for that? I've not used Matlab but have some familiarity with Python from over 20 years ago (including Numpy and a little Scipy) and they don't seem too dissimilar. Or stick with Python of course.

 

[bigshot]

Frequency is dependent on time. You don't have one without the other. That is what sampling rate is all about. That should be obvious.