Part of the problem is that, since AudioDiffMaker is a digital product, it requires the samples it compares to be converted into a digital format (so the A/D converter used comes into the equation). The other question is about how it calculates the overall "null depth". Within a continuous repetitive signal, there will be a continuous steady difference.... for example, if one signal has 1% THD and the other has 2% THD, then there will be a steady and continuous difference of somewhere between 1% and 3% (depending on how the distortions present in each signal compare). However, if the differences aren't continuous, then the results may be difficult to compare or even to understand.
Let's take an obvious and simplistic example. My office and the office next door are about equally quiet.... However, once every ten seconds, I clap my hands together loudly in my office. Therefore, once every ten seconds, for a period of a few milliseconds, the noise level in my office is hundreds of times higher. So, the noise level in my office is 10,000% higher... for 1/10,000 of the time. If we were to express that as an AVERAGE, the average noise level in my office wouldn't be that much higher.... but it would be far different if we expressed it as a peak measurement.
When we're looking at analog signals, many types of issues, like THD and IM distortion, tend to be steady... while a few, like crossover notch distortion, may result in dramatic differences, but only for a tiny percentage of the time. Likewise, the ringing present in a digital filter is very time dependent. For most of the time, the signal is almost precisely what it should be... but, for a very tiny percentage of the time, the distortion is virtually 100% (there is output signal while the input signal is zero - so, for that brief interval of time, ALL of the output signal is pure noise or distortion).
Therefore, if I were to use AudioDiffMaker, I would want to actually see or listen to the actual null rather than just see something like the AVERAGE level of the null.
And I would also still be concerned that the response of the A/D converter might cover up differences that only occur for a short time during signal transitions.
At a minimum, I would suggest digitizing the signals to be compared at a much higher sample rate than the original signals were at.
So, if I wanted to compare the outputs of two DACs, while both playing a 44k CD audio file, I would digitize the signals to be compared at 192k.
(That way we can at least hope that an A/D operating at 192k can accurately resolve errors of the magnitude a DAC operating at 44.1k is likely to produce.)
More along these lines. If i'm using the software I need to know where the cutoff is to determine whether there is an audible difference between two components. If the correlated null depth between flac and 320kbps mp3 is 70db and I consider that barely audible then perhaps I should only worry about changes in my system that are less that 70db. Another option would be to use the same gear and change the FR to a level where you know you can detect a difference and use that as an audibility cutoff.
This is with the knowledge that Audio Diffmaker hears very differently than we do. Something that might be barely audible to us could be very different from it's perspective.
This is all less than scientific. I'm just throwing around ideas for people to play with. Recently I had a problem with my dac and know it either wasn't working correctly or there was some low-level interference and I couldn't tell the difference so maybe it is all moot for me if I can't hear that (could see it on the meters though!).
