Actually, let's talk about measurements for a bit.
One of the problems in audio is that most common audio analyzers (Audio Precision, Stanford, DScope) have a number of pre-set measurements that make determining things like THD, IMD, S/N ratio, crosstalk, etc relatively simple. Which means this is what most people focus on. And then we get into a numbers game that ends in PPM distortion numbers, etc, which, in our opinion, do not always correlate to great sound.
Now, these types of measurements are important. But you need to take into account two things when making audio measurements, besides the pre-set menus:
1. Is the system set up correctly?
First, whenever anyone says "making measurements is easy and absolute," they haven't read the 312-page manual that comes with a Stanford SR1. I like to ask, "So, how many thousands of hours do you have using an Audio Precision, Stanford, and DScope?" when people assert that measurements are absolute.
For example, something as simple as a bad cable can make your measurements completely meaningless. I've run into this a number of times. Same goes for bad jacks, power cables sitting on top of signal cables going to the analyzer, cellphones sitting next to the analyzer, ground loops, etc. Hell, we even blew up a generator output on one of our Stanfords somehow, and didn’t know it until a bunch of weird measurements led us to do a loopback test.
And measurement can look very different depending on how things are set. Is it spectrum-weighted? Are there hardware filters in place? What’s the FIR window? How much averaging? How many points? What’s the noise floor of the analyzer on wide bandwidth and narrow bandwidth? What THD spikes does the analyzer create?
The reality is, measurements aren’t 100% repeatable. Which is why we provide measurement summaries that are very conservative, rather than long printouts from the Stanfords.
2. Are you measuring the right things?
This is the big one. Looking at standard THD at 1K or 20K or CCIF IMD is one thing—and, to be fair, you still have to look at it over a wide bandwidth, not high-res measurements limited to 30 or 40kHz.
Looking at a multitone source (which the Stanford allows you to create—up combining several different frequencies at once) is much more revealing and interesting. We don’t publish this data because it doesn’t match any industry standards, and also because we’ve come to our own measurement regime that produces what we consider to be meaningful results. However, we could be fooling ourselves, so best to keep it internal.
Also, looking at jitter beyond a Jtest-style signal from the analyzer is more interesting. This is one case where you should not be using an audio analyzer—where it’s better to use an interval counter at the actual clock pins. Measuring things this way quickly makes you question the advantages of today’s buzzword-compliant femto clocks.
But these are only a couple of examples. Do we know all the measurements correlate exactly to a DAC or amplifier’s perceived performance? No. Do we have a better idea of how our products perform? Yes.
So, we’ll keep measuring…and we’ll keep listening…and we’ll keep driving some people crazy because we do both.