[MWSVette]

Oh, then no one cares that the bit depth has increased by four orders of magnitude? My bad, I thought that kind of stuff mattered to audiophiles

Yes, I am sure that audiophiles do care about measurements that matter.

The thing is this one does not...

 

[Makiah S]

Actually, the ASR measurements and Atomic Bob's agree for the Yggy version 1, linearity is the same. It is for the Yggy 2 where the linearity differs. ASR's measurements looks almost like they are two sets of the Yggy version 1 since the linearity is nearly the same, and something is off with the THD+N, maybe it was an issue with that particular unit. @amirm has offered to remeasure if someone would loan him a Yggy 2. ASR's THD+N for the Yggy 1 actually agrees pretty well with Atomic Bob's THD+N measurements for the Yggy 2 for the single-ended outputs.

So, before we start attacking people again, saying Schiit is the evilest organization known to humanity or @amirm is a hack who hates Schiit and HF and the world, can we focus on the measurements? There is a lot of agreement here and some consensus could be achieved if everyone just stopped. with. the. character. attacks. just stop.

Linearity at around -95 dBFS (red line on Atomic Bob's graph):





THD+N vs frequency for single-ended at around -87 dB (purple on ASR graph):

I don't think he's evil, and I'm not attacking him, I'm just disappointed. He's had a bias and it's becoming more manifest now than in the past! Which is a shame, he has some good content! I enjoyed his EL Dac measurements but his credibility will be questioned more often now as a result of this. But I'm not here to attack him or schiit.

That's good the Yggy 1 measurement Match, what is the date on them? I mean I enjoy some of his content, the speaker stuff is quite good

@L0rdGwyn, please read my response to @Mshenay below.



Thank you, @Mshenay. Speaking of reddit and bias, @amirm posted the following on reddit, in response to a poster named frizo who suggested @amirm was cherry-picking by not showing balanced measurements. Among the things @amirm said in his response:



And this gem:



But did we only measure the balanced outputs? I'm quite certain @amirm knew very well that I showed both unbalanced and balanced measurements for frequency response, THD+N, and linearity -- and that Bob showed far more measurements than that from both outputs (not to mention different inputs).

So why would @amirm suggest otherwise to the reddit folks? I think it's because he assumes most will take him at his word that we only measured balanced outputs (and most there did believe him) and will not bother to verify for themselves. And I believe he also thinks that many there probably don't understand some of this measurement discussion, and so, again, will take him at his word (again, most did).



I don't think it fair to say Bob's measurements "nearly mirror" @amirm's for the Yggdrasil 2. Looking at two of the basic measurements (frequency response and THD+N) which he uses to illustrate what he sees as two major problems with the Yggdrasil 2, his measurements are actually not like Bob's or mine. His THD+N plot shows a >20 dB difference at 20 Hz (which I'll revisit in a minute).

Additionally, @amirm complains of the downward THD+N slope, most of which can obviously be explained by the significantly higher THD+N he's showing in the lower frequencies (versus either measurement from Bob and me). Additionally, it appears to me that he measured THD+N all the way out to 20 kHz and may not have had his analysis bandwidth set wide enough to even include the higher frequency harmonic distortion (the "THD" in THD+N) through that range.

The Yggdrasil 2's noise floor is quite low, so the THD+N measurement will be dominated by THD -- as such, if you don't include the THD going into the higher frequencies (yet you show an X-axis that goes out to 20 kHz), then, yes, it'll downward-slope as you increase frequencies to the point where their harmonics exceed the upper range of the analysis bandwidth. I posted an example of the difference between my THD+N measurement with bandwidth set to 90k (Fig.4, solid line), and then with bandwidth limited to 22.4k (Fig.4, dashed line). You can see that the more bandwidth-limited THD+N reading starts to separate from the other one quite early (<40Hz) as the high-order harmonic frequencies become scarcer as the sweep frequency increases closer to the bandwidth limit.

Look at the THD+N measurements below, make sure to keep in mind that @amirm and I are measuring from 10 Hz to 20 kHz (though, again, I strongly suspect his measurement bandwidth is set far lower than the 90k I had set), and @atomicbob is measuring from 20 Hz to 10 kHz. If you look at the level of THD+N at 20 Hz, @amirm is showing around -61 dB (Fig.1). If you look at @atomicbob's at 20 Hz, he's showing around -82 dB (Fig.2). If you look at mine at 20 Hz, I'm showing around -82 dB at 20 Hz (fig.3). In other words, @amirm is showing >20 dB higher THD+N at 20 Hz than the measurements from both Bob and me. @amirm's THD+N doesn't fall below -80 dB until around 400 Hz, again, accounting for most of the downward slope he's talking about.


Fig.1 @amirm's THD+N measurement, unbalanced output (10 Hz to 20 kHz)


Fig.2 @atomicbob's THD+N measurement, unbalanced output (20 Hz to 20 kHz)


Fig.3 My THD+N measurement, unbalanced output (10 Hz to 20 kHz)


Fig.4 My THD+N measurement, unbalanced output (10 Hz to 20 kHz), with an overlaid plot (dashed lines) showing the same measurement with bandwidth limited to 22.4k (versus 90k with the solid line).


And then there's the frequency response...


Fig.5 @amirm's frequency response measurement, unbalanced output (10 Hz to 20 kHz)


Fig.6 @atomicbob's frequency response measurement, unbalanced output (20 Hz to 20 kHz)


Fig.7 My frequency response measurement, unbalanced output (10 Hz to 20 kHz)

@amirm's unbalanced output frequency response (Fig.5) has an unusual concave sag <300Hz, and a greater overall dip than either mine (Fig.7) or Bob's (Fig.6). Of this, he says:



And then to assert that this is an audible problem, he continues:



Neither Bob nor I showed a frequency response that came close to crossing that threshold within the audioband. Bob even magnified the frequency response measurement to make clearer the deviation from flat, and you can see this below (Fig.8).


Fig.8 @atomicbob's frequency response measurement with magnified Y-axis, unbalanced output (20 Hz to 20 kHz)

While my measurement is not magnified (Fig.7), the the deviation on that measurement is +/- 0.139 dB for the left and +/- 0.136 dB for the right from 10 Hz to 20 kHz. Within the audioband (20 Hz to 20 kHz), that deviation is even smaller at +/- 0.058 db and +/-0.056 dB. So, whether restricted to the audioband -- or even going lower than that to 10 Hz -- the deviation is well below the threshold that @amirm cited to suggest an audible problem.

So, do Bob's measurements "nearly mirror" @amirm's? No. Again, this is not at all similar to what either mine or Bob's measurements show. Nevertheless, @amirm invokes Bob's measurements (in my opinion underhandedly) to suggest Bob's concurrence with his measurements (vis-a-vis mine) when it's convenient to his argument. Why on earth would he do this? Again, I think @amirm knows the reddit readers will likely take him at his word (most there did), and will not likely check for themselves.

He then goes on to say that measuring the balanced outputs is not necessary (or even represents cherry-picking -- see his quote above) because he says most people use unbalanced outputs. How does he know this about Yggdrasil users? Also, since he posted a frequency response from the balanced outputs of the Yggdrasil 2, the DAC was at one point hooked up that way on his test bench -- why not show the rest?



Really?

Even if it was true that most Yggdrasil 2 owners used unbalanced outputs (and not balanced), it made me think of another analogy: Last week a friend took me to an autocross track in a Roush Mustang to do some laps. This Roush Mustang had several different performance settings selectable via a knob. I don't remember the exact mode names, but there was something that was more compliant, Sport, and Track. If you're curious about how fast that car could lap a given track in its more compliant, more tame mode, that's fine. But is that representative of what the car can really do around the track? If you know there's a performance difference between the settings, simply show both. But to show what it's really capable of, you'd have to at least show Track.

Now, if you choose to only measure this car's track time using the more sedately tuned, more softy-sprung mode because that's all you care about and that's the only mode you'll use, and that information is solely for your personal perusal and consideration, that's fine. However, if you're going to make a post reaching thousands of people, or hundreds of thousands (or even more) to discuss this car's performance level, I think most would agree that at least including the Track-mode lap time would be the fairer approach.

All in all, I'm actually less bothered by @amirm's measurements than some of the other things he says that I feel are quite deceptive. While his measurements might be explained by a bum Yggdrasil 2 unit (of all people, perhaps Murphy's law would just so happen to put a bum one in his hands), I believe many of his quoted statements above can not be described as anything but far less than forthcoming.

So, yes, @Mshenay, I do also have some funny feelings about some of the things I'm seeing here. And as I said before, I'm feeling a sense of déjà vu with this one.

I agree, you likely know that your and the "other" site often have some small discrepancies. An yes we usually talk about it because the devil is in the finest details especially in High end Audio, that said though, it's again disappointing that ASR would for what ever reason post such EXTREME set of measurements! I get going again'st the grain for the sake of views, but this much of an extreme just in-validates future content from him/them

I know in the past regarding headphones others have even regarded your measurements as the "outlier" but even then your data was still relevant to the overall sample! Different yes but not... SO VASTLY out there. An for a hobby that often exaggerates the smallest details it's disappointing ASR or rather @amirm choose to go this route, oh well I might still check in from time to time as more data is helpful. But I have to ask how much of what he's posted is valid? There's some content or measurements that's exclusive to their site, how credible are they?

Oh well. I hope he's able to re-establish his credibility as he has an exceptional set of hardware to measure with, and I believe he has quite a bit of experience! An I personally would like a third credible community to further add to the discussion and content creation for Amps DACs and the like! Though I don't think at this time ASR will be that community

 

[Alcophone]

He then goes on to say that measuring the balanced outputs is not necessary (or even represents cherry-picking -- see his quote above) because he says most people use unbalanced outputs. How does he know this about Yggdrasil users? Also, since he posted a frequency response from the balanced outputs of the Yggdrasil 2, the DAC was at one point hooked up that way on his test bench -- why not show the rest?

I think the intention of his website is to provide objective criterions for purchasing decisions. I think it is true that most people in general have single ended systems. And if you're simply looking to get a better DAC, you may not be willing to upgrade the rest of your chain as well just to make it balanced - especially if the DAC sets you back $2400 + sales tax. As such, the single ended measurements are certainly of value, and maybe of value to a larger audience than the balanced measurements. And if the Yggdrasil is indeed not that great when using its single ended outs, then it would be difficult to recommend it for that use case.

That said, I agree with you in the sense that the Yggdrasil is clearly meant to be used balanced, and does perform its best this way, so it is definitely essential to provide balanced measurements as well (and justifiable to provide only balanced measurements). Some people have balanced systems, some are willing to go there, and some are planning new systems. So seeing Yggy's balanced performance might be the reason to shoot for a balanced system throughout.

 

[433521]

What is Yggy 2? Is this a misnomer for Yggy with analog 2 and usb 5? Or is there a new Yggy?

 

[Alcophone]

What is Yggy 2? Is this a misnomer for Yggy with analog 2 and usb 5? Or is there a new Yggy?

Yeah, somehow people started calling the Analog 2 version Yggy 2. Probably because there's never going to be an actual Yggy 2:

Can't speak for Raggy but certainly I am the last word on an Yggy 2: Over my Dead Body.

 

[433521]

Now that's a statement!

 

[ProfFalkin]

Oh. It makes sense now.

 

[adydula]

Finally another black hole plugged up....now back to listening with my Schitt.....

Alex

 

[jude]

Yesterday, Dan Foley from Audio Precision (he's also the president of ALMA International, the International Association of Loudspeaker Manufacturing and Acoustics) and Chris Gill (also of Audio Precision) stopped by Head-Fi HQ for a visit.

While here, Dan took the time to discuss linearity measurements, as he knew I had asked the AP team many questions about this topic last week. He reviewed the revised linearity measurements I posted a few days ago, which were made with a lot of help and guidance from his colleagues at Audio Precision (especially Dan Knighten, the Vice President Business Development & Product at Audio Precision).

Dan (Foley) looked at the measurements we made here, the settings of those measurements, and confirmed that they were made correctly (not surprising, especially given the guidance of his colleagues).

To give another point of comparison -- another measurement to compare to what we'd already done -- Dan suggested a very simple type of linearity test which involves doing nested FFT measurements at declining dBFS levels. (Read my previous post if you want a little more explanation of "dBFS.") We hooked up the Yggdrasil 2 to the Audio Precision APx555, since we had ample data for that DAC with which to compare. Dan then set the analyzer to do successive FFT measurements with a 1 kHz sine stimulus beginning at -130 dBFS, and then another at -131 dBFS, and then another at -132 dBFS, and so on, until we reached -140 dBFS. (We first went with XLR digital input and XLR analog outputs.) This is what we saw (Fig.1):

Fig.1 Nested FFT measurement of Yggdrasil 2 (XLR digital input, XLR analog output), from -130 dBFS to -140 dBFS. FFT length = 96K.

What you're looking at is the very tip of the 1 kHz spikes in FFT view, zoomed in a lot -- like looking at the tip of a needle through a powerful microscope. For this measurement, the analysis bandwidth was the audioband -- we did not bandpass-filter as we had done with the bandpass level sweep linearity test. However, by zooming into the 1k spike -- by looking at such a narrow window -- it might be described as a sort of visual bandpass filtering, observing only the 1 kHz test signal.

For the Y-axis (in Fig.1 above), I set the reference level (0 dBrA) to the DAC's balanced output at -130 dBFS (1.163 µVrms) so that we could more easily observe if the output of the test signal from the Yggdrasil 2's analog outputs was correspondingly decreasing each step by 1 dB with each successive nested FFT measurement. As you can see, while not exactly perfect, the levels were indeed dropping by a decibel (within fractions of a decibel) with each step.

Using the same project file, I decided to run this test again, this time starting at -100 dBFS (which we've established in the latest linearity measurements the Yggdrasil 2 is solidly linear to), and then dropping from there to -140 dBFS, in decreasing 1 dBFS steps with each measurement. I also increased the FFT length to 1.2M points for even finer detail on the frequency axis (X-axis). Again, 0 dBrA = 0 dBFS (4.261 Vrms @ 1 kHz). Following is the result (Fig.2):

Fig.2 Nested FFT measurement of Yggdrasil 2 (XLR digital input, XLR analog output), from -100 dBFS to -140 dBFS. FFT length = 1.2M.

NOTE: If you want to see the above measurement down to the floor (with same X-axis span), click here.

I put a cursor on the last step where the output level was within 1 dB linearity, and that last step was -124 dBFS (2.418 µVrms), where the deviation was -0.920 dB. Also, given the finer points available by increasing the FFT length to 1.2M, I was able to zoom in on the frequency axis (X-axis) to an absurdly detailed level. The leftmost extreme of the X-axis is 0.9995 kHz, and the rightmost extreme is 1.0005 kHz, meaning the entire X-axis spans only 0.001 Hz (or 1/1000 Hz). Again, this is essentially an extremely magnified view of the tip of the 1 kHz spike(s).

Here's the same nested FFT measurement run using the the Yggdrasil 2's unbalanced (RCA) output (Fig.3):

Fig.3 Nested FFT measurement of Yggdrasil 2 (XLR digital input, RCA analog output), from -100 dBFS to -140 dBFS. FFT length = 1.2M.

NOTE: If you want to see the above measurement down to the floor (with same X-axis span), click here.

I put a cursor on the last step where the output level was within 1 dB linearity. That last step was -125 dBFS (1.045 µVrms), where the deviation was -0.942 dB.

While the balanced output does cross the +/- 1 dB error threshold before the unbalanced output (-124 dBFS and -125 dBFS, respectively), the balanced output does show better overall linearity down to -140 dBFS. You can see this when you compare the lowest levels of the above two measurements (Fig.2 and Fig.3), where the unbalanced output loses more symmetry at the bottom. You can also see that the balanced outputs maintain more symmetry along the outer edges on both sides.

These results were very similar to the results from bandpass level sweep linearity tests I previously posted. In the bandpass level sweep plot showing both the unbalanced outputs (dotted lines) and balanced outputs (solid lines) overlaid, both the balanced and unbalanced outputs actually cross the 1 dB error threshold at -125 dBFS (Fig.4 below). However, you can see that the overall linearity of the balanced outputs is better, especially down to the lowest levels of that measurement.


Fig.4 Schiit Yggdrasil 2 linearity error (dB deviation versus dBFS) from its unbalanced outputs (dotted lines) and balanced outputs (solid lines), shown together.

Now one might ask at such absurdly low testing levels is there even a 1 kHz sine waveform present? To answer the question, let's look at a 1 kHz signal at -130 dBFS from the balanced outputs. I chose this level because it's below (worse than) the +/- 1 dB linearity error threshold we've been using, for either balanced or unbalanced outputs.

From the Yggdrasil 2's balanced outputs, the voltage of a 1 kHz sine signal at -130 dBFS is around 1.2 µVrms. From my previous post, we mentioned that the analyzer's noise level (across the audioband) is around 1 µV, and the Yggdrasil's around 5.6 µV. With a 1.2 µVrms signal, we're well below the noise level of the DAC, and approaching that of the analyzer. So going to the scope view, with the noise across the audioband in the mix, this is what a 1 kHz sine wave looks like at -130 dBFS from the Yggdrasil 2's balanced outputs (Fig.5):

Fig.5 Schiit Yggdrasil 2 output of 1 kHz sine at -130 dBFS (from its balanced analog output), with no bandpass filtering.

While you do see some shape and periodicity that might suggest a sine wave, it's obviously not cleanly sinusoidal. And this is where we get to something I discussed in the previous post. If you want to measure something that's below the noise levels of both the DAC and analyzer, you can limit the passband to the stimulus we're analyzing. We know we're looking at a 1 kHz sine signal, so setting an elliptic high-pass filter at 1 kHz, and an elliptic low-pass filter at 1 kHz, we're going to be very tightly analyzing essentially just the 1 kHz stimulus, and avoiding the noise from the rest of the audioband. Looking at the scope view after filtering, this is what a -130 dBFS 1 kHz sine looks like from the Yggdrasil 2's balanced analog outputs (Fig.6):

Fig.6 Schiit Yggdrasil 2 output of 1 kHz sine at -130 dBFS (from its balanced analog output), with 1 kHz elliptic high-pass filter and 1 kHz elliptic low-pass filter applied.

So if you were wondering if the Yggdrasil 2 is actually decoding a 1 kHz sine at a very low level like -130 dBFS, the answer is yes. Sure, it is well below the Yggdrasil's noise level across the audioband -- and approaching the APx555's noise level -- but, as you can see, it is decoding it (with a linearity error of -1.306 dB (-131.306 dBrA)). Just to clarify: In both of the measurements (Fig.5 and Fig.6), what's coming out of the Yggdrasil 2 is exactly the same -- what I changed is the measurement bandwidth on the analyzer's input side.

Thanks to Dan Foley of Audio Precision for his idea to look at this from still another angle, which ended up dovetailing nicely with the bandpass level sweep linearity measurements from my previous post.

Okay... I'm done talking about linearity for now, especially where this DAC is concerned.

We've now discussed and compared the frequency responses from both sets of outputs, the THD+N from both sets of outputs, and linearity from both sets of outputs.

NOTE: There are a lot of numbers in this post, so if you catch any errors, please let me know.

 

[jude]

I've been asked a lot about the linearity measurements I posted: Why the different ones? What do they mean? Which is more right? Etc. Etc.

I think one of the reasons why I'm being asked this is because @amirm said my first linearity error measurements were inaccurate. Let me post those again (they were originally included in this post from 2018-06-07):


Fig.1 (above) Linearity error measurement of the Yggdrasil2 from its unbalanced analog outputs (281 steps, 0.500 dBFS per step, from -140 dBFS to 0 dBFS). Measurement bandwidth is 20 Hz to 20 kHz (elliptic high-pass filter @ 20 Hz, elliptic low-pass filter @ 20 kHz).


Fig. 2 Linearity error measurement of the Yggdrasil2 (below) from its balanced analog outputs (281 steps, 0.500 dBFS per step, from -140 dBFS to 0 dBFS). Measurement bandwidth is 20 Hz to 20 kHz (elliptic high-pass filter @ 20 Hz, elliptic low-pass filter @ 20 kHz).

Both of the above linearity measurements were made using the Stepped Level Sweep measurement as configured in Audio Precision's APx software. As you can see in each description above, these measurements include the audioband (from 20 Hz to 20 kHz), which then includes the presence of the noise and distortion in the audioband.

As you go lower in level (moving left on the X-axis), the stimulus signal (1 kHz sine in this measurement) gets smaller and smaller, and eventually becomes lower than the noise floor of the Yggdrasil 2 (with around 5.6 µV of noise from the balanced outputs at this bandwidth). The reason the error increases (positive error) as you go lower is because of the noise. The smaller and smaller the stimulus signal is supposed to be (with the decreasing dBFS level), the greater the static noise is relative to the target level -- so once you dive below the noise floor, there's an increasing positive error as you go left on the x-axis, as seen in Fig.1 and Fig.2 above. (The cursor lines you see indicate where the linearity error crosses a +/- 1 dB error threshold.) In other words, the further you go below the DAC's noise floor, the more you're measuring the increasing static noise above the sinking stimulus target level -- hence the positive error.

The reason I bring this up is because @amirm stated on his website that my linearity measurements (as shown in Fig.1 and Fig.2 above) were inaccurate, stating:

Careful! Have you seen documentation how that linearity test is performed on APx555? This is very important because out of the box, APx555 measurements for linearity are completely unsuitable for this use. The analyzer itself heavily influences the measured results making the outcome useless.

To demonstrate that, I provide the stock linearity measurements on APx555 together with my customization to get me to where my older analyzer is at:

This is a loopback test meaning the Audio Precision APx555 is measuring its own performance. The connection from input to output is direct (so not even a wire).

Now look at at the gray/blue curve labeled unoptimized. This is the stock measurement of Linearity that comes from Audio Precision. As we see as soon as you go to levels below 100 dB, the error starts and it quickly climbs way high (well above 10 dB which I have picked to match Jude's graph).
What this means is that the AP has a positive error in "linearity" (It is more than that but let's go with it) at levels less than 100 dB. As such, you can not, let me repeat, NOT measure any DACs with it as you will be showing the sum total error of both the DAC and ADC measurement errors in the Audio Precision.

The graphs in my measurements of Schiit came from my much older Audio Precision analyzer which in this care, provides much better results. Reason is that my older analyzer is a combination of an analog analyzer and digital one. For the purposes of linearity tests, both are combined to produce spectacularity clean analysis of linearity down to -120 dB as you see in my graphs. The contributions from the analyzer are in the order of half a dB or less at the extreme.

Fortunately, if you know signal processing, and understand how linearity is measured, you can correct for most of the error in APx555 as I am showing in red graph. Notice how the error is less than 0.5 dB or so extending to -140 dB. I am still fine tuning this measurement to make it even more accurate. But for now, I don't accept linearity measurements from APx555 without documentation on how they are performed. And you should not either.

To wit, if I feed the stock measurement a DAC that as negative linearity error, it can cancel the error in the APx555 and incorrectly show it to have good linearity!!!

Summary
Some of you scrutinize my measurements left and right every which way. But you put your guards down completely when you saw other measurements to your liking. Alas, in this case, we are talking about a very delicate measurement that digs deep, way deep into what the DAC is doing and with it, lots of signal processing and advanced toppings like that come into play. So while the measurements may be correct, you cannot rely on them without further documentation that demonstrates any error is from DAC and not ADC+DAC.

A few key quotes from @amirm in that post that I'll address:

• "out of the box, APx555 measurements for linearity are completely unsuitable for this use."
• "As such, you can not, let me repeat, NOT measure any DACs with it as you will be showing the sum total error of both the DAC and ADC measurement errors in the Audio Precision."
• "Fortunately, if you know signal processing, and understand how linearity is measured, you can correct for most of the error in APx555 as I am showing in red graph. Notice how the error is less than 0.5 dB or so extending to -140 dB. I am still fine tuning this measurement to make it even more accurate. But for now, I don't accept linearity measurements from APx555 without documentation on how they are performed. And you should not either."
• "To wit, if I feed the stock measurement a DAC that as negative linearity error, it can cancel the error in the APx555 and incorrectly show it to have good linearity!!!"

Let me address those in order:

• The APx555 is Audio Precision's flagship analyzer. It is perfectly suitable for linearity measurements -- even out of the box. (I'll demonstrate this shortly.)
• The error he speaks of is the transition to a positive error in Fig.1 and Fig.2 -- the upward turn -- as you get to the very low dBFS levels (moving left on the X-axis), and he suggests this is due to "error" in the the APx555. That upward turn is due primarily to noise, and in this case it's the noise of the DUT (device under test) that is largely defining that positive error (not the APx555's noise).
  • Sometimes, for the levels the DAC can linearly decode down to, you reach analog output levels not just below the DAC's noise floor, but also sometimes below the analyzer's noise floor.
  • If you want to see how linearly the DAC is decoding (even below the noise floor), you have to limit the bandwidth of the measurement. To be clear, you're not changing a thing about what the DUT is doing, you're changing what it is you're measuring on the analyzer's input side -- or specifically in this case changing how much bandwidth to either side of the stimulus signal you're measuring.

Since @amirm posted this loopback measurement (in the quoted post from him above) as an example of what to do (and what not to do)...

Fig.3 (above) @amirm's APx555 loopback measurement, with and without bandwidth limiting as included in his quoted post above.

...it was obvious that he was suggesting some kind of aggressive bandwidth limiting. You simply can't be flat (+/- 0.5 dB) that far below the analyzer's noise floor without bandwidth-limiting the measurement.

Since @amirm did not like the linearity measurements I showed (in Fig.1 and Fig.2) -- ostensibly for the noise-swamping of the measurement at levels below the DAC's noise floor -- I decided to post additional linearity measurements with all of this in mind, and with the help of Audio Precision after contacting their technical support team. They recommended I use a measurement called the Bandpass Level Sweep. They later came to me with some very quick adjustments (literally a couple of drop-down selection adjustments that take just seconds to make) that further optimize the test for this specific measurement. (I will compare the result to the bone-stock result later. I will also then show you a comparison of this result with @amirm's own self-touted custom settings using Benchmark DAC3 linearity measurements for the example, since he still has one on-hand, as do we.)

Some words from the Audio Precision APx500 software user's manual about the Bandpass Level Sweep:

The Bandpass Level Sweep measurement provides a sine wave stimulus signal that is moved across a range of levels in a series of points. The DUT output is acquired by the analyzer, and filtered using a band- pass filter...

...Selectivity: In the Bandpass Level Sweep measurement you can choose from a number of bandpass filter widths using the Selectivity control. The list is ordered from narrow- est at the top, to widest at the bottom. The Window Width and x Octave filters are tuned according to the Bandpass Tuning Mode setting.

• Window width:
  This is the window width of the underlying FFT, typically only a few hertz wide. This selection has very steep skirts and a flat top.
• 1/24 octave:
  This selection has octave apart...

In the version we used, the Selectivity setting was changed from 1/24 octave (default) to Window width, for a stricter, narrower measurement window around the stimulus signal. Here are the linearity measurements I posted using those settings on 2018-06-15 (Fig.4 below):


Fig.4 (above) Schiit Yggdrasil 2 linearity error (dB deviation versus dBFS) from its unbalanced outputs (dotted lines) and balanced outputs (solid lines), shown together using modified Bandpass Level Sweep.

So here we're doing what @amirm's loopback measurement above in Fig.3 seems to strongly suggest must be done. We're bandwidth-limiting the measurement. As it turned out, he didn't like this one either. Among the things he said in response to this:

• These measurements can be complicated and lots of knowledge is necessary to understand them and the capability of the equipment. As I pointed out to Jude and even explained to Audio Precision folks, you can't just push a button on a test and expect the results to be correct.
  The APx555 that Jude is using out of box is NOT setup to produce these measurements. The settings need to be modified significantly to produce correct results for DACs. I am in the good position of having that machine and my last generation Audio Precision analyzer so I can cross check the results. I also know how the DACs and measurement gear operate so can spot anomalies in data. I routinely cross-check my results by testing other audio products at the same time and make sure the results "make sense."
  Jude is used to simple measurements like frequency response where we are dealing with very strong signals. With DACs we know they can all produce "the loud." It is the fainter signals that need measuring and there, it is so easy to mistake analyzer issues for the device being tested.
  I will explain all of this there if they remove their current stance of moderating my posts.

• My interpretation is that he is saying...
  • ...that the APx555 is not suitable for linearity measurements out of the box;
    • My response: It is. I'll show this later.
  • that one needs access and knowledge of one of Audio Precision's 25-year-old SYS2522 analyzers and the new APx555 to have any chance at making proper linearity measurements with the APx555;
    • My response: Bollocks.
  • that my familiarity with strong-signal measurements is misleading me at these low levels;
    • My response: He's no doubt far more experienced than me, but I do understand what we're doing here, and I'll show later that getting proper linearity measurements with the same analyzer that he's now using (the APx555) did not require that I have his level of experience. Contrary to what he's stating, I had already posted proper linearity measurements of a few different types prior to his asserting that.
  • that (as in his quotes at the start of this post) one must have his level of signal processing experience, to understand how linearity is measured so that one can correct for most of the "error" in APx555.
    • My response: The noise in my first linearity measurements are largely from the the DUT, not the analyzer. In other words, the positive error -- the upward turn -- in those measurements (Fig.1 and Fig.2) are defined more by the DUT's noise through the audioband (around 5.6 µV) than the analyzer's (around 1 µV). As for his assertion of his level of signal processing experience being necessary to making a proper linearity measurement, see my point above.

He further stated:

• ...What they were asked to help setup was detection of level while eliminating all distortion and noise. This is NOT what we want. If the device creates X amount of noise and distortion on top of Y signal, we want to measure both. What they needed to ask instead was how to replicate this measurement I made on AP2522 but on APx555.

  • My interpretation: He is reiterating his points above about the SYS5255 vis-a-vis the APx555, and asserts that without the knowledge of the former, the latter is all but useless for linearity measurements.
    • My response: This suggestion is absurd, plain and simple. Isn't what we're doing in Fig.4 what he's doing in Fig.3? This will make even more sense when I post both his and our Benchmark DAC3 linearity measurements later.

• The test I ran takes into account distortion and noise and hence is able to differentiate between DACs easily. Theirs does not. I know because I replicated their method and it would no longer do anything useful.

  • My response: Our first linearity measurements (Fig.1 and Fig.2) take into full account the noise and distortion contained within the audioband (20 Hz to 20 kHz). Because of this, you can see the positive error at low levels (going left on the X-axis) as we hit the DUT's noise floor. He stated that including that noise floor was inaccurate, stating in err that those measurements show the analyzer's noise floor. No. They don't. The DUT's noise floor from its balanced outputs within that audioband is around 5.6 µV. The analyzer's is around 1 µV. I can also assure you that the analyzer's distortion is virtually nonexistent in the audioband and far below that of the Yggdrasil 2. Again, the DUT's noise is largely driving the upward turn in Fig.1 and Fig.2, not the analyzer.
  • My response: Based on his loopback measurement in Fig.3, it's obvious he's bandwidth-limiting. In other words, again, he's saying that's how we have to do it. In other words, the results of a loopback test that he runs with the APx555 using his SYS2522-inspired customizations (that he won't detail, but that I'll venture a guess about later) will look virtually identical to the bandwidth-limited linearity measurements I posted back on 2018-06-15 in Fig.4, given the same device under test. Again, I'll illustrate this shortly with the Benchmark DAC3 linearity measurements:
    • In other words, I feel he's trying to have you believe that (a) without having an SYS2522 on-hand, and (b) without his level of signal analysis experience, one simply can not make a meaningful linearity measurement with the Audio Precision APx555, and that is just a silly thing to say.

• ...I can pull rank on Jude and even AP folks on my understanding of such topic. But I am not. So let's move on such tactics.

  • My response: I'll just let that statement stand proudly on its own with no additional commentary from me.

So if you're still with me, here's where we're at:

• I posted linearity measurements (Fig.1 and Fig.2). @amirm didn't like them for reasons that don't stand up and for at least one reason (that these measurements are useless by virtue of an alleged analyzer "error") that is simply not true. Again, the positive error in both measurements is defined more by the noise floor of the DUT not the analyzer. That is, by the time they stray with an error >1 dB (my arbitrary threshold) -- or even more so >0.1 dB (@amirm's arbitrary threshold) -- you're more swamped in the DUT's noise floor than the analyzer's.
• To demonstrate the alleged error of my ways in Fig.1 and Fig.2, he posts Fig.3 -- a bandwidth-limited measurement -- to show how it must be done. But...
• ...I posted bandwidth limited measurements (using Bandpass Level Sweep), as shown in Fig.4. He didn't like that either, stating that I have neither the experience nor the old SYS2522, both of which he alleges are necessary to making proper linearity measurements. In addition to this being a downright silly assertion, I didn't see former use or ownership of older AP System One or System Two analyzers as a prerequisite to properly using the APx555. Yes, compared to @amirm I am a pleb when it comes to experience, with only three years experience with the APx555 (and none with their analyzers before that).
  • That's actually the beauty of the APx analyzer / software combo: It's extraordinarily capable, with incredibly deep measurement capabilities, and with remarkably easy menu and drop-down access to most of it. Whenever I use it, I'm reminded of the movie Fantastic Voyage whenever I hook up a DUT of any kind. Why is this happening? What am I seeing? How does it correlate to this or that? It really is fascinating. The ease of moving through measurements and customizing measurements encourages exploration and experimentation, and one can while away hours playing with it (which I all too often do).
  • When I get stuck, I call or email AP's Tech Support team. It all comes together so that even a commoner like me can properly and accurately generate and understand many meaningful, interesting tests -- like, for example, linearity measurements.
• Again, @amirm didn't like my second set of linearity measurements either (the bandwidth-limited ones in Fig.4), discrediting them by stating that these measurements eliminate noise and distortion to focus on the low-level decoding linearity -- again, this is what he's doing in Fig.3. Now exactly what he's doing in Fig.3 I do not know because he hasn't detailed it, but I do know that he's bandwidth-limiting. Again, you don't get to +/- 0.5 dB in loopback at levels that low without limiting bandwidth.
• So what is he using? What are his secret settings? Only he knows for sure, but given his repeated statements that it's based on the linearity measurements from his SYS2522, another audio engineer familiar with that analyzer contacted me to suggest that perhaps he's using a hardware filter option for it called the Audio Precision FBP-500X Bandpass Filter, High-Q 500 Hz bandpass for CD DAC linearity measurements. I can't say for sure that's what he's using, but it seems a real possibility, since it's for that generation of analyzers and specific for the purpose. If that's what he's using in his SYS2522, here's the filter's graph:
  
  
  
  
  (above) Graph showing the Audio Precision FBP-500X Bandpass Filter (High-Q 500 Hz bandpass for CD DAC linearity measurements)
  
  If that's what he's using, that's pretty aggressive out-of-band rejection -- what's under the blue line defines the passband. Would that filter a lot of noise and distortion from the linearity measurement outside of 500 Hz (in this filter, the stimulus would be a 500 Hz sine)? You bet.
• So you might be thinking here that whatever customizations he's applied (whether based on the FBP-500X Bandpass Filter or something else) -- these secret customizations that are imbued with his far greater experience (and he definitely does have far greater experience than me, and (according to him) enough experience to "pull rank on even AP folks") -- you might be thinking he's going to get a spectacularly different linearity result with the same DUT hooked up than we would using our customized Bandpass Level Sweep. The thing is he did not get much of a different result at all. Let's take a look:

Very recently, @amirm posted his measurements of the Benchmark DAC3 on his forum. While I do want to revisit his Benchmark DAC3 J-test jitter measurements (since I'm getting a different result using the stock output of AP's J-test utility), let's instead focus on linearity for now.

Yesterday, when being questioned about bandwidth limiting, whether or not it should be done, etc., etc. he ended up posting this linearity plot (Fig.5 below) and this accompanying commentary:

FYI I created Jude's/AP's setting for this latest analysis and there is no practical difference. Here is my method and theirs overlayed on top of each other:

You can see there is a problem with his method creating those dips all along the line to the right but overall message below -90 dB remains the same.

Fig.5 (above) @amirm comparing his linearity error measurement using his customizations to our settings in Fig.4.

Wait...what?! Yes, according to Amir, "there is no practical difference" between the results we're getting using the settings we applied for the measurements in Fig.4 -- that the "overall message below -90 dB remains the same."

NOTE: In response to his statement about "those dips all along the line to the right," I'm not getting the errors he's seeing with the same settings used for Fig.4. Here's what I'm getting:

Fig.6 (above) Bandpass Level Sweep linearity measurement of Benchmark DAC3's unbalanced outputs (digital input is unbalanced).

So, yes... After all of this... After claiming our first linearity measurements were useless (they're not -- they just include the noise and THD within the audioband). After claiming our second measurements were useless for eliminating wideband noise and distortion (which they do, as do his APx555 customized linearity measurements). After his "pull rank" comment. After stating that this can't be done without also having his level of experience and having on-hand a 25-year-old SYS2522 analyzer. After all the talk of the extensive customizations he made...

...After all of this, I the pleb (who will admit to having far less experience than @amirm, and with only three years experience with the APx555) am getting the same dang result, not just according to my linearity measurements but also according to his.

Among the immediate takeaways of many: A well-placed phone call or email message to AP Tech Support can seriously save some time. It would have saved @amirm some serious time.

So why all the bluster then? Why claim one needs his level of experience in signal analysis and processing, experience with (and concurrent possession of) a 25-year-old Audio Precision SYS2522 analyzer, and oodles of time customizing the settings to get them just-so before you can have a meaningful linearity measurement or make good use of the APx555? I think the answer's obvious, but I'll let you answer it for yourselves.

By the way, @amirm, here's the bandwidth-limited setting I'm using in Fig.4 and Fig.6:

• Sequence Mode-->Bandpass Level Sweep
• Change the default Selectivity from 1/24 octave to Window width.
• Go into Advanced Settings-->Signal Acquisition and Analysis-->Settling. Here, change Algorithm from Flat to Average, and change the Averaging Time from the default 200.0 ms to 1s.

That's it. I just timed myself with a stopwatch. It took just over six seconds to make the changes.

I'm confident @amirm will come up with a catalog of ways he feels these settings are sub-par -- perhaps he'll "pull rank" after all -- despite the fact that even he's showing we're getting the same result. (Again, the exact same settings we used in Fig.6 were used for the measurements in Fig.4.)

Oh, before I go, let's look at the linearity error measurements out of the box (solid lines) compared to the results after six seconds of changing settings (dotted lines) (Fig.7 below):

Fig.7 (above) Bandpass Level Sweep linearity error measurements of the Benchmark DAC3's unbalanced analog outputs (digital input is unbalanced), using the stock APx settings (solid lines) compared to the results after optimizing (dotted lines).

Would you call those stock, out-of-the-box settings useless? I would definitely not call those stock results useless.

Yes, of course the APx555 can be an increasingly effective tool with more experience (professional experience and experience specifically using it). Compared to other analyzers, though, it's also democratizing, in that it makes a lot of measurements and routines that previously required scripting and coding a lot more accessible and easy to use.

NOTE: As with my previous few posts here, there are a lot of numbers and such in this post. If you catch any errors, please let me know.

 

[adydula]

and I thought this black hole was "plugged"!!

Jude....keep him honest, its very apparent this guy has an EGO as big as well......

Alex

 

[Alcophone]

@jude,

I have a few questions. For the loopback test, the analyzer has to have its own DAC, to generate its own analog signal and then measure that, correct? So basically, in the loopback test, the analyzer itself is the DUT (device under test), or more precisely, the analyzer's DAC is.
And the noise floor of the APx555 is much lower than -100 dB, right? So in his unoptimized graph, the increasing positive error cannot be explained by the analyzer's noise floor, correct?
If so, it would seem that he's measuring the analyzer's DAC itself as being not super linear. Conversely, if it were indeed linear down to -140 dB, as his optimized graph seems to suggest, maybe Audio Precision should be making audio DACs?
You assume that he does apply bandwidth limiting to achieve the optimized results. To me it looks like he's applying the inverse of a smooth version of the deviation of the analyzer's DAC. Does the APx555 have such a feature? Where you can define a mathematical formula to apply to the measurement result? The deviations from perfect linearity in his graph seem to correspond inversely to deviations from an idealized curve (regression, I suppose) approximating the sample points below -100 dB in his unoptimized result.

Maybe I'm completely wrong about this, but if so, he would apply the inverse of the analyzer's DAC imperfection to every measurement of other DACs (because he attributed them to the analyzer's measuring components, not its generating components).
[Edit: I don't know likely that is, given that he is clearly aware of that possibility when he says "What this means is that the AP has a positive error in "linearity" (It is more than that but let's go with it) at levels less than 100 dB. As such, you can not, let me repeat, NOT measure any DACs with it as you will be showing the sum total error of both the DAC and ADC measurement errors in the Audio Precision." That would be wonderfully ironic.]

But, again, just spitting out hunches.

 

[atomicbob]

Just another point of reference. dScope loopback linearity measurement with measurement parameter full disclosure on screenshot:

Blue and red lines are Left / Right loopback through analog output / input with XLR cables, measured by the Continuous Time Domain Analyzer.
Yellow and orange are Left / Right loopback through analog output / input with XLR cables, measured by FFT Analyzer.
This is out-of-the-box performance with operator knowing which dScope settings to adjust to achieve desired measurement.

 

[Currawong]

TLDR summary:

Amir: "When I do something, it's right. When Jude does the same thing, it's wrong, and even AP don't know what they are doing."

Oh, and @amirm, the decision to create a new Yggdrasil impressions thread was mine, and mine alone. No "they" involved. I am not staff.

I am actually eating popcorn right now.

 

[jude]

@jude...For the loopback test, the analyzer has to have its own DAC, to generate its own analog signal and then measure that, correct? So basically, in the loopback test, the analyzer itself is the DUT (device under test), or more precisely, the analyzer's DAC is.
And the noise floor of the APx555 is much lower than -100 dB, right? So in his unoptimized graph, the increasing positive error cannot be explained by the analyzer's noise floor, correct?...

Actually, this does raise an interesting discussion because with the APx555, we can't directly work with or adjust the APx555 DAC's digital level (dBFS) for loopback, unless we're doing a full digital loop -- digital out to digital in -- and that's obviously not equivalent to what we're looking for here.

So as you mention, @Alcophone, the best loopback choice to venture a comparison here is to go from the analyzer's analog output to its analog input -- analog loopback. As for the noise floor of the APx555 in the audioband (20 Hz to 20 kHz), I believe it's around 1 µV (-120 dBV). However, it's important to keep in mind that when we plug in a DAC as DUT, the signal into the DUT is in the digital domain -- so as one audio engineer pointed out in a discussion about this recently, a DAC could potentially measure more cleanly than the APx555 analog-looping-back into itself.

So looking at @amirm's loopback measurement in Fig.3 in my previous post, it seems dBV was the unit that was used, and treated as the equivalent to dBFS for the purpose. That is, if we would test a DAC's linearity using a range of -120 dBFS to 0 dBFS, then for comparison the analog linearity testing range was simply set to -120 dBV to 0 dBV. Without filtering to narrow the passband, then, you can see how the lower range of that will run into the analyzer's noise floor.

What I've done -- and I welcome discussion / feedback / criticism about this method -- is to look at what level the DAC in question (in this case the Yggdrasil 2) is outputting at 1 kHz at 0 dBFS (full scale output at 1 kHz). In the case of the Yggdrasil 2, that level is 12.591 dBV from its balanced outputs. What I do then is, in balanced analog loopback mode, set the generator output reference level (0 dBrG) to 12.591 dBV.

To better understand my rationale here, let's step out of APx555 analog loopback for a moment and look at the output level of the Yggdrasil 2. Again, at 0 dBFS, it's outputting 12.591 dBV. Down at -120 dBFS, the Yggdrasil 2 is outputting -107.7 dBV (4.121 µVrms) not -120 dBV (1 µVrms).

Now, going back to analog loopback, with generator output reference level (0 dBrG) set to 12.589 dBV, I set the loopback linearity range from -120 dBrG to 0 dBrG. At the bottom of that then, at -120 dBV, I should get a similar dBV value (on the Y-axis) as when I'm at -120 dBFS with the Yggdrasil 2 hooked up. Here's how they compare with those settings:

Fig.1 Schiit Audio Yggdrasil 2 linearity (RMS), Bandpass Level Sweep -140 dBFS to 0 dBFS (141 steps), balanced digital input, balanced analog output

Fig.2 Audio Precision APx555 balanced analog loopback linearity (RMS), Bandpass Level Sweep -140 dBrG to 0 dBrG (141 steps), 0 dBrG = 12.591 dBV

In the above graphs, I've cursored the levels at -120 dBFS (Yggdrasil 2, Fig.1) and -120 dBV (analog loopback, Fig.2), and also at 0 dBFS and 0 dBV, respectively. With the Yggdrasil 2 (Fig.1), the level at -120 dBFS is -107.730 dBV. In analog loopback (Fig.2), the level at -120 dBrG is -107.464 dBV. Of course, the highest output on those plots for both the Yggdrasil 2 (0 dBFS) and analog loopback (0 dBV) is 12.591 dBV.

Below (Fig.3 and Fig.4) are the linearity error plots for both the Yggdrasil 2 (-140 dBFS to 0 dBFS, Fig.3) and analog loopback (-140 dBrG to 0 dBrG, Fig.4), in case you were interested to see how they compare through this range.

Fig.3 Schiit Audio Yggdrasil 2 linearity error, Bandpass Level Sweep -140 dBFS to 0 dBFS (141 steps), balanced digital input, balanced analog output

Fig.4 Audio Precision APx555 balanced analog loopback linearity error, Bandpass Level Sweep -140 dBrG to 0 dBrG (141 steps), 0 dBrG = 12.591 dBV

Is this a reasonable way to compare under the circumstances? I think so. Again, when we plug in a DAC as DUT, the signal into the DUT is in the digital domain -- we're not using the generator and analog output of the analyzer (the way we do when we do an analog loopback) -- so a DAC being tested could potentially measure more cleanly than the analyzer analog-looping-back into itself. So, testing as above in analog loopback within the entire voltage range the analyzer would be seeing (on its input side) from the DAC we'd be testing should generate a result that (due to the additional analog stage in loopback) should suggest comparative performance headroom for the purpose of this analysis when the DUT is a DAC.

Am I missing something? I don't think so -- but I'm also not qualified to pull rank on anyone, so maybe I have missed something (seriously). So, again, I welcome any comments, suggestions, and criticism of this method.

Long story short, though, I think the utility of analog loopback for trying to make a comparison to a DAC-as-DUT configuration is limited.

...If so, it would seem that he's measuring the analyzer's DAC itself as being not super linear. Conversely, if it were indeed linear down to -140 dB, as his optimized graph seems to suggest, maybe Audio Precision should be making audio DACs?
You assume that he does apply bandwidth limiting to achieve the optimized results. To me it looks like he's applying the inverse of a smooth version of the deviation of the analyzer's DAC. Does the APx555 have such a feature? Where you can define a mathematical formula to apply to the measurement result? The deviations from perfect linearity in his graph seem to correspond inversely to deviations from an idealized curve (regression, I suppose) approximating the sample points below -100 dB in his unoptimized result.

Maybe I'm completely wrong about this, but if so, he would apply the inverse of the analyzer's DAC imperfection to every measurement of other DACs (because he attributed them to the analyzer's measuring components, not its generating components).

But, again, just spitting out hunches.

Because he hasn't divulged what modifications he's making, I'm not sure what @amirm is doing -- so, as you stated, we spit out hunches. Regardless of what he's doing, it seems we're getting similar linearity results with the Benchmark DAC3, our method (using Bandpass Level Sweep) definitely filtering to strictly limit bandpass to focus on the stimulus signal. He has not measured the Yggdrasil 2 on the APx555 yet. I think that if he measures an Yggdrasil 2 on the APx555, he'll see results similar to Fig.4 in my previous post.

EDIT 2018-06-28 1537 EDT: Just made some clarifications, and made additional comments in the paragraphs after Fig.4.

EDIT 2018-06-29 0938 EDT: In line immediately above Fig.3, I corrected the loopback range value from "-140 dBV to 0 dBV" to "-140 dBrG to 0 dBrG." These values in Fig.4 and the description underneath Fig.4 were correct from the start.