[prot]

By definition, any DAC that can't pass a perfect waveform is defective.  For the same of argument, I'm going to assume that both can pass a perfect waveform -- neither tech would have lasted in the market if it couldn't.

So where does that leave us?  The sound difference has to exist somewhere....

Jitter? That's pretty easy to measure. But that's very implementation dependent and I haven't seen anyone make that claim.

If the difference exists, it is somewhere .. that surely is true . And I am quite confident that it will be measurable too.

But I think that you are pushing too hard for the perfect, 100% measurable & sure evidence .. that may even be impossible in this case. Most of the time it's better to start with something smaller .. some of the tests I posted could be that. And we'll have something concrete to talk about instead of this 'philosophy about dacs'

 

[cjl]

   
By definition, any DAC that can't pass a perfect waveform is defective.  For the same of argument, I'm going to assume that both can pass a perfect waveform -- neither tech would have lasted in the market if it couldn't.
 
So where does that leave us?  The sound difference has to exist somewhere....
 
Jitter? That's pretty easy to measure. But that's very implementation dependent and I haven't seen anyone make that claim.

No dac passes a perfect waveform - the question is in what level of inaccuracies the final waveform has. Any audible difference must be visible in the output waveform - jitter, distortion, noise, etc will all show up in the final waveform.

 

[watchnerd]

 
No dac passes a perfect waveform - the question is in what level of inaccuracies the final waveform has. Any audible difference must be visible in the output waveform - jitter, distortion, noise, etc will all show up in the final waveform.

 
I should have been more precise "with no errors that cross the threshold of audibility."

 

[charleski]

  If I read too much into this, I would say the trained listeners think P & I are only mediocre products (or are leaving room for better products) and think that M is the worst thing they have ever heard (they gave it less than a 0.5!). Everyone else thought P & I were wonderful (best they heard...so far), and poor M was bad, but not the worst thing they heard. 
 
Figures taken from Sean Olive's Blog and his 3-part series on trained vs. untrained listeners:
http://seanolive.blogspot.com/2008/12/loudspeaker-preferences-of-trained.html
 
I would have preferred Mr. Toole use "essentially the same order" as that is a more accurate and nuanced statement. Unfortunately, the "suits" and marketing hate accuracy and nuance...

Unfortunately Olive's actual paper is no longer available for free, but Olive has repeated his experiment:
Some New Evidence that Teenagers And College Students May Prefer Accurate Sound Reproduction
The Science and Marketing of Sound Quality
 
Although he tries to spare the manufacturers' blushes by using initials in the AES presentation, he's not trying that hard. 'A' is the $500 Infinity Primus 362, 'B' is the $800 Polk Rti 10, 'C' is the $600 Klipsch RF35 and 'D' is the $3,800 Martin Logan Vista. The Martin Logan that costs 6 times more than the others is almost universally reviled, and reviled more by the trained listeners.
(I'm not entirely sure what point he's trying to make by comparing lossless signals to those compressed using CBR 128kbps mp3, since no-one would claim that compression rate to be transparent, but it's a useful control to confirm that the listeners were capable of detecting subtle degredation in audio quality.)
 
Others have performed similar experiments comparing 'naive' and experienced listeners, eg Rumsey et al. 2005 and Schinkel-Bielefeld et al. 2013. Schinkel-Bielefeld found the same correspondence of rank ordering: both naive and experienced listeners largely agreed in terms of the order of quality of the presentations. While it's not surprising that the naive listeners were more easily satisfied and generally rated audio quality higher than the experts, it's interesting that she found the discrepancy between the two groups grew larger for the higher quality conditions.

 

[mmerrill99]

 

By definition, any DAC that can't pass a perfect waveform is defective.  For the same of argument, I'm going to assume that both can pass a perfect waveform -- neither tech would have lasted in the market if it couldn't.

So where does that leave us?  The sound difference has to exist somewhere....

Jitter? That's pretty easy to measure. But that's very implementation dependent and I haven't seen anyone make that claim.

No dac passes a perfect waveform - the question is in what level of inaccuracies the final waveform has. Any audible difference must be visible in the output waveform - jitter, distortion, noise, etc will all show up in the final waveform.

There are alway differences in the final waveform - the question is what type of test signal you use, what type of waveform analysis you do & what you consider below the threshold of audibility when dealing with complex waveforms!

 

[cjl]

There are alway differences in the final waveform - the question is what type of test signal you use, what type of waveform analysis you do & what you consider below the threshold of audibility when dealing with complex waveforms!

There are some things that are pretty clearly always below the threshold of audibility - if all the differences are below -100dB for example, there's definitely not an audible difference. On the other hand, artifacts at -30dB are usually audible, though surprisingly subtle. I'd be concerned with a DAC that had artifacts any louder than -70dB or so, even though I suspect that's completely inaudible, just because it is so easy to reach that level of performance with modern electronics. You can argue about where exactly those differences become audible, but on any bandlimited waveform (no content above 20k), I'd expect the differences between any two competent DACs (I'll use my earlier examples of the Yggdrasil, Benchmark DAC2 HGC, and ODAC, though there are many more competent DACs out there at a wide range of prices) to be below the threshold of audibility.

 

[mmerrill99]

There are alway differences in the final waveform - the question is what type of test signal you use, what type of waveform analysis you do

There are some things that are pretty clearly always below the threshold of audibility - if all the differences are below -100dB for example, there's definitely not an audible difference. On the other hand, artifacts at -30dB are usually audible, though surprisingly subtle. I'd be concerned with a DAC that had artifacts any louder than -70dB or so, even though I suspect that's completely inaudible, just because it is so easy to reach that level of performance with modern electronics. You can argue about where exactly those differences become audible, but on any bandlimited waveform (no content above 20k), I'd expect the differences between any two competent DACs (I'll use my earlier examples of the Yggdrasil, Benchmark DAC2 HGC, and ODAC, though there are many more competent DACs out there at a wide range of prices) to be below the threshold of audibility.

I'm sure you also read Jason Stoddard's post in which he used all the normal measurements on a DAC & all were perfectly fine & below audibility & yet when listened to found that there was an audible problem?

Has anybody ever asked him what are the measureable differences between his SD & MB DACs from the same model?

 

[castleofargh]

Baldr came as a guest star for a few episodes, but he was more about how he does different stuff because they're not super easy or because they're fun(I badly paraphrase, but I don't think I'm too far off).
but anyway the OP question was more along the line of audibility than measurable differences. if we want to use measurements to show inaudibility, we will need great measurements and a lot of them(which I'm not sure any manufacturer will ever accept to provide). or we could record the 2 outputs and see how well they can null out, but that involves an ADC that will be either of those techs. so it's easier to do, but not ideal and I'm not sure if there are many ADCs that are clearly better resolving than good DACs?

 

[charleski]

I'm sure you also read Jason Stoddard's post in which he used all the normal measurements on a DAC & all were perfectly fine & below audibility & yet when listened to found that there was an audible problem?

Has anybody ever asked him what are the measureable differences between his SD & MB DACs from the same model?

Jason Stoddard has over a thousand posts here on head-fi. So you're going to have to post a link to it.
 
But frankly I regard audio manufacturers as the least reliable source of information.

 

[charleski]

  Baldr came as a guest star for a few episodes, but he was more about how he does different stuff because they're not super easy or because they're fun(I badly paraphrase, but I don't think I'm too far off).
but anyway the OP question was more along the line of audibility than measurable differences. if we want to use measurements to show inaudibility, we will need great measurements and a lot of them(which I'm not sure any manufacturer will ever accept to provide). or we could record the 2 outputs and see how well they can null out, but that involves an ADC that will be either of those techs. so it's easier to do, but not ideal and I'm not sure if there are many ADCs that are clearly better resolving than good DACs?

Actually, you could invert and add the signals in the analogue domain without the need for redigitisation. I'm not sure what conclusion (if any) you could draw from the result though.

 

[mmerrill99]

I'm sure you also read Jason Stoddard's post in which he used all the normal measurements on a DAC

Jason Stoddard has over a thousand posts here on head-fi. So you're going to have to post a link to it.

But frankly I regard audio manufacturers as the least reliable source of information.

Well then there's not much point in me finding & posting the link!

 

[mmerrill99]

 
Baldr came as a guest star for a few episodes, but he was more about how he does different stuff because they're not super easy or because they're fun(I badly paraphrase, but I don't think I'm too far off).
but anyway the OP question was more along the line of audibility than measurable differences. if we want to use measurements to show inaudibility, we will need great measurements and a lot of them(which I'm not sure any manufacturer will ever accept to provide). or we could record the 2 outputs and see how well they can null out, but that involves an ADC that will be either of those techs. so it's easier to do, but not ideal and I'm not sure if there are many ADCs that are clearly better resolving than good DACs?

Actually, you could invert and add the signals in the analogue domain without the need for redigitisation. I'm not sure what conclusion (if any) you could draw from the result though.

Here's where the problems start - the analogue signals from two different DACs will have clock drift between them & impossible to achieve anything of any use
Castle is correct - the quality of the ADC needs to be clearly better than the DAC - in fact in most measuring systems the resolution of the measuring device needs to be at least double the device under test

 

[cjl]

I'm sure you also read Jason Stoddard's post in which he used all the normal measurements on a DAC & all were perfectly fine & below audibility & yet when listened to found that there was an audible problem?

Has anybody ever asked him what are the measureable differences between his SD & MB DACs from the same model?

I missed the part where he proved the problem was audible with double blind testing. It was a nice anecdote for the subjectivists though...

 

[mmerrill99]

I'm sure you also read Jason Stoddard's post in which he used all the normal measurements on a DAC

I missed the part where he proved the problem was audible with double blind testing. It was a nice anecdote for the subjectivists though...

So you missed that he used a multitone test & found the problem in this "non-standard" measurement - do you still want the flawed pseudo-scientific DBT "proof" or did you want him to set up a large scale, rigorous & carefully administered blind test that has some semblance of scientific rigour?

For those interested Stoddards post is here

And the relevant section is at the bottom

Measuring the Unexpected

Okay. Is that enough? No. Let’s go deeper, and talk about one of the measurements we do that is off the beaten path. This measurement appears to correlate at least loosely to subjective impressions, and it unearths some surprising problems in gear that otherwise measures very well.

“So why not release it for the world?” you ask. “If this is such a breakthrough, everyone should be using it!”

Well, we’re not sure it’s a breakthrough. Our sample size is very small. And any correlation it has with sonics is loose at best. And it won’t matter for the hard-core objectivists who have decided that there are no sonic differences between competently designed components, no way, no how, nuh-uh.

And it’s not a breakthrough test. It’s a simple extension of the old IMD idea, but this time with three, four, or five sines—a multitone distortion test. The theory is the same as IMD—can we reveal non-harmonically related stuff with a more complex signal? Usually we use four tones, ranging from 50 Hz to 15,000 Hz, but we’ve run more and less. We’ve used different tones. We’re still playing with this, so don’t take it as gospel.

Aside: it is dead-easy to set up a multitone test on the Stanfords, but I’m not sure how easy it is to do on other products.

So what do we see when we do multitone tests? More non-harmonically related stuff in designs that sound not so hot. Sometimes in very surprising places…not even related to the beat frequencies themselves. Like the example of the Perfect DAC.

The Perfect DAC was not one of ours. It was sent to us by a friend who wanted to get some measurements for it. This was a delta-sigma DAC, manufacturer and chipset redacted, with a very fancy power supply and all the buzzword-compliant stuff people like to hear about these days. We said, “Sure, why not.” And ran it through its paces.

And…in terms of standard measurements, this DAC blew everything we’ve ever measured away. I mean, vanishingly low noise floor, virtually undetectable power supply harmonics, insanely low THD, flat frequency response…

…until you looked at the IMD, which gave numbers a bit higher than you’d expect, given the THD results. And the numbers weren’t related to the 1K spike…they appeared down low, below 100Hz.

What? We ran through our multitone test (it’s easy to do digital multitones on a Stanford as well, not sure about other analyzers) and the low-frequency numbers went bonkers. As in, there was a broad range of non-harmonically related distortion components from 10-90 Hz, at a fairly high level (-50dB or so). -50dB is potentially audible. And it was up nearly 90dB from the baseline measurement.

So what happened? I don’t know. With digital, there are more variables, and noise-shaping and decimation are math-intensive, algorithmically based operations. Perhaps there’s a glitch in their algorithm. I don’t know. It’s not our DAC, and it’s not something we were going to spend the time to dive into.

So…while we putter around confidently with all of the accepted measurements, maybe there are still realms out there where “here be there monsters.”

That’s why we still listen. And measure. And come up with new measurements. And listen again.

So, tell me your list of tests that should be used to characterise a DAC & "prove" that it is "audibly transparent"

 

[RRod]

You don't consider IMD to be on the list of things to test for in a DAC?