Testing audiophile claims and myths
Oct 20, 2018 at 6:21 PM Post #9,766 of 17,589
I will, however, note that I have had a lot of conversations with a lot of people about "how DACs sound" (since part of my job is providing support for customers who purchase DACs).
In find that roughly somewhat more than half of the people I talk to, if asked, agree that Sabre DACs "seem to have a distinctive sound".
Furthermore, OF THOSE WHO CLAIM TO HEAR A DIFFERENCE, almost all of them describe the difference they think they hear quite similarly, whether they like or dislike it.
(Those who like it say that they find Sabre DACs to be "more detailed"; those who dislike it consistently describe Sabre DACs as sounding "more grainy" or "etched".)
Feel free to consider this to be "a sociaological study" if you like... but the majority of people who believe they hear a difference seem to believe they hear a SIMILAR difference.
(And, oddly, it is much the same difference that I think I hear.)

Again, you've described a Petri dish for cognitive biases.

there is very little "pure science" left in the world - and virtually none when it comes to audio equipment.

There's very little science in high-end audio; it's primarily snake-oil, at least outside of speakers. The use of science is, in general, increasing in the world.

Speaking of science, if you claim there is a difference in DACs, the onus is on you to support that claim with evidence. If you don't do that, reasonable people are fully justified to dismiss your claims out of hand, no matter your background. That's science.

A company who sells $500 interconnects has little incentive to do a proper study (because it will probably show that their product is eitehr snake oil or only marginally better).

Right. Demonstrating to people that they are wasting their money if they buy your product instead of a much less expensive alternative is generally a bad business strategy. On the other hand, a manufacturer who is selling something that isn't snake-oil has every incentive to demonstrate that their product provides value. If all they provide are excuses, it's a safe bet they are selling snake oil.

Yet the company who sells $5 interconnects ALSO has little incentive to sponsor the test.

Correct. They aren't making any ridiculous claims about their interconnects, other than they will connect one component to another, which is easy to demonstrate conclusively.

What the manufacturer of those $5 interconnect is doing, in most cases, is methodically sampling and testing production units so that reliable estimates about production yield rates can be made with known levels of confidence, using well-established statistical methods.
 
Last edited:
Oct 20, 2018 at 7:24 PM Post #9,767 of 17,589
And which of the twenty or so iPod models did you compare to which of the hundreds or thousands of devices that use the rather popular Sabre DAC chip? And, by the way, which speakers or headphones, and which source content, did you use to compare them?

I have eight iPods from the raised wheel one through the last brushed aluminum classic. I have an Oppo HA-1 that includes the Sabre chip. I don't generally use my speaker system for comparison tests, because that introduces too many variables with distortion and room acoustics. Instead, I use Oppo PM-1s. I've compared a Pioneer blu-ray player, an Oppo BDP-103D, a Sony blu-ray player, a Philips 963SA DAD/SACD player, three different versions of the iPhone, 8 0r 10 different Macs- from the 8500AV to a recent iMac, and a cheapo DVD player from Walmart. Every one of them sounded the same for the purposes of listening to music in the home. That menagerie of gear runs the price range from $40 to 300 times that. It covers portable gear, home gear, A/V gear, phones and players capable of HD audio. I'm looking for something that sounds different. I haven't been able to find it yet. I've just given you a whole laundry list of things you can use to verify my claim that they all are audibly transparent. You probably own some of this stuff or gear just like it. Please! Go check me. I welcome your verification. If there is a difference I missed, I would like to pinpoint it myself, quantify it and find the reason it occurred so I don't make that mistake in the future.

I would be happy to entertain the idea of a DAC or player that sounds different. I would actually be pleased to find one. The problem is, whenever I ask someone in an audiophile group for an example that sounds clearly different, I get a verbal runaround as soon as I try to pin down the claims. Either they point to differences in specs that are clearly not audible, or they base their opinion on subjective impressions and sloppy testing procedures, or they do what you just did... tell me that I have to jump through a million hoops to verify the claim properly. When I call them on the runaround, the conversation usually degenerates to them pulling out the old saw "Either your equipment sucks or you're deaf."

I'm sorry, but I'm not deaf, and I'm not dumb. Just because someone claims in an internet forum that they did a controlled test and clearly heard a difference, that doesn't force me to accept the fact that in "one comparison, under one specific set of circumstances a difference was proven." At the beginning of this, I gave you the benefit of the doubt on that for the sake of argument in the hopes that you would produce a way to get verification. But that led nowhere.

I'm not claiming to try to prove a negative by saying different DACs don't exit. That's just a straw man. I'm just asking for one clear example that we can all verify. But you've worked very hard to make it as impossible as possible for anyone to verify your results... tthe model you used is out of production, you don't own it any more, your system is different than other people's, to capture it you need 10x the audio quality of what you're recording, we would have to sample 50 people and run the probabilities to know for sure, etc, etc... The only supporting arguments you've put forward are that lots of people say Sabre DACs sound different and a bunch of stuff about how filters might theoretically cause differences... but you haven't proven yet that a difference exists!

I don't want you to think I haven't been reading your posts. I was just politely ignoring the prevarication and slips in logic in the hopes that it would lead to a way for me to find a recent DAC or player that sounds different. That's what I do. I don't try to grab on to every little semantic argument, because I don't want to feed the circular arguments that go on here in Sound Science all the time. I just keep my eye on the prize. I skip past the rhetoric and I focus on the nugget of real stuff buried underneath. I patiently wait for the truth to reveal itself... at least until it seems pretty much guaranteed that it's never going to arrive.

A clear audible difference should be clearly audible. If you really did regularly hear clear differences between DACs, it would be easy for you to point to an example that would be relatively simple to verify and audible on any good system. I actually think you honestly believe DACs do sound different. But that belief is based on bias, not actual experience or comparison tests. You might not want to know the truth. That is very common among audiophiles.
 
Oct 20, 2018 at 7:30 PM Post #9,768 of 17,589
you've described a Petri dish for cognitive biases.

Head-Fi is a hothouse for them... a partical generator for them!

Just because a lot of people say something, it doesn't mean it's true. A lot of people claim to have been abducted and violated by aliens. A lot of people have seen bigfoot and the Loch Ness monster. A lot of people believe the lies snake oil salesmen tell them too. They repeat them as gospel and make the job of the snake oil salesman that much easier.
 
Last edited:
Oct 20, 2018 at 8:16 PM Post #9,769 of 17,589
I have eight iPods from the raised wheel one through the last brushed aluminum classic. I have an Oppo HA-1 that includes the Sabre chip. I don't generally use my speaker system for comparison tests, because that introduces too many variables with distortion and room acoustics. Instead, I use Oppo PM-1s. I've compared a Pioneer blu-ray player, an Oppo BDP-103D, a Sony blu-ray player, a Philips 963SA DAD/SACD player, three different versions of the iPhone, 8 0r 10 different Macs- from the 8500AV to a recent iMac, and a cheapo DVD player from Walmart. Every one of them sounded the same for the purposes of listening to music in the home. That menagerie of gear runs the price range from $40 to 300 times that. It covers portable gear, home gear, A/V gear, phones and players capable of HD audio. I'm looking for something that sounds different. I haven't been able to find it yet. I've just given you a whole laundry list of things you can use to verify my claim that they all are audibly transparent. You probably own some of this stuff or gear just like it. Please! Go check me. I welcome your verification. If there is a difference I missed, I would like to pinpoint it myself, quantify it and find the reason it occurred so I don't make that mistake in the future.

So you're listing some devices that have their own DAC and headphone stage, and then presumably other devices that either have digital out or analogue out where you have another device being your headphone amp. How is this any kind of validation for the output of a DAC? Personally, I have an iPhone, several computers, my speaker system with Music Hall SACD player, Oppo BD player, and my main headphone system (of digital transport going to Benchmark DAC and then tube amp). Every one of my setups sound different...but there are also different components in the chain (which means there's no constant metric). An iPhone is fine by itself...and it can drive some of my headphones OK. With others, it just doesn't have enough amperage for their lower sensitivity (to have transparency as you put it). I have an aunt who was an opera singer in Vienna. With a recent family reunion, she picked up a Sennhieser bluetooth headphone for herself, but I also let her try my headphone setup with Benchmark DAC and tube amp. She had no problems telling the difference in sound and said it was the most live sounding she's heard from headphones (just that for her own use, it would be way to heavy and overboard compared to convenient BT headphones).

Recent posts also have mentioned how Sabre DACs are "generally" thought of as being bright. Again, I'd say it's very hard to ascertain because what we are actually hearing are different components down the audio chain. With my main headphone system, for example, my music streamer (SMSL DP3) has the most recent high end Sabre dual channel DACs. I think it's great as a digital transport...but when I've tried plugging my headphones into it, it's far from bright. Its headphone stage is the warmest sounding with muddy treble (even compared to any tube I've had with my headhone amp). So to sum things up, I'm not sure how you can audibly judge a DAC by itself when the stage we tend to be listening to is further down the chain.
 
Oct 20, 2018 at 8:39 PM Post #9,770 of 17,589
So you're listing some devices that have their own DAC and headphone stage, and then presumably other devices that either have digital out or analogue out where you have another device being your headphone amp. How is this any kind of validation for the output of a DAC?

The null hypothesis is that there exist no audible differences among recent (non-defective) DAC offerings. The null hypothesis need not be validated. The alternative hypothesis, that there exist at least two DAC models between which audible differences can be heard (when both are in spec), must be validated.

That someone doesn't hear any differences across such a wide variety of devices suggests that the alternative hypothesis probably isn't worthy of any additional research.
 
Oct 20, 2018 at 8:53 PM Post #9,771 of 17,589
The null hypothesis is that there exist no audible differences among recent (non-defective) DAC offerings. The null hypothesis need not be validated. The alternative hypothesis, that there exist at least two DAC models between which audible differences can be heard (when both are in spec), must be validated.

That someone doesn't hear any differences across such a wide variety of devices suggests that the alternative hypothesis probably isn't worthy of any additional research.

Well to me this seems to be inconclusive and will never be resolved. I don't believe anyone has claimed that Sabre or Burr-Brown DACs or others are outright "defective". There are some different approaches. I know there's still arguments about whether 1bit processing or oversampling to 32bit is "best". But things are also complicated in which what one hears with a device isn't limited to one DAC.
 
Last edited:
Oct 20, 2018 at 9:03 PM Post #9,772 of 17,589
I don't believe anyone has claimed that Sabre or Burr-Brown DACs or others are outright "defective".

No one is saying any of them are defective. I'm just explicitly stating that audible differences caused by manufacturing defects are not of any interest.
 
Oct 20, 2018 at 9:06 PM Post #9,773 of 17,589
No one is saying any of them are defective. I'm just explicitly stating that audible differences caused by manufacturing defects are not of any interest.

And with components that have been discussed, that doesn't seem to be a factor.
 
Oct 20, 2018 at 9:10 PM Post #9,774 of 17,589
And with components that have been discussed, that doesn't seem to be a factor.

Right. Also no one has been able to provide any examples of two DACs that demonstrably sound different, not even the folks who most adamantly claim such differences exist.
 
Oct 20, 2018 at 9:23 PM Post #9,775 of 17,589
Right. Also no one has been able to provide any examples of two DACs that demonstrably sound different, not even the folks who most adamantly claim such differences exist.

Well I doubt anyone who says that their iPhone "sounds" exactly the same as any other system...but what's compounded by the statement about DACs is that they're a primary source in a chain that continues on. The last signal being that of source to your headphone amp. There are different methodologies with DACs. I have a Sony Discman I like that has 1bit processing and a DSP for "surround" sound that sounds nice with headphones. It's very different then my Benchmark DAC (which is focused with oversampling with consistent neutral frequency range). Neither one are "faulty" and if anyone listened to them, they could hear differences (to me, subjectively they're both nice). With my SMSL DP3, if my gauge for its Sabre DACs with its headphone stage...then my impression would be that it's very warm sounding with noticeable treble roll off.
 
Last edited:
Oct 20, 2018 at 9:42 PM Post #9,776 of 17,589
Well I doubt anyone who says that their iPhone "sounds" exactly the same as any other system...but what's compounded by the statement about DACs is that they're a primary source in a chain that continues on.
Right. That should make it easier, not harder, to detect audible differences among devices. Yet, only excuses from those who claim such differences exist.
 
Oct 20, 2018 at 9:49 PM Post #9,777 of 17,589
And which of the twenty or so iPod models did you compare to which of the hundreds or thousands of devices that use the rather popular Sabre DAC chip?

I use my last Mac Classic iPod as my control source. I compare everything to it, and if they sound the same as it, they sound the same as each other. I already answered the questions about the headphones I use. No need for an intermediate amp for iPod or HA-1. Just a headphone splitter.

Are you really interested? Because I've already answered these questions for you. Set aside your bias and ego and just try to figure it out logically.
 
Oct 20, 2018 at 9:50 PM Post #9,778 of 17,589
I'm just curious....
You said "Every one of them sounded the same for the purposes of listening to music in the home"....

Does this mean that you actually did in fact conduct a bunch of proper, level matched, double blind tests between a bunch of them....
Or just that "you actually tested one or two and didn't notice what you considered to be obvious differences between the rest"?
I only ask because, to be quite honest, in many cases I find the differences to be small enough that I can ONLY notice them,
when listening specifically FOR differences, with certain content, and with certain speakers or headphones.

Therefore, I would agree that "for purposes of listening to music in the home" I wouldn't notice the differences either.

When you asked for "an obvious example" I mentioned the Wyred4Sound DAC2 and my DC-1 because I found the differences in sound between them to be quite obvious.
I had planned to sell the Wyred4Sound, and a friend of mine, who designs speakers for a living, had expressed an interest.
I had used the W4S as my main DAC for several months - and found it quite acceptable - before switching to a different one.
I had never compared it directly to another DAC before, but my friend, who already owned a DC-1, was curious if there would be an audible difference.
We tried comparing them, with three different pairs of very neutral speakers, and two different amplifiers...
And were quite surprised at how different they sounded.
(He remarked that the difference was actually greater that the difference between the various speakers.)

I did NOT run a frequency response sweep on the Wyred4Sound at the time.
However, at some point when I was using it, I did run one (I don't recall the details - but an error of even a fraction of a dB in frequency response would have been noticed as odd.)
SInce, as far as I know, W4S has a decent service record, I doubt that it had mysteriously drifted (that sort of error, the same in both channels, would be extremely unusual).
It has been my experience that DACs rarely suffer the sorts of failures that would cause errors in frequency response.
(Also, since they would be due to a failed analog component, the odds of the same flaw appearing in both channels would be very tiny.)
We've tested plenty of DC-1's, some with circuit problems, but I don't recall ever hearing one where everything else worked fine, but the frequency response was off.
(Most DACs sound different, when they do, because of deliberate or unintentional DESIGN differences.)

As for the cause being due to differences in the filters....
That is simply a pet theory of mine.
Knowing that the filters are different, and that, on DACs like the W4S, which offers several filter choices, the choices DO sound different, it seems like a reasonable hypothesis.
Also, offhand, I can't think of anything else that might account for it.
However, to be fair, I am NOT specifically convinced that it's the cause.

There could quite possibly be some OTHER factor that I haven't thought of.
For example, Sabre DACs include an internal mechanism that operates something like an ADSR, and is intended to reduce jitter.
It uses a sort of "intelligent oversampling" that "calculates corrections to compensate for jitter and inserts them in the data stream".
(Sabre has a cool name for it.... but it amounts to figuring out when there's jitter, and adjusting the upsampled data stream, to "compensate" for the errors.)
Therefore, since the data stream is altered, it is not unreasonable to suspect that it may produce some sort of audible effect.

Also, one of the previous Emotiva DAC models included an ADSR for jitter elimination that could be instantly switched on and off by a button.
An ADSR alters the sample rate, but in no way alters the amplitude or frequency response of the data itself.
The ADSR acts as a sample rate converter, but the resampled data is synchronized to a new clock, which eliminates jitter.
However, as with the mechanism in the Sabre DACs, the actual math used to calculate the corrections is very complex and quite proprietary.
(Analog Devices offers a "conceptual description", but the math is proprietary; all they claim is that "any errors it introduces are at least 130 dB down.)
Be that as it may, it is simple enough to switch the ADSR in and out by pressing a button, with no measurable effect on the S/N or the frequency response.
Yet, interestingly, MOST people agree that the ADSR changes the sound (usually seeming to cause slight changes in the sound stage).
(Although the asserted purpose is to reduce or eliminate jitter, the change seems audible on many sources, even those that seem unlikely to have jitter to remove.)

Now, in this case, it should be somewhat trivial to analyze the data stream, and confirm that "all the bits are new and different".
However, the basic measurements remain virtually the same, yet the "sound" changes slightly.
(I would suggest that neither is better, but, when switching quickly back and forth, a "change" is somewhat obvious.)

To me, as a generalization, all this strongly suggests that "there are things going on we haven't accurately quantified yet"...
(Of course, it could turn out that there really is a small difference in frequency response, and it is simply the claims about the limit of audibility on things like that are incorrect.)

I wonder whether, for example, you have actually done a double-blind test between an Oppo 93 and an Oppo 95...
(Oppo claims that the xx5 models sound rather different than the xx3 models because of their Sabre DAcs.)
Or whether you have simply CHOSEN to believe the results of a few outdated, and poorly executed, tests.

I will ask again....
Have you ever actually run or seen the results of a properly designed and operated test to determine whether Sabre DACs sound different than other DACs?
Or, for that matter, to determine whether various DAC filters are audibly different - EVEN IF THEY DELIVER ARBITRARILY LOW DISTORTION AND FLAT FREQUENCY RESPONSE.
Or are you INFERRING that to be the case, based on a little of your own personal anecdotal data, and the results of a bunch of vague and not totally relevent tests?

I have eight iPods from the raised wheel one through the last brushed aluminum classic. I have an Oppo HA-1 that includes the Sabre chip. I don't generally use my speaker system for comparison tests, because that introduces too many variables with distortion and room acoustics. Instead, I use Oppo PM-1s. I've compared a Pioneer blu-ray player, an Oppo BDP-103D, a Sony blu-ray player, a Philips 963SA DAD/SACD player, three different versions of the iPhone, 8 0r 10 different Macs- from the 8500AV to a recent iMac, and a cheapo DVD player from Walmart. Every one of them sounded the same for the purposes of listening to music in the home. That menagerie of gear runs the price range from $40 to 300 times that. It covers portable gear, home gear, A/V gear, phones and players capable of HD audio. I'm looking for something that sounds different. I haven't been able to find it yet. I've just given you a whole laundry list of things you can use to verify my claim that they all are audibly transparent. You probably own some of this stuff or gear just like it. Please! Go check me. I welcome your verification. If there is a difference I missed, I would like to pinpoint it myself, quantify it and find the reason it occurred so I don't make that mistake in the future.

I would be happy to entertain the idea of a DAC or player that sounds different. I would actually be pleased to find one. The problem is, whenever I ask someone in an audiophile group for an example that sounds clearly different, I get a verbal runaround as soon as I try to pin down the claims. Either they point to differences in specs that are clearly not audible, or they base their opinion on subjective impressions and sloppy testing procedures, or they do what you just did... tell me that I have to jump through a million hoops to verify the claim properly. When I call them on the runaround, the conversation usually degenerates to them pulling out the old saw "Either your equipment sucks or you're deaf."

I'm sorry, but I'm not deaf, and I'm not dumb. Just because someone claims in an internet forum that they did a controlled test and clearly heard a difference, that doesn't force me to accept the fact that in "one comparison, under one specific set of circumstances a difference was proven." At the beginning of this, I gave you the benefit of the doubt on that for the sake of argument in the hopes that you would produce a way to get verification. But that led nowhere.

I'm not claiming to try to prove a negative by saying different DACs don't exit. That's just a straw man. I'm just asking for one clear example that we can all verify. But you've worked very hard to make it as impossible as possible for anyone to verify your results... tthe model you used is out of production, you don't own it any more, your system is different than other people's, to capture it you need 10x the audio quality of what you're recording, we would have to sample 50 people and run the probabilities to know for sure, etc, etc... The only supporting arguments you've put forward are that lots of people say Sabre DACs sound different and a bunch of stuff about how filters might theoretically cause differences... but you haven't proven yet that a difference exists!

I don't want you to think I haven't been reading your posts. I was just politely ignoring the prevarication and slips in logic in the hopes that it would lead to a way for me to find a recent DAC or player that sounds different. That's what I do. I don't try to grab on to every little semantic argument, because I don't want to feed the circular arguments that go on here in Sound Science all the time. I just keep my eye on the prize. I skip past the rhetoric and I focus on the nugget of real stuff buried underneath. I patiently wait for the truth to reveal itself... at least until it seems pretty much guaranteed that it's never going to arrive.

A clear audible difference should be clearly audible. If you really did regularly hear clear differences between DACs, it would be easy for you to point to an example that would be relatively simple to verify and audible on any good system. I actually think you honestly believe DACs do sound different. But that belief is based on bias, not actual experience or comparison tests. You might not want to know the truth. That is very common among audiophiles.
 
Oct 20, 2018 at 9:51 PM Post #9,779 of 17,589
Well I doubt anyone who says that their iPhone "sounds" exactly the same as any other system..

Have you ever seen how an iPhone measures? Would you like to? You can compare it to your system.
 
Oct 20, 2018 at 9:53 PM Post #9,780 of 17,589
Right. That should make it easier, not harder, to detect audible differences among devices. Yet, only excuses from those who claim such differences exist.
So are you also claiming every device sounds the same? My assertion has been that audible differences aren't just with DACs (and what we often hear is the different stages)...but if you believe it is hard to detect audible differences with all devices, then that's a different matter.
 

Users who are viewing this thread

Back
Top