[skamp]

Objectivists claim that all DACs that aren't severely broken sound the same.
Audiophiles claim that different DACs sound different, more or less noticeably. I've been told, for instance, that a Clip+ and an iPod Classic sound completely different.

What do you think? Here's a chance to find out and / or to substantiate your claims.

I've made a 30s clip of the song "Cannonball" from Supertramp. The source recording sounds very good, it's very well mastered, and it has a high dynamic range (DR15 on foobar's Dynamic Range Meter). I've played it back through several devices, and recorded the output. I've aligned the recordings to the best of my abilities, and volume matched them.

Here's the list of devices that I've used, in alphabetical order: Clip+, EMU 0204 USB, Galaxy Nexus, iPod Classic, my laptop (Realtek ALC663), O2/ODAC. I used the EMU 0204 USB for recording the outputs.

Here's the list of FLAC files (16 bit / 44.1kHz / stereo), named randomly (I used /dev/urandom to generate random rankings):

A.flac, B.flac, C.flac, D.flac, E.flac, F.flac, G.flac. There are 7 files because I've included the original sample in there.
You may download them all at once with this zip file.

EDIT: C2.flac (channel imbalance corrected)

I will reveal which is which a month from now, on February 28, 2013, in a downloadable text file. In order to give you the assurance that I haven't tampered with it just to prove my point, here's the MD5 sum of that file (you may save it somewhere or quote it in this thread to make sure I don't edit it):

cf9dcc3d0303009135af733724479ce8
EDIT: ab6a17183e1b2d320c6e4d4c3d8195c0 (dac2.txt that includes C2)

You may try to identify a particular piece of gear, or which sample is the original, or rank the samples in descending order of preference (the one you like the most first, the one you like the least, last), or ABX any two samples… whatever you're most comfortable with.

P.S.: I don't expect that many people will venture a guess or two, but at least the opportunity is here.

 

[eucariote]

Very cool!  I think that a blind dac comparison is sorely needed.  For the sake of analysis, I'd suggest that you ask people to give you results in a way that pairwise comparisons would be possible, like rankings or a>b comparisons.  That way you could run statistics against a binomial distribution of independent superior/inferior comparisons against the null hypothesis that each dac would be chosen with a 50% probability.  You could then use standard criteria for accepting or rejecting the null hypothesis, the latter in cases where people's choices are not random.

 

[merkil]

I am a little confused about this whole thing. You recorded the samples from several dacs and now want us to rank which sound better? But when we play the files they will be going through our dacs, doesn't that negate everything?

 

[skamp]

Whatever DAC you're going to use for playback, any differences present in the recordings will still show. Ideally, you should use a high quality DAC that's perfectly neutral, if you can.

 

[davidsh]

I don't understand neither

 

[zenpunk]

I believe the test incorporate too many samples and is too complex to be useful. However,  what is pretty obvious from quickly listening the samples one after the other one is how ridiculously small  the differences are, if they even exist. 
Personally, I spent days A/Bing four very different DACs that I was sure sounded different when heard in isolation - I could I have written a review explaining their differences in detail- but when testing  blind in controlled conditions I was incapable to tell them apart.

 

[spaark]

Quote:

I don't understand neither

He recorded the analogue signal that the DACs produced. If the DACs sound different, then the recorded signals will sound different (assuming other factors are negligible).

 

[skamp]

I believe the test incorporate too many samples and is too complex to be useful.

I've come across the claim that some gear that I own has a very specific "sound signature" and is easily recognizable. Identifying one piece of gear among 7 samples is more interesting than the 50/50 chance of a set of only two samples. Identifying TWO pieces of gear is all the more interesting.

when testing  blind in controlled conditions I was incapable to tell them apart.

That's not surprising. Unfortunately, I doubt that any of the people who claim to hear obvious differences will take this test, much less post their findings. Which wouldn't be surprising either.

 

[stv014]

Quote:

That's not surprising. Unfortunately, I doubt that any of the people who claim to hear obvious differences will take this test, much less post their findings. Which wouldn't be surprising either.

 
I have offered to create similar tests, even with samples of their choice, for those who believe the devices I would record are audibly inferior to more expensive ones, but so far no one took the challenge. Also, in an older thread where I posted loopback recordings of a sound card and onboard audio, the few people who voted were unable to tell the sound card apart from the original file; there was one positive result voted for the onboard sound, but no one commented on what the audible difference was.

 

[eucariote]

Quote:

 Identifying one piece of gear among 7 samples is more interesting than the 50/50 chance of a set of only two samples. Identifying TWO pieces of gear is all the more interesting.

 
This is how you would set up a test to answer the question: can people reliably hear a difference between one dac and another?  This first has to be established before venturing to answer less parsimonious/derivative questions of picking one out from a crowd, etc.  I don't know if even this basic discriminative ability has even been demonstrated, notwithstanding tens of thousands of posts at head-fi that that swear it is possible.

 

[skamp]

I have offered to create similar tests, even with samples of their choice, for those who believe the devices I would record are audibly inferior to more expensive ones, but so far no one took the challenge.

I know. But I felt compelled to make my own samples after some people swore that two pieces of my gear in particular sound very different from each other. If they sound so different, surely they must show some very specific qualities / flaws, and surely it would be easy for them to pick them out of the pool.

I doubt there's any chance that those people will show up in this thread, though. But at least I gave them a fair chance to prove that claim in particular.

 

[zenpunk]

I also agree that sadly the test is doomed to fail, but nice try 

. I think the vast majority of people prefer not to challenge their own belief system or question their own perceptions. 

 

[skamp]

Not to mention that many people probably ignore the Sound Science forum altogether! Happy 1,000th post btw.

 

[merkil]

Quote:

He recorded the analogue signal that the DACs produced. If the DACs sound different, then the recorded signals will sound different (assuming other factors are negligible).

 
 
Quote:

Whatever DAC you're going to use for playback, any differences present in the recordings will still show. Ideally, you should use a high quality DAC that's perfectly neutral, if you can.

 
 
I understand now a little more clearly that if you listen to the files from your system that the same "flavour" will be put on the files and that you might hear the differences. What I don't understand is how you would guess. There are still to many variables at play. For example, if file A measured the best then it should sound the best on all our systems, but that might not be the case depending on how each one of us has our system set up. No? File A might sound the best on mine but file C might sound the best on yours. Its one thing to have all the dacs in a room hidden, being fed by the same sources and all have the same amps/headphones and you go from one to another listening cause the only thing changing is the dac. If i was to measure one dac on a certain set up and then measure another one a different set up can you really compare them subjectively? Thats how I look at it, maybe my reasoning is wrong though.
 
I mean, I look forward the measured results because they were all measured using the same equipment so you can compare them subjectively to see if one sounds better than another.

 

[zenpunk]

@ Skamp,
Oh my God! you are right. I didn't realised. Thanks