[skamp]

If i was to measure one dac on a certain set up and then measure another one a different set up can you really compare them subjectively?

That's a fair point, and that's why it would be preferable to use a high quality (lowest distortion possible) and perfectly neutral (ruler flat frequency response) DAC for listening to the samples, like I said. That would ensure some kind of common ground. I've been told about "more transparent" gear by their owners so many times, that I can only assume that's not too much of a problem!

 

[AManAnd88Keys]

I don't hear any difference when using my best gear and foobar2000 as player. However, I don't own and have never used any of your stuff, so my opinion probably isn't of much use here.

 

[striker]

Shouldn't it be DAPs instead of DACs?
You recorded the headphone-out, not the line-out, right?

 

[NA Blur]

Perhaps a better indication of their differences would be to invert the phase of one and subtract it from the master.  We could all see the differences visually in that case regardless of our gear and our hearing.

 

[merkil]

Quote:

That's a fair point, and that's why it would be preferable to use a high quality (lowest distortion possible) and perfectly neutral (ruler flat frequency response) DAC for listening to the samples, like I said. That would ensure some kind of common ground. I've been told about "more transparent" gear by their owners so many times, that I can only assume that's not too much of a problem!

I understand what you are saying here. The dac being used to listen to these files should be transparent so it does affect anything but even then I don't think you can say which one is better and pin it to a specific dac with all the other variables. If file A measures the best then it should technically sound the best on every system but that might not be the case due to everyone having different configurations and hearing ability.
 
I could rank the files in order of what I think sounds best to worst according to me on my system but that won't tell me anything about the dacs. When you revel the measurements it will tell me about how good my set up is or how good my hearing is. Im not trying to negate all the work you have done because the measurements have value. They were measured using the same gear for all the dacs so the numbers aren't biased based on the system and hearing.

 

[merkil]

Quote:

Perhaps a better indication of their differences would be to invert the phase of one and subtract it from the master.  We could all see the differences visually in that case regardless of our gear and our hearing.

 
This is interesting.
 
Im going to play devils advocate and go ahead and say what is the real point of even ranking them. Wont everyone assume the least differences from the master (most true to the recording) should be from the most expensive dac out of the 6? Thats what we have all been lead to believe, more expensive better quality, better sound. It might not be the case and thats why it is interesting to see the results but even if I was to rank these files from best to worst I would just assume that the best one is the more expensive dac.

 

[BlindInOneEar]

Quote:

Perhaps a better indication of their differences would be to invert the phase of one and subtract it from the master.  We could all see the differences visually in that case regardless of our gear and our hearing.

The Audio Diffmaker seems tailor made for this.  http://www.libinst.com/Audio%20DiffMaker.htm
 
However, with 7 files to test, that's 42 different iterations to be explored.  Perhaps the OP, who knows which is which, could choose a few samples of interest and then post the .dyf files for us to hear? 

 

[stv014]

Edit: removed spoiler

 

[skamp]

Perhaps a better indication of their differences would be to invert the phase of one and subtract it from the master.  We could all see the differences visually in that case regardless of our gear and our hearing.

That wouldn't tell you anything about what you hear.

If file A measures the best then it should technically sound the best on every system but that might not be the case due to everyone having different configurations and hearing ability.

How do you "measure" arbitrary recordings? This isn't a RMAA test. Also, audiophiles often rave about "audiophile" gear that doesn't measure well, so that's not necessarily true. There's no telling which recording one might prefer, when listening blind.

Wont everyone assume the least differences from the master (most true to the recording) should be from the most expensive dac out of the 6? Thats what we have all been lead to believe, more expensive better quality, better sound. It might not be the case and thats why it is interesting to see the results but even if I was to rank these files from best to worst I would just assume that the best one is the more expensive dac.

Except you don't know which is which here. So we might not get the most intuitive results.

 

[skamp]

stv014: I don't know why you always feel the need to tell people how to cheat in listening tests.

 

[Currawong]

Did you hook headphones in the chain when testing? I recall previous discussion about the sound being different was with people who were plugging headphones into them. That will be totally different to plugging them into a EMU0404 or whatever which will have a very high input impedance, leading them to very likely sounding pretty much identical IME. 

 

[kr0gg]

imo, this test is absolutely useless.
in my experience all under $300 dacs sound almost the same and are not even worth the time used to compare them.

 

[skamp]

Did you hook headphones in the chain when testing? I recall previous discussion about the sound being different was with people who were plugging headphones into them.

That's a fair point: no, I didn't hook up headphones while recording, for two reasons:
1) I wanted to test the DACs more than the attached headphone amps
2) Headphones would have altered the recordings to a certain extent (frequency response variations due to high impedance, stereo crosstalk, and possibly some level of distortion), but different headphones would have had different effects, so nothing close to a common ground could have been met.

That will be totally different to plugging them into a EMU0404 or whatever which will have a very high input impedance, leading them to very likely sounding pretty much identical IME. 

Ha! That didn't take long. Are we claiming that all DACs sound the same now, and that it's the headphone amp that makes all the difference? That is not what I've been told again and again.

 

[skamp]

in my experience all under $300 dacs sound almost the same and are not even worth the time used to compare them.

Again, I've been told the contrary, many times, in no uncertain terms. And I don't think that your conclusion (that they all sound the same) is useless. It's the whole point of this test.

 

[kr0gg]

Quote:

Again, I've been told the contrary, many times, in no uncertain terms. And I don't think that your conclusion (that they all sound the same) is useless. It's the whole point of this test.

 
 
i should also say you can't actually call this test a DAC comparison, since most of your tests consist of a DAC+amp combination.
 
actually, the only device that can be used as a pure DAC in your test is the E-mu.