CybDev
100+ Head-Fier
- Joined
- Jan 21, 2013
- Posts
- 353
- Likes
- 25
December 2013 Mid-Level DAC Comparison
- Thread starter Gary in MD
- Start date
-
- Tags
- emotiva-xda-1 b-m-c-puredac anedio-d2-dac dangerous-music-source telex-300104-801-oh-4v-odyssey-headphone nad-m51-dac chord-chordette-qutehd-dac matrix-x-sabre-dac resonessence-labs-concero lynx-hilo schiit-gungnir grace-design-m903-reference-headphone-amplifier-dac audeze-lcd3-planar-magnetic-headphone emotiva-stealth-dc-1-balanced-24-192-fully-differential-dac yulong-audio-sabre-da8 metrum-acoustics-octave benchmark-dac2-hgc
Toe Tag
100+ Head-Fier
- Joined
- Feb 13, 2009
- Posts
- 233
- Likes
- 16
Where are you getting this idea. Mark simply said he'd spend an hour or two total testing USB OTG. Since he has all the equipment gathered in one place.
The Android source won't be used for the main comparison, which will spend far more time with each unit. Can't we all just get along?
Drsparis
500+ Head-Fier
- Joined
- Mar 1, 2010
- Posts
- 505
- Likes
- 43
A lot of this information is not yet available since it has not been tested yet. This is all we are asking. Personally I don't want a full review our any review for that matter, just a thumbs up or down depending on if sound comes Out or not.
Noone is forcing anyone. If he does not want to check it out he won't.
Tablix
100+ Head-Fier
- Joined
- Aug 12, 2011
- Posts
- 345
- Likes
- 51
I got no issue with a check box for each saying yes its plug and play compliant, but if you are going to start mixing source equipment for the comparison then you have lost the point of comparing in the first place, before too long you start to get the "is it better from a phone/tablet/PC?" type questions which is not the point of the exercise from my understanding. If any phone/tablet OS cant deal with USB audio then that's not down to the DAC, which is what is being measured by the tests. I am not trying to make enemies or create arguments but the work involved in comparing 12 DAC's is enormous if you are going to do it properly.
We should be appreciating the efforts being undertaken already without stamping our own personal preferences on the tester. I asked for more pro-audio gear in the test and gave a sold reason why and left it at that, if included fine/if not also fine.
My point was more that there is already an extensive test being undertaken and complicating more will probably prove counter-productive.
We should be appreciating the efforts being undertaken already without stamping our own personal preferences on the tester. I asked for more pro-audio gear in the test and gave a sold reason why and left it at that, if included fine/if not also fine.
My point was more that there is already an extensive test being undertaken and complicating more will probably prove counter-productive.
Gary in MD
500+ Head-Fier
- Joined
- Jun 28, 2012
- Posts
- 696
- Likes
- 101
Folks:
I think I've made it fairly clear what I'm willing to do. Plug an OTG cable into my phone, plug each DAC's USB input cable (I will have one for each) into the OTG cable, hit play. If the selected song comes out of the headphones that's a positive check for that DAC. If not, it's a negative check. Then I move to the next DAC, same procedure. There will be no comparisons of sound quality as part of this simple testing, other than "does that sound like the song I intended to play, or like static, or like garble, or like nothing." It shouldn't take an hour in total for all 7 DACs in each group. In fact I'm gonna limit my time to ~1 hour per group, and whatever I get done in that period will have to satisfy the Android source advocates here.
The non-Androidians are mostly right: I have 15 (not 12) total DACs to test (including my current Emo XDA-1) and I will not let this tertiary objective get in the way of the primary focus of the testing, which is to compare and characterize the sound of these DACs using the same source (my computer) playing basically the same list of songs via the same amp (Odyssey Cyclops) and cans (LCD-3s).
I would note in full disclosure mode that I might add or subtract one or two songs from some of the DACs based on something I think I might want to hear, or not hear, due to the specific characteristics of the DAC. For example, I use RHCP's Californication as a torture test, given the "Wall of Sound" nature of that recording. There might be a DAC that would torture ME too much if I played that music through it, so I won't. Sorry, I'm only willing to do so much for the cause.
Or I might try a specific song or album that is not on the basic list with a particular DAC to test out a theory that that particular DAC is well suited for that recording. The point is to try to characterize each DAC as fully as I can, not to run a perfectly uniform test. To do that I'd have to ensure that everything was exactly the same (time of day, my mental and physical condition, background noise conditions, blah, blah) and I just can't guarantee that in testing 15 DACs over a 3 week period. You folks will have to settle for fair and informative, not perfect.
Also, if time permits, I might try comparing how a given DAC sounds with the Cyclops amp vs. the Millenia amp, to see if there is better synergy between that DAC and one or the other of the amps. Yeah, it complicates things, but it might help give some insight into how the DAC interacts with different amps, allowing somebody here to try a particular DAC for a particular amp or sound signature that they might prefer. This type of comparison will be done on a "non-interference basis." Completing the testing of each and every DAC with the Odyssey and LCD-3s will always take precedence. But synergy with the rest of the chain is important, and I don't want to condemn any piece of hardware just because it doesn't mesh well with the Cyclops.
Another secondary objective is headphone outs: Some of the DACS have them, some don't. It isn't fair to knock those that don't, since that isn't the focus of this comparison (otherwise I wouldn't have included any DACs that didn't have a headphone jack). However, I will try to find the time to test the HP-outs for those units that have them, so that folks have my opinions on them (as always, FWIW). I might try the HP outs with HD650s as well, since not everybody has LCD-3s. Again, the point here is to provide as much information about these DACs as I can, with the caveats, as always that this is using my ears, my brains (such as they are), my setup, YMMV, etc., etc., etc.
In the end, I hope to have provided some useful information, or some entertainment, or both, particularly to those who are so kindly donating equipment to the cause. I owe those folks at least that. The rest of you are getting free goods... and while that might be all they're worth, it isn't cool to complain about the quality of free stuff. So no whining or bitching allowed on this thread, except by me (I'm likely to be pretty tired and cranky by the end of this... hell, I'm already pretty tired and cranky just trying to work out the logistics), and perhaps by those sponsoring me, if I break their toy, or treat it unusually cruelly.
Everybody else please just chill out and enjoy.
I think I've made it fairly clear what I'm willing to do. Plug an OTG cable into my phone, plug each DAC's USB input cable (I will have one for each) into the OTG cable, hit play. If the selected song comes out of the headphones that's a positive check for that DAC. If not, it's a negative check. Then I move to the next DAC, same procedure. There will be no comparisons of sound quality as part of this simple testing, other than "does that sound like the song I intended to play, or like static, or like garble, or like nothing." It shouldn't take an hour in total for all 7 DACs in each group. In fact I'm gonna limit my time to ~1 hour per group, and whatever I get done in that period will have to satisfy the Android source advocates here.
The non-Androidians are mostly right: I have 15 (not 12) total DACs to test (including my current Emo XDA-1) and I will not let this tertiary objective get in the way of the primary focus of the testing, which is to compare and characterize the sound of these DACs using the same source (my computer) playing basically the same list of songs via the same amp (Odyssey Cyclops) and cans (LCD-3s).
I would note in full disclosure mode that I might add or subtract one or two songs from some of the DACs based on something I think I might want to hear, or not hear, due to the specific characteristics of the DAC. For example, I use RHCP's Californication as a torture test, given the "Wall of Sound" nature of that recording. There might be a DAC that would torture ME too much if I played that music through it, so I won't. Sorry, I'm only willing to do so much for the cause.
Or I might try a specific song or album that is not on the basic list with a particular DAC to test out a theory that that particular DAC is well suited for that recording. The point is to try to characterize each DAC as fully as I can, not to run a perfectly uniform test. To do that I'd have to ensure that everything was exactly the same (time of day, my mental and physical condition, background noise conditions, blah, blah) and I just can't guarantee that in testing 15 DACs over a 3 week period. You folks will have to settle for fair and informative, not perfect.
Also, if time permits, I might try comparing how a given DAC sounds with the Cyclops amp vs. the Millenia amp, to see if there is better synergy between that DAC and one or the other of the amps. Yeah, it complicates things, but it might help give some insight into how the DAC interacts with different amps, allowing somebody here to try a particular DAC for a particular amp or sound signature that they might prefer. This type of comparison will be done on a "non-interference basis." Completing the testing of each and every DAC with the Odyssey and LCD-3s will always take precedence. But synergy with the rest of the chain is important, and I don't want to condemn any piece of hardware just because it doesn't mesh well with the Cyclops.
Another secondary objective is headphone outs: Some of the DACS have them, some don't. It isn't fair to knock those that don't, since that isn't the focus of this comparison (otherwise I wouldn't have included any DACs that didn't have a headphone jack). However, I will try to find the time to test the HP-outs for those units that have them, so that folks have my opinions on them (as always, FWIW). I might try the HP outs with HD650s as well, since not everybody has LCD-3s. Again, the point here is to provide as much information about these DACs as I can, with the caveats, as always that this is using my ears, my brains (such as they are), my setup, YMMV, etc., etc., etc.
In the end, I hope to have provided some useful information, or some entertainment, or both, particularly to those who are so kindly donating equipment to the cause. I owe those folks at least that. The rest of you are getting free goods... and while that might be all they're worth, it isn't cool to complain about the quality of free stuff. So no whining or bitching allowed on this thread, except by me (I'm likely to be pretty tired and cranky by the end of this... hell, I'm already pretty tired and cranky just trying to work out the logistics), and perhaps by those sponsoring me, if I break their toy, or treat it unusually cruelly.
Everybody else please just chill out and enjoy.
Toe Tag
100+ Head-Fier
- Joined
- Feb 13, 2009
- Posts
- 233
- Likes
- 16
I think we're good here. Very much looking forward to this. One last question though. Gary what is your position or decision regarding, blind (or double-blind) testing.
Happy Camper
Headphoneus Supremus
No DBT BS. This is for entertainment value and all opinion will be that of Gary in MD. He has stated he will go as far as db level match for the science minded. This isn't for the Science Templar crowd but for the entertainment of hobbyists.
Toe Tag
100+ Head-Fier
- Joined
- Feb 13, 2009
- Posts
- 233
- Likes
- 16
What a contentious lot. Actually I'm not sure who if anyone does DBT testing here. Does ljokerl or average_joe.
Gary in MD
500+ Head-Fier
- Joined
- Jun 28, 2012
- Posts
- 696
- Likes
- 101
No need for contention. I think there is a time and place for science, including double-blind testing with fully calibrated systems... but this just isn't it. I have no calibrated hardware, and no mechanisms to do blind switching here -- my dogs are pretty smart but teaching them to switch inputs without me watching is beyond my capabilities to train them.
And in the end, all of the results will still just be my opinion anyway. There is no large group of people here to provide a reasonable sample size as a statistical basis for demonstrating anything. Even if I had another person here to do the switching, if I were to do hundreds or thousands of documented double blind tests myself, I would need at least months, not weeks, to get enough samples to establish a statistically significant basis for clear preferences. To do a true, scientific double blind test I would also have to account for all other potential variables, e.g, in my own condition, in the environment, in all of the possible things that could influence my preferences on any given day at any given time... That just isn't possible. Remember, I have to evaluate 15 DACs.
And even if I could do such tests, after all of that effort I would end up providing just data -- not information. I wouldn't have time to allow any real analysis or detailed write-ups about what I thought I heard and why I picked each winner in each trial. The results would tell people that I preferred a particular DAC over another one, but not why. Without knowing why I picked a winner, nobody would be able to determine whether my preferences reflected their tastes. If I survived this ordeal, I might have an excellent start for an undergraduate thesis paper, and might even end up with a clear winner... but I'm not in college anymore, and the test wouldn't be much fun for anybody, particularly me.
So I'm not doing that. I'm gonna listen, analyze, and write down what I think I hear, including all of the little details and delta details that my whacko brain likes to pick out when I'm doing critical listening. By doing that, I help myself form opinions, but I also give you folks detailed descriptive information about what I think each DAC sounds like. If you've heard that hardware before, you can either agree with me or not. And your responses to my write-ups of each DAC's performance can help calibrate the rest of the group as to the accuracy (or inaccuracy) of my findings. These are just our opinions, after all, but there is information for everybody in both our disagreements and our agreements, if people are paying attention.
For those who haven't heard the hardware, and for the hardware that few others around here have tried, I can only hope that my honestly-arrived-at assessments can provide some insight, enabling them to determine whether they might want to try that DAC themselves... or not. Because I will say it again and again and again -- nobody should buy anything based on my preferences. Most of the DACs in this test are available for auditioning for at least 2 weeks, if not longer. I strongly urge you to try several before making a long-term commitment, no matter what I write.
And finally, let me say that I have absolutely no preconceptions going in about what any of these DACs will sound like -- I've never heard any of them, and I don't care what others have written about them. Yes, I did use others' opinions to put some boundaries on the candidates, but now that a DAC is in the field, I don't care about anything other than what I hear when I test it. I have absolutely no preference for which one wins, and I think my track record testing cans and amps indicates that when I say that, I mean it.
So buckle up folks, the fun starts in about a week, though I don't know that I'll get the first write-ups done immediately. Sometimes I write stuff, and then want to go back later to listen again to make sure I agree with what I wrote the first time. So give me a chance to really do some listening, to write some stuff down, to do more listening, and to really try to nail the analysis and verbiage before posting it. I don't want to post anything without being sure of what I thought I heard, and that the words actually describe the experience accurately.
And in the end, all of the results will still just be my opinion anyway. There is no large group of people here to provide a reasonable sample size as a statistical basis for demonstrating anything. Even if I had another person here to do the switching, if I were to do hundreds or thousands of documented double blind tests myself, I would need at least months, not weeks, to get enough samples to establish a statistically significant basis for clear preferences. To do a true, scientific double blind test I would also have to account for all other potential variables, e.g, in my own condition, in the environment, in all of the possible things that could influence my preferences on any given day at any given time... That just isn't possible. Remember, I have to evaluate 15 DACs.
And even if I could do such tests, after all of that effort I would end up providing just data -- not information. I wouldn't have time to allow any real analysis or detailed write-ups about what I thought I heard and why I picked each winner in each trial. The results would tell people that I preferred a particular DAC over another one, but not why. Without knowing why I picked a winner, nobody would be able to determine whether my preferences reflected their tastes. If I survived this ordeal, I might have an excellent start for an undergraduate thesis paper, and might even end up with a clear winner... but I'm not in college anymore, and the test wouldn't be much fun for anybody, particularly me.
So I'm not doing that. I'm gonna listen, analyze, and write down what I think I hear, including all of the little details and delta details that my whacko brain likes to pick out when I'm doing critical listening. By doing that, I help myself form opinions, but I also give you folks detailed descriptive information about what I think each DAC sounds like. If you've heard that hardware before, you can either agree with me or not. And your responses to my write-ups of each DAC's performance can help calibrate the rest of the group as to the accuracy (or inaccuracy) of my findings. These are just our opinions, after all, but there is information for everybody in both our disagreements and our agreements, if people are paying attention.
For those who haven't heard the hardware, and for the hardware that few others around here have tried, I can only hope that my honestly-arrived-at assessments can provide some insight, enabling them to determine whether they might want to try that DAC themselves... or not. Because I will say it again and again and again -- nobody should buy anything based on my preferences. Most of the DACs in this test are available for auditioning for at least 2 weeks, if not longer. I strongly urge you to try several before making a long-term commitment, no matter what I write.
And finally, let me say that I have absolutely no preconceptions going in about what any of these DACs will sound like -- I've never heard any of them, and I don't care what others have written about them. Yes, I did use others' opinions to put some boundaries on the candidates, but now that a DAC is in the field, I don't care about anything other than what I hear when I test it. I have absolutely no preference for which one wins, and I think my track record testing cans and amps indicates that when I say that, I mean it.
So buckle up folks, the fun starts in about a week, though I don't know that I'll get the first write-ups done immediately. Sometimes I write stuff, and then want to go back later to listen again to make sure I agree with what I wrote the first time. So give me a chance to really do some listening, to write some stuff down, to do more listening, and to really try to nail the analysis and verbiage before posting it. I don't want to post anything without being sure of what I thought I heard, and that the words actually describe the experience accurately.
brunk
Headphoneus Supremus
- Joined
- May 21, 2012
- Posts
- 3,648
- Likes
- 97
...damn, that popcorn bag isn't big enough
Happy Camper
Headphoneus Supremus
Every well intended comparison gets bogged down by science and I admit being sensitized to it.
Chris J
Headphoneus Supremus
There is a whole school of thought that says double blind testing doesn't really work anyway.................
NinjaHamster
500+ Head-Fier
- Joined
- Nov 8, 2005
- Posts
- 554
- Likes
- 86
Do what most professional reviewers do ... use "double deaf" testing, and give the best reviews to the manufacturer who offers you the most money and/or free gear ...
Drsparis
500+ Head-Fier
- Joined
- Mar 1, 2010
- Posts
- 505
- Likes
- 43
Armaegis
Modern Modder Man of Manitoba
HTML... uphill, both ways!
- Joined
- Oct 18, 2009
- Posts
- 12,552
- Likes
- 1,403
Users who are viewing this thread
