[ColtMrFire]

   
I can make up consistent differences all day if I know which piece of equipment is in front of me. So it's not just "can I tell a difference" it's "to what are these differences actually attributable?" If doing the comparison blind removes my ability to differentiate audio gear, then it ain't my hearing that is giving me adjectives. And at the end of the day I am 100% fine with people liking equipment for the broad "feeling" it gives them. But I'm not fine with said people dogging the abilities of others to hear "depth" or "PRAT" or whatever, when they themselves are foiled by the simple trick of "not knowing which piece of gear is which."

 
It's important to remember blind tests are not conclusive.  Some people pass, some don't (and not everyone's hearing is the same).  Which gives them little value in any kind of real world application.  As I said, they should be used for individuals who want to experiment.

 

[RRod]

   
It's important to remember blind tests are not conclusive.  Some people pass, some don't (and not everyone's hearing is the same).  Which gives them little value in any kind of real world application.  As I said, they should be used for individuals who want to experiment.

 
Well that's a new one. So how do you want your pharmaceuticals tested? And what does "not everyone's hearing is the same" have to do with how you set up a proper test of, say, two DACs? It's not like the audiologists busts out different tools when you tell him you're an audiophile...

 

[ColtMrFire]

 
Well that's a new one. So how do you want your pharmaceuticals tested? And what does "not everyone's hearing is the same" have to do with how you set up a proper test of, say, two DACs? It's not like the audiologists busts out different tools when you tell him you're an audiophile...

 
Pharmaceuticals have the potential to be dangerous, so testing is vitally important, no?
 
Testing two DACs is fine, but people need to listen to them.  And people generally don't agree on how they sound anyway (as can be seen in any DAC forum), so what is the point of testing them for a wide consumer base?  Hence "real world application".  If you want the satisfaction of being able to measure the technical differences, then sure, have at it.
 
Can you tell me the purpose behind blind testing as you see it?

 

[RRod]

   
Pharmaceuticals have the potential to be dangerous, so testing is vitally important, no?
 
Testing two DACs is fine, but people need to listen to them.  And people generally don't agree on how they sound anyway (as can be seen in any DAC forum), so what is the point of testing them for a wide consumer base?  Hence "real world application".  If you want the satisfaction of being able to measure the technical differences, then sure, have at it.
 
Can you tell me the purpose behind blind testing as you see it?

 
I find spending money vitally important. And I want to spend money first and foremost on what my ears actually hear. Fuzzy feelings about how the equipment looks or who makes it or anything else come a distant second, but those feelings can bias what I attribute to sound. Blind testing helps me isolate the sonics of the devices so that differences can be attributed to those sonics. $100 saved on a DAC is 10+ albums I can buy or a year extra of streaming music, so yeah it matters to me.
 
Have you ever considered that the reason why people don't agree on the sound of two DACs could be that many DACs don't actually sound different but people desperately want them to?

 

[ColtMrFire]

 
Have you ever considered that the reason why people don't agree on the sound of two DACs could be that many DACs don't actually sound different but people desperately want them to?

 
Or maybe not everyone has the same audio chain, since what's in your chain affects the (perceived) sound of the DAC, or any component for that matter.
 
Since manufacturers don't do blind testing (for good reason), where are you getting these blind tests results?  And how can you be sure they're being set up properly?  Also, what if a conclusion is made during the blind testing and you end up not agreeing with it?  This is the reason I keep questioning the usefulness of blind tests.

 

[RRod]

   
Or maybe not everyone has the same audio chain, since what's in your chain affects the (perceived) sound of the DAC, or any component for that matter.
 
Since manufacturers don't do blind testing (for good reason), where are you getting these blind tests results?  And how can you be sure they're being set up properly?  Also, what if a conclusion is made during the blind testing and you end up not agreeing with it?  This is the reason I keep questioning the usefulness of blind tests.

 
Well, "all our DACs sound the same" isn't great marketing. Neither is "we deliberately muck up the response of our DACs to sound different." So let's let the companies do their thing; they wanna make money by taking mine. And yes, testing "difference in DACs for audio chain A" is different than "difference in DACs for audio chain B", but that's why experimental design exists. Tests can be administrated badly; who would deny this? But that holds for both blind and sighted tests. Pointing to potential issues with a blind test doesn't remove the issues of bias associated with sighted testing. People don't even bother to match volumes most of the time, I'd reckon, so I have never understood this negativism towards those of us who want to exercise even a modicum of control.

 

[ColtMrFire]

 
Well, "all our DACs sound the same" isn't great marketing. Neither is "we deliberately muck up the response of our DACs to sound different." So let's let the companies do their thing; they wanna make money by taking mine. And yes, testing "difference in DACs for audio chain A" is different than "difference in DACs for audio chain B", but that's why experimental design exists. Tests can be administrated badly; who would deny this? But that holds for both blind and sighted tests. Pointing to potential issues with a blind test doesn't remove the issues of bias associated with sighted testing. People don't even bother to match volumes most of the time, I'd reckon, so I have never understood this negativism towards those of us who want to exercise even a modicum of control.

 
So then, the only way to be absolutely sure you're getting what you want is to test it yourself, since a) companies won't blind test, b) the chain matters and everyone's chain is different, c) blind tests done by others can easily be mucked up.
 
This is why I am fine with blind testing for schiits and giggles on ones own, but find it very questionable in any other capacity.  

 

[ColtMrFire]

I also find Schiit to be a very transparent, reliable company, since they are openly active on many forums and sell equipment for much less than it's actually worth.  My Bifrost 4490 was a pretty major upgrade from the Modi 2, and I only paid $200 more for it.  Their DACs definitely don't all sound the same, so far.  I hear very good things about Gungnir and Yggy.
 
I don't have the faintest idea about other companies, since I have only be buying Schiit so far.  Looks like I found a sweet spot!

 

[castleofargh]

if all you care about is your opinion, then you always have it and everything is fine within the confinement of your own mind. but outside of it, things may become more complicated.
 
objectively we trust repeatability, and we trust in isolating variables so that we can test them one by one and not mistake the real cause of events. because we care about knowing the reality of things. instead of looking for real stuff to agree with us, reassure us, and reject what doesn't agree with us for no other reason than refusing to be wrong.
when there is no readily available measurement method(the most repeatable thing available), people sometimes need to rely on subjective elements, for things like pain, vision, hearing, if some symptoms are psychosomatic... scientists and professionals use blind tests to rule out given variables all the time, and they're confident enough touse it even for health related issues. so how some audiophiles decide that blind tests can't be trusted in audio when at the same time doctors rely on them for serious issues(or something as simple as a hearing test), can make one wonder about the seriousness of the arguments.
 
if things are conditional to testing, then the results are limited to those conditions, and it's very fine. a rational guy doesn't expect easy black and white answers for every single questions he asks. on the other hand you have people you may trust, talking about the sound of a device when they have made zero effort in removing biases, and have no trouble talking about the model of device with certainties when they have "tested" 1 sample ^_^. so there is no need to be gullible and accept anything any blind test will say, but maybe exercise at least the same skepticism toward all the even less reliable sources of information including yourself when you don't bother putting up a correct testing method before drawing conclusions.
 
taking your own example, modi vs bifrost, did you bother matching the loudness(how?) before listening and decide how much improvement there was? one is 1.5V max, the other is 2V. without volume matching, whatever conclusion you came up with is not reliable and your statement about the major upgrade means nothing. did you use a switch? how do you assess that something is superior by ear? or were you only talking about taste?
see, you don't mind making such statement about those gears, and at the same time talk about how you don't trust something like blind test. strong two-tier logic here IMO?
to me any source of information has potential, and deciding to simply reject one is depriving myself of potentially learning more.

 

[upstateguy]

  if all you care about is your opinion, then you always have it ........<snip>

 
Where do you come up with these great one liners?

  Great Stuff !
 
On the other hand, I'm going to stick my neck out and say the only opinion that matters to me in this hobby is the one I have.  It doesn't matter how great someone else thinks something is.  The only thing that matters is what I think about it.  This is something we all do.  That's why we develop preferences for various combinations of headphones, DACs and amps.
 
I have accumulated a bunch of DACS over the years.  I have some that work and some that don't work anymore and some I've given away.  I'm going to say that these DACs don't all sound the same.  The main difference I hear is in their respective resolution.  Just last week I thought I'd fire up my portable PICO DAC to check out the built in usb converter.  It has a wolfson chip and upsamples to 24/96.  When I got it, it was a pretty respectable portable DAC but when I plugged it into my GS-1 amp, I found that it didn't have the resolution that my NorthStar MK2 had.  It was dull and disappointing compared to the NorthStar and I shortly unplugged it and put it away.  The resolution just wasn't there.... but, never the less, it had more resolution and sounded better than my much older NOS Constantine Plus.
 
This reminds me of when I first got my GS-1 amp.  At the time I was listening to a Woo3 with two JAN-Philips 6922s and one Cetron 7236 tube  and an M^3 with 637/627 chips, which incidentally, had extremely similar sound signatures, but no where near the resolution of the GS-1 which became my favorite amp.  So now I'm going to say that although the Woo and the M^3 sounded relatively the same, the GS-1 did not sound like the other two.  Headphones back then were 650s and '04 880s.
 
Another DAC I have is a Stello DA100.  I like this DAC a lot and recently upgraded it's usb converter so I could use it's full 24/96 capabilities.  The Stello also has a 192 upsample switch with a bypass.  In the manual it explains that "You should be the judge to decide which mode is better"  so you will have to experiment with the 192 upsampling to see if you want to use it or not.  The point I'm making is that within this DAC, there were two different sounds. 
 
Another DAC I have is a NorthStar MK2.  It has greater resolution than any of my previous DACs and because of this has become my go to DAC along with the GS-1 amp and T-1 headphones.  With this rig I could easily hear the difference between usb converters.  My first converter was a Blue Circle thingee.  I thought it was a great transport until I plugged in one of the HiFace converters.  Again, the increase in resolution was an eye opener or in this case an ear opener.  So now I'm going to say that not all usb converters sound the same.
 
And since the only opinion that matters is the one I have,

I've gone ahead and made a case for DACs, amps and converters not sounding the same even though I know all the mathematical reasons why they should.
 
[[disclaimer:  although the Woo3 and M^3 amps don't sound the same as the GS-1, I don't know if I could reliably tell the difference between the GS-1 and nwav's O2.]]

 

[UtopiaB]

The headfi guy at the shop let me do a hearing test. The FiiO X5 II DAP versus a $10,000 tabletop dedicated DAC/Amp component system type thing. Of course I expected and wanted the tabletop setup to sound better, I was anticipating very good sound from it, but after several back&forths, the FiiO X5 II sounded better imo, which had me perplexed.The FiiO had more air, more prat and better neutral tone. The tabletop unit sounded smooth with reasonable detail and warmish, yet not ultra detailed either, the FiiO sounded 'real' and 'neutral' and 'clear' with good dynamic pace and good detail as far as I could tell. Call me a heretic, but now I have to buy the HD600 that the listening test was done on

.

 

[krismusic]

The headfi guy at the shop let me do a hearing test. The FiiO X5 II DAP versus a $10,000 tabletop dedicated DAC/Amp component system type thing. Of course I expected and wanted the tabletop setup to sound better, I was anticipating very good sound from it, but after several back&forths, the FiiO X5 II sounded better imo, which had me perplexed.The FiiO had more air, more prat and better neutral tone. The tabletop unit sounded smooth with reasonable detail and warmish, yet not ultra detailed either, the FiiO sounded 'real' and 'neutral' and 'clear' with good dynamic pace and good detail as far as I could tell. Call me a heretic, but now I have to buy the HD600 that the listening test was done on .

600's are O/T in this thread. Joke. You should pick up a pair of 600's. Can't go wrong IMHO.

 

[RRod]

   
So then, the only way to be absolutely sure you're getting what you want is to test it yourself, since a) companies won't blind test, b) the chain matters and everyone's chain is different, c) blind tests done by others can easily be mucked up.
 
This is why I am fine with blind testing for schiits and giggles on ones own, but find it very questionable in any other capacity.  

 
Well there is nothing physically stopping people with more energy/time than myself from doing a large-scale controlled test of possible DAC/amp chains. But the companies won't do it for obvious reasons, and a society like the AES probably has better things to worry about than things that are pretty well demonstrable via measurement. That doesn't make blind testing "questionable", just time- and resource-consuming. If people don't got the time and just want to grok their gear just by pressing play, fine by me, but that doesn't suddenly give them hearing super-powers.

 

[gregorio]

[1] Testing two DACs is fine, but people need to listen to them.  
[2] Can you tell me the purpose behind blind testing as you see it?

 
1. Agreed.
2. To ensure #1 !!!!
 
By definition, a sighted test is not "listening to them", it's looking at them. The human brain is a pattern matching machine, it will attempt to match it's different sensory inputs with what it expects (based on what it knows, believes or supposes). If there is some disagreement between it's sensory inputs or between what it expects, it will weigh the probabilities and simply change those inputs to match. None of this is new or disputed by any sane person, it's been known and employed by various artists for centuries. It's why optical illusions and aural illusions exist, it's why the word "perception" exists. If your hearing couldn't be fooled and couldn't be changed/influenced by what you're seeing then virtually all of the TV/Films you've ever watched would be completely nonsensical.
 
The purpose of blind testing is therefore to ensure that we are actually listening and not just subconsciously manufacturing some erroneous listening perception. It achieves this by eliminating a sensory input which the brain would otherwise need to match and therefore eliminates one or more of the many biases the brain relies on to make sense of the world. There are relatively few tests which are perfect and even perfect tests can be applied incorrectly and give incorrect results. And, I don't think you'll find any moderately educated/sane person who believes that even double blind testing is a perfect ("conclusive") test to start with!
 
Your stated position is to list all the potential flaws of double blind testing and conclude that these flaws render double blind testing worthless. Our position does not disagree that those flaws exist (or can exist), it merely states that whatever flaws double blind testing has, it's always far fewer and less significant flaws than sighted testing. For example: blind test flaws: "b) the chain matters and everyone's chain is different, c) blind tests done by others can easily be mucked up." Sighted test flaws: Expectation bias, confirmation bias, placebo effect, nocebo effect, bandwagon effect, contrast effect, subjective validation, dunning-kruger effect, mere exposure effect, blind spot bias, overconfidence effect, pro-innovation bias, third-person effect, anchoring AND IN ADDITION: ALL the exact same flaws as double blind testing (eg. Your "b" and "c")! In other words, you state that double blind testing has "too many variables" and recommend instead a test with virtually all the same variables plus a dozen or more additional variables. It's like saying driving at 100mph is too dangerous, you should therefore drive at 200mph instead. Is it really just a complete lack of honesty and/or logic or is there something I'm missing?
 
BTW, I've ignored your "a" because many audio companies do in fact employ or participate in double blind testing, probably not so many audiophile companies though.
 

  Pharmaceuticals have the potential to be dangerous, so testing is vitally important, no?

 
No! They should just run sighted tests instead because double blind tests have too many variables/are too flawed. Isn't that your argument?
 

  they are openly active on many forums and sell equipment for much less than it's actually worth.

 
You have little idea what it's "actually worth", only assumptions based on the absolute lowest quality information/evidence available; marketing materials, anecdotes and sighted tests. If you apply that same logic to pharmaceuticals; homoeopathy, leeches, mercury, various herbs and radium would still be about the only options. However, you don't appear to apply that same logic to pharmaceuticals, which makes you either illogical/irrational or a hypocrite, doesn't it?
 

  My Bifrost 4490 was a pretty major upgrade from the Modi 2, and I only paid $200 more for it.

 
How do you know it was a "pretty major upgrade", because it cost more? Because the salesman told you it was an upgrade? Because one of your many human cognitive biases led you to believe it to be an upgrade? Or, do you have any more reliable evidence that it's any sort of upgrade, let alone a major one?
 

  Their DACs definitely don't all sound the same ...

 
You've apparently made no attempt to measure or find any other reliable evidence which eliminates doubt that "their DACs" don't sound the same. Do you really not know what the word "definitely" means or are you just deluding yourself and lying to me?
 

   
Looks like I found a sweet spot!

 
Looks like you've found your sweet spot indeed, a sweet spot of cognitive biases, price and the manufacturer's marketing. My personal sweet spot is centred around the actual sound itself though. I've got no objection to you basing your sweet spot on factors which have little or nothing to do with the actual sound, that's your choice, but I do object to you then using terms like "definitely" with regard to the sound.
 
G

 

[Sc00p]

I just think its a mix of placebo and definition behind words used.
My friend changed headphone wires, said it made a big difference. His new dac and interconnects got the same treatment.
I tend to think that if there is a difference then it is only over time that it becomes apparent. Hence why many fail blind tests. However, if the headphone wire made a big difference and we x3 that difference (dac, interconnects and headphobe wire) why cant he pass a blind test? Surely all these differences add up.
Assuming there is indeed a difference, and making this difference larger via cumulative changes. Then it can only be the slightest change. Not the big difference he describes.
But back to my original sentence, maybe what I would call slight, he will call big.
Before I got into av, he was my "expert". I had an entry level £300 kef speaker setup, his advice was to upgrade my cables. £200 on cables that to his ear make a big difference. But for my ears, I have now learnt that I shouldve kept that 200 and put it towards a speaker upgrade. Now with a speaker upgrade, I could clearly hear a difference. Night and day? Probably not, but easily noticeable.