[mevans]

I promise not to continue to beat this dead horse as the last thing I wish to do is annoy (or bore) fellow members.  But, in light of this discussion, I continue to be baffled as to why the community appears to be opposed to blind testing.  After all, we're talking about components that reproduce sound.  Why should an analysis of the quality of any component to reproduce sound be complicated by the reviewer's knowledge of the item being tested?
 
I can see why the professional reviewing community might oppose blind testing as perhaps such tests might conclude that much money is being misspent and thus have an adverse effect on advertising revenue of relevant publications.  And, I can see why manufacturers might be opposed as well, as listeners might learn that the latest 'improvements' are not really improvements at all. 
 
But, how about the hobbyists?  Certainly, there are many knowledgeable hobbyists, including those (unlike myself) with electronics backgrounds, who could devise blind tests of not only headphones, but of all related components.  Perhaps headphones might be tougher than other components (amps, cables, converters) to analyze blindly because even blindfolded reviewers might be able to identify a headphone from its fit.  But, I'm sure clever minds, if so inclined, can work around this problem.
 
I believe that if a serious effort were put forward to assess such components blindly, it could have a significant impact upon buying practices.
 
I promise to exercise the willpower not to raise this topic again unless others wish to continue the discussion.

 

[ColtMrFire]

The problem with blind tests is you're applying rigorous scientific method to something that is incredibly subjective, not to mention not everyone's hearing/state of mind is the same, not to mention not every pair of the same headphones is exactly the same (slight manufacturing differences, not even getting into various mods), not to mention not everyone's audio chain is the same.  That's not even getting into different forms of EMI/RFI that is different in everyone's living space, with can have a detrimental impact on components.
 
There are too many variables to conduct reliable blind testing and have it be an effective guide for audio buyers.  Not to say blind testing is completely useless... it can serve as a VERY rough guide for comparing components, but nowhere near definitive in trying to prove snake oil, which is what I feel most blind testers are trying to accomplish.
 
I trust my ears.  I know placebo exists... I also know differences in gear exist.  Neither is necessarily more prevalent than the other.  Schiit is one of the better, more transparent audio manufacturers, and that's why I've been buying their equipment.  I paid $200 more for the Bifrost 4490 than I did the Modi 2, and that extra $200 felt like it should've been more like $500 considering the night/day improvement I got.

 

[Billheiser]

As a "hobbyist", I have no objection to blind testing.  That is, if it is done well, and that's hard to do.  Also, for headphones (unlike speakers, amps, DAC's, etc.) we wear a headphone.  We fit it to our head, we adjust it and move it around on our ears.  And comfort matters a lot.
If someone can make a blind headphone test where comfort, pressure, & size of the headphones don't matter - well, I actually don't think it's doable.

 

[thefitz]

Oh, I'd love to see someone pass off A/Bing the HD600 vs the HiFiMAN HE-500. You'd know which is which the instant the unit hit your head.

 

[UtopiaB]

  Oh, I'd love to see someone pass off A/Bing the HD600 vs the HiFiMAN HE-500. You'd know which is which the instant the unit hit your head.

 
Is the price difference between these two a sticking point? And which one is considered better anyway for overall neutrality and sound quality?

 

[thefitz]

   
Is the price difference between these two a sticking point? And which one is considered better anyway for overall neutrality and sound quality?

I'm referring to the notion of blind testing. Unless you're going to do blind-and-numb-in-the-head-and-neck testing, you're going to know which headphones are which in short order.

 

[MorrisL]

Nobody in his/her right mind would appose blind testing, which is the only way we can judge audio quality without any of the external influences coloring our perception. But I doubt discussing it in a 1000-page HD600 impressions thread is any use. Especially when it's brought up without any reference to HD600. 

 

[ColtMrFire]

I don't have a problem with blind testing when people are doing it for themselves to make a decision.  
 
What I don't care for is the idea of blind testing meant to review products vs other products as an objective measure, and making recommendations or passes based on blind tests.  Since there are too many variables involved from person to person for it to be useful to anyone but the tester.

 

[thefitz]

How can you take a blind test when you know what headphone you're wearing the moment you put it on?

 

[DavidA]

About the only headphones you could blind test are the HD-600/650, HE-400/400S/400i/560, HEK/HEX, HD-800/800S
 
For amps and DACs it would not be very hard to set this up in a blind test, and there have been a few done on the Pono and on DACs (Toms Hardware is the first that comes to mind).  I think there should be more blind testing done, might get rid of some snake oil, but most manufactures will not do this since there is no upside for them.

 

[Billheiser]

  Nobody in his/her right mind would appose blind testing, which is the only way we can judge audio quality without any of the external influences coloring our perception. But I doubt discussing it in a 1000-page HD600 impressions thread is any use. Especially when it's brought up without any reference to HD600. 

If you really meant to say that blind testing "...is the only way we can judge audio quality" validly, then even a blind testing proponent like me would beg to differ - strongly.  I and we judge audio quality all the time, in listening and digging music.  Some things sound better than others.  Someone can go do blind testing of everything versus everything else, but in the meantime I'll be listening to stuff that I judge to be good and that I like.*
 
* Such as the HD600

 

[ColtMrFire]

 
I think there should be more blind testing done, might get rid of some snake oil, but most manufactures will not do this since there is no upside for them.

 
Of course there's no upside for them.  They'd be blind testing components others may not even have.  You could blind test a DAC, but the amp, tubes, cables and/or headphones will have an affect on the sound... that goes for anything in the chain.  What's the point in a company blind testing its DAC when someone else's amp/cables/tubes may alter the sound signature?  It'd be a useless test for real world purposes.  
 
Those kinds of tests are useful for individuals trying to decide on something on their own.

 

[DavidA]

   
Of course there's no upside for them.  They'd be blind testing components others may not even have.  You could blind test a DAC, but the amp, tubes, cables and/or headphones will have an affect on the sound... that goes for anything in the chain.  What's the point in a company blind testing its DAC when someone else's amp/cables/tubes may alter the sound signature?  It'd be a useless test for real world purposes.  
 
Those kinds of tests are useful for individuals trying to decide on something on their own.

For blind testing a DAC, if the only thing that changes is the DAC why would this not be a valid test, yes the amp, tubes, cables music and headphones would influence the sound but it will do so equally to both DACs? 
 
Since this is the HD-600 thread, you could paint the housings of both the HD-650 and HD-600 (or blind fold test subjects, lol) so from exterior impression they would look identical (sound levels would have to be matched to +/- .5db at a minimum since most can detect it), the DAC/amp/cables/music would all remain the same and would affect either headphone equally there by the only thing that would be different is the headphone.
 
These test would be difficult or useless for individuals to conduct since most don't have access to equipment that would be needed to set up a proper testing environment.

 

[ColtMrFire]

  For blind testing a DAC, if the only thing that changes is the DAC why would this not be a valid test, yes the amp, tubes, cables music and headphones would influence the sound but it will do so equally to both DACs? 

 
I'm talking about the consumer end.  A/B'ing two DACs (or amps or whatever) means there is other equipment in the chain, equipment consumers may not have, hence they may get different results.  If a manufacturer blind tested DACs, they are testing their entire chain along with it, and that chain may not be the same for consumers.
 
I imagine that's why manufacturers don't blind test.  Too many variables for it to be of any use to anybody.  Again, the only point in blind testing is of value to the one doing the testing, since their chain is unique to their situation.  The next guy may have a different amp/tubes/cables/cans... so of what use is the blind test to him since he may get different results with his chain?
 
It's why people always disagree on the sound signature of components, since they all have different equipment.  Some call the Bifrost "warm", some call it "neutral", some call it "analytical" etc, etc.... hell, a few pages ago we were arguing about the sound signature of the HD600/650...

 

[DavidA]

   
I'm talking about the consumer end.  A/B'ing two DACs (or amps or whatever) means there is other equipment in the chain, equipment consumers may not have, hence they may get different results.  If a manufacturer blind tested DACs, they are testing their entire chain along with it, and that chain may not be the same for consumers.
 
I imagine that's why manufacturers don't blind test.  Too many variables for it to be of any use to anybody.  Again, the only point in blind testing is of value to the one doing the testing, since their chain is unique to their situation.  The next guy may have a different amp/tubes/cables/cans... so of what use is the blind test to him since he may get different results with his chain?
 
It's why people always disagree on the sound signature of components, since they all have different equipment.  Some call the Bifrost "warm", some call it "neutral", some call it "analytical" etc, etc.... hell, a few pages ago we were arguing about the sound signature of the HD600/650...

Agree with your point about people not having the same equipment, great point, also everyone hears differently and have different levels of experience to draw on.
 
To me blind testing is to weed out junk or truth in advertising claims, like the issue with the Pono (comparing really low rez MP3 to 24/96 FLAC), anyone will hear the difference but in the real world most consumers don't use 8kb/64kb vbr MP3 now, most streaming services are 256 or higher from what I've read.  Another useless thing I found is hi-rez music that is made from a 16/44.1 master, what's the point?
 
As for DACs, there was a interesting test on Tom's Hardware that compared DACs from $2000 to $4 using HD-800 and some high end speakers?, most/all couldn't tell the difference from DAC to DAC, but it was a small sample size but those that did participate were owners of some high end equipment of their own so they had some experience.  This type of comparison is hard (the Bifrost example is one), if not impossible since you are dealing with individuals and as noted above, everyone is different.