Audio gd Sparrow Blind Test - Page 29  

Quote:

Originally Posted by haloxt 

 

I accuse both sides of being crazy :p, it's like two people strangling each other to death while drowning in a swimming pool. This is truly the craziest audio science forum I've come across on the internet.

Awesome 

Nick_charles, I don't like unfettered subjectivism, I think pro-cablers who totally ignore objective data are as obnoxious as anti-cablers who think they're so smart because they take for granted the difficulty of gauging human perception. I agree there's some good tests that honestly tried to find a difference and are not to easy to criticize, but they're not the ones usually cited on head-fi. For serious tests I could only give hypothetical arguments against, to disprove them actually would take proper experiments which I don't have the resources or desire for. But usually it's flawed subjective tests or flawed objective tests discussed on head-fi, and they're just houses of cards.

Prog rock man, here's a few pointers I consider essential in any experiment testing difficult to prove or disprove audio claims. It's just some rough ideas, but far different from some of the retarded experiments often cited on head-fi.

http://www.head-fi.org/forum/thread/435801/propose-your-protocol-here/15#post_5986220

Originally Posted by Crazy*Carl 

I agree, wonderfully summarized and excellent post.

 

 

Quote:

Originally Posted by Prog Rock Man 

Great post, apart from your conclusion highlighted in bold. A failed blind test proves that there is no difference in sound to that person. A few failed blind tests indicate a potential for there to be little or no sound difference. Many failed blind tests prove that it has become more reasonable to conclude that most would fail a blind test. The repetition of the tests with different subjects and equipment covers any supposed deafness on the part of the subject.

 

Thanks, Carl and Prog Rock Man.  I felt I was trying to be reasonable to all sides.  Just to quickly address Prog Rock Man's point, when I said "physically deficient," I wasn't intending that to be an insult, and I wasn't just using a synonym for "deaf."  Certainly some people are literally deaf, but more importantly, EVERYONE'S hearing becomes more and more deficient as they age.  Like death, that's just an unfortunate fact of life.  I'm not saying that Carl's hearing is deficient, I'm just saying that that's one of the variables any scientific test would have to try to take into account.  The average 16 year old and the average 84 year old are just not going to hear the same frequency spectrum.  

What blind tests "prove" is a hard question.  I think the second and third statements below are more accurate than the first.  Failing a blind test (and again, we have to settle exactly what we mean by 'blind test' - ie., the exact parameters) is very strong evidence that the individual subject will continue to fail that blind test.  Does that imply that "there is no difference in sound to that person"?  Like I said in the concluding paragraph of the original post, I think there's a big difference between a blind test where you're rapidly A/Bing, blindly, the same song, same levels, etc, and trying to find differences.  I think differences are more likely to turn up there, then, say, if you listen to A, leave, have some coffee, come back, and listen to B.  (I'm exaggerating for the sake of illustration, but think about it - if you listened to A, and then listen to B a few days later, the sound would have to be DRAMATICALLY different for you to immediately and unequivocally discern a difference.)  Thus it would seem reasonable that the closer we move the A and B together in time, the easier it would be to discern a difference.  

Now whether this difference is big enough to justify the cost/concern/interest between A and B is a TOTALLY different subject.  I think everyone can agree that there's always a "point of diminishing returns" where the increase in SQ is either completely negligible in relation to the expense of the equipment involved.  And the further more you approach that asymptote, the more important placebo effects will become.  (But as a slight wrinkle, the higher SQ the rest of your gear, the more easily you'll be able to discern differences in one link in the chain - so if I'm starting out with $5 earbuds, I'm obviously not going to tell any difference between a $30 filo amp and a $1500 RSA apache, but it would seem silly to conclude that this test would represent strong evidence that there is no difference between these two amps for people with higher quality headphones.)

So I guess my conclusion would be something like, properly done blind testing has a definite role to play in the serious evaluation of audio gear.  I don't think we can or should reduce reviewing to a single method of blind testing, but I do think that the blind testing proponents make reasonable points regarding the great and scientifically demonstrable placebo effects that will intrude, unconsciously, amongst even those trying to be as neutral and vigilant as possible in their reviews.
 

Quote:

Originally Posted by AVU 

Carl, after all that's been written since the original post, I went back and reread your original post again: it's actually quite reasonable!  I don't really disagree with any of it.  You say that onboard sound cards and ipods have gotten quite good - correct.  You say that there's very little difference between these and some of the most inexpensive DAC/amps, and speculate that there's probably an incredibly small difference between components that would cost much, much more - correct again.  You say that the law of diminishing returns applies in hi-fi more than in many other things - correct again.   You declare yourself uninterested in hi-fi anymore because of these things, and will now settle for mid-fi - I feel the same way!  I didn't have enough money to get incredible headphones and an incredible amp, and I felt that I was wasting money buying an incredible amp and an incredible DAC to try and improve mediocre headphones.  You correctly state that the headphones are the single most important thing, which is why I sold my expensive DAC/amps and used the money to buy the JH13s (used, at a VERY good price.)  

 

So basically, I think everything you've said is eminently reasonable.  The only thing that bugs me, and seems to bug a lot of other people as well, is that you additionally seem to be saying that hi-fi itself, in general, is just idiotic, that everyone who thinks they hear minor differences is just fooling themselves into wasting their money, and so forth.  And your basis for this seems to be that everyone doesn't immediately adopt double blind testing in all their reviews.  

 

But I just don't see the basis for these larger claims.  On the contrary, I've found most of the posts on all kinds of equipment by serious reviewers are actually incredibly accurate and reasonable.  They'll tell you that as you spend more and more, you get less and less.  They'll tell you that two amps are virtually indistinguishable, even though one costs much more than the other.  They'll tell you that a cheap amp and dac can sound as good or better than an amp twice or three times as much. They'll tell you how to build a fantastic sounding hi-fi setup for less than $150.  All this is quite reasonable.  I don't see anyone, except for the random crank, who goes around claiming that if you don't spend thousands of dollars on audio equipment your sound will suck.  I just don't see that being the consensus view on any of the forums.

 

And the thing with DBT is not only how it's done, but what it actually proves.  I'm sure we could find plenty of people who couldn't distinguish the JH13 from the apple IEM.  Or the 580 from a decent set of $80 headphones.  But what does that prove?  Simply that that person's hearing is not that well trained (or may be physically deficient.)

 

If I were to do a DBT (which I will, for fun, once I get my JH13s back next week), I'm not going to listen to a source, and then listen to another source after a bit, and so forth.  That's not a good way to distinguish critical details, because my (untrained) ear cannot remember sound that accurately.  I'm sure they'd all just sound 'good.'  Instead, I'd have someone rapidly switch between A (macbookpro headphone out) and B (macbookpro to sparrow) over and over again, on the same piece of music, with the levels matched, and write down all the differences in sound I can find and which I find more preferable and why.  Then I'll put the winner up against the D4. 

Quote:

Originally Posted by AVU 

I think there's a big difference between a blind test where you're rapidly A/Bing, blindly, the same song, same levels, etc, and trying to find differences.  I think differences are more likely to turn up there, then, say, if you listen to A, leave, have some coffee, come back, and listen to B.  (I'm exaggerating for the sake of illustration, but think about it - if you listened to A, and then listen to B a few days later, the sound would have to be DRAMATICALLY different for you to immediately and unequivocally discern a difference.)  Thus it would seem reasonable that the closer we move the A and B together in time, the easier it would be to discern a difference.

Not strictly as it applies to DBT's, but what about where you listen few weeks (or longer) with A, then switch to B for a few weeks, where you don't really notice much of a difference; then come back to A and BAM!, all of a sudden you can really appreciate the difference between the two components?

I have noticed this with not only headphones, but amps/DACs as well. In fact this is where I notice the largest differences between like equipment overall, and this phenomenon seems somewhat consistent with the experiences of others from what I have read.

Quote:

Originally Posted by grokit 

Not strictly as it applies to DBT's, but what about where you listen few weeks (or longer) with A, then switch to B for a few weeks, where you don't really notice much of a difference; then come back to A and BAM!, all of a sudden you can really appreciate the difference between the two components?

 

I have noticed this with not only headphones, but amps/DACs as well. In fact this is where I notice the largest differences between like equipment overall, and this phenomenon seems somewhat consistent with the experiences of others from what I have read.

My experience as well.

I find jazz and classical to be really helpful when doing A/B. How a trumpet or a sax sounds tells me a lot about the midrange. Others use strings, but that isn't something I know well. The human voice... it depends. Music with a lot going on at the same time can tell you a lot about the headphone's overall total balance, especially whether the bass is overriding the mids, but isn't as useful when looking for small differences. Also, when looking for the subtler differences, I think it's more useful to use specific points in time and shorter sections, note them down, it's easier to find and replicate it that way. It's really useful when I think there's a difference in sound signature, but I need to explain it. 

Some just random thoughts, maybe you all agree, maybe not.

Quote:

Originally Posted by grokit 

 

Not strictly as it applies to DBT's, but what about where you listen few weeks (or longer) with A, then switch to B for a few weeks, where you don't really notice much of a difference; then come back to A and BAM!, all of a sudden you can really appreciate the difference between the two components?

 

I have noticed this with not only headphones, but amps/DACs as well. In fact this is where I notice the largest differences between like equipment overall, and this phenomenon seems somewhat consistent with the experiences of others from what I have read.

Quote:

Originally Posted by aimlink 

 

My experience as well.

As have I :)

Agree with everything said.  I'd just add that the one thing that hasn't been mentioned nearly enough is synergy.  While some amps or dacs or headphones are just going to be inferior to others across the board, I'd say the vast majority differ a bit depending on whether they "play well with each other" or not.  

And a lot of that, in turn, is simply whether it's a sound YOU like.  Some people want a lot of detail in the high end, even at the cost of it being a little fatiguing to listen to for long periods.  Other people prefer a mellower (more 'musical') sound, and so forth.  Plus IEMs are going to take differently to amps than headphones, etc.

Synergy is a tough one to figure out. With headphones and amps, it has to do with impedance, efficiency, voltage, and current, and how those factors interact. I am beginning to get a handle on that part. As for how DACs interact, it a bit more mysterious IMHO. It seems to me that with DACs it is more about how they interact with the source more than anything else, so what we really have is two independent systems that complement, but don't really interact, with each other.

In the end I enjoy trying different combinations of all the above, and trusting my ears

Quote:

Originally Posted by AVU 

Agree with everything said.  I'd just add that the one thing that hasn't been mentioned nearly enough is synergy.  While some amps or dacs or headphones are just going to be inferior to others across the board, I'd say the vast majority differ a bit depending on whether they "play well with each other" or not.  

 

 

Do you have some strong **empirical** evidence for this "synergy" and that it is not just one of these subjectivist audiiophile terms that cannot be tested. If for instance you have evidence that a given CD player has a non-flat FR with some amps but not with others (measured of course) I would accept that. Did you notice the name of this subforum ?

wow. the thread's still alive. nice.

Yes, its alive! Myself, Nick_Charles and Crazy Carl are waiting for evidence to prove us to the contrary..........the wait is on 

Quote:

Originally Posted by Prog Rock Man 

Yes, its alive! Myself, Nick_Charles and Crazy Carl are waiting for evidence to prove us to the contrary..........the wait is on 

I don't get it.  What would you do if the evidence came along?  This is downright amusing.

Quote:

Originally Posted by aimlink 

I don't get it.  What would you do if the evidence came along?  This is downright amusing.

I am going to go out on a limb and say if the evidence came along, they would change their stance. Its like believing in Bigfoot. I don't believe in him, but if there were evidence to prove without a doubt he was real, I would change my stance.