[OverlordXenu]

I will probably be flamed for this, but, whatever.

Listening tests are useless. There, I said it. And I will simply say why, if you know what you are using, your bias/preconceived notions are going to influence you, either overtly or subtly. That is to say, you will either lie outright (for whatever reason), or your subconscious will make you think something sounded better or worse.

Also, I have to say I'm really tired of all this equivocation that happens, all the time. I'm talking about the flowery language. It doesn't mean anything...It can be interpreted in any way. It also, is completely and utterly useless.

So, at least in my opinion, double-blind tests are the only way to actually have any certainty in audio. Headphones, cables, speakers, sources, etc. -- anything. I'm not even talking about something like Randi's challenges, I'm just talking about when comparing, you need to set up a double blind test. You cannot know what is being used, and whomever you can see/hear cannot know what is being used. That way, no one can give you any cues (again, either overtly or subtly, that is secret signs or body language). I'm sure you all remembered that horse that could do math. It turned out that the horse learned to stop tapping when his owner stopped tensing his back, and the crowd responded positively. Clever Hans - Wikipedia, the free encyclopedia

If anyone has any reason why I'm wrong, please, tell me. I'm on Head-Fi because (for the most part), it is a highly insightful, helpful, friendly community. If I'm wrong, I want to be told so.

 

[Czilla9000]

Well....I don't agree completely, since the ultimate goal - after all - is to have a setup that pleases you when you are listening to it. It is like saying you can't review a car by driving it.

However, I do agree that our minds tend to adapt to whatever setup you are listening to. In particular, I think our minds tend to equalize our systems for us. Bright phones will stop sounding bright if you listen only to them, for instance. Same with dark phones. This is why I think things like details (resolution) and soundstage are more important than flat frequency response.

EDIT: As to things like power cords and cables, I tend to be skeptical. I think the answer to those lies in a combination of euphonic distortion (the cable, using excessive capacitance and inductance, distorts the signal in a pleasing manner) and placebo effect.

 

[colonelkernel8]

I agree 100%. And you are right, you will get flamed by a bunch of mindless Head-Fi zombies and the same people that think power cords make a difference.

 

[OverlordXenu]

Quote:

Originally Posted by Czilla9000 /img/forum/go_quote.gif Well....I don't agree completely, since the ultimate goal - after all - is to have a setup that pleases you when you are listening to it. It is like saying you can't review a car by driving it. However, I do agree that our minds tend to adapt to whatever setup you are listening to. In particular, I think our minds tend to equalize our systems for us. Bright phones will stop sounding bright if you listen only to them, for instance. Same with dark phones. This is why I think things like details (resolution) and soundstage are more important than flat frequency response.

Well, isn't it more important to find a setup that actually pleases you, rather than a setup that pleases you because of what you think of it?

Then again, I guess there are people who would be more happy with a setup that they think is the best (again, either overtly or subtly), rather than one that they knew was the best. (Replace best with most enjoyable, or whatever.)

But that would have no place on this forum, because that would be personal to them. It wouldn't apply to anyone but them. Ugh, I can't think of the correct wording for what I'm trying to say, but I hope you can get the gist out of what I am saying...

 

[Wodgy]

What do you propose as the alternative?

I'm sympathetic to your position in general, but without listening tests, you'll never develop the ability to correlate measurements with sonic characteristics. Taking absolute THD as an exclusive metric is misleading, as is using on-axis speaker measurements exclusively. Ultimately it's your ears that matter, and the way to develop an understanding of what measurements are important and how to read them is to listen.

 

[meat01]

Quote:

And you are right, you will get flamed by a bunch of mindless Head-Fi zombies and the same people that think power cords make a difference.

I don't think the intent of the thread to bring up more debate on cables and power cords.

Head-Fi exists as a place where people share their personal subjective opinions about the products they hear. Of course when there are biases and opinions get tossed around like facts, there are problems, but such is life.

Listening tests here are more of which components sound best to the reviewer, rather than which component is the best ever. Listening is subjective, so you really can't have objective evidence like Consumer Reports. How would you decide which amp is better with a DBT, if 10 people like 1 and 10 people like the other one?

edit: If you want to prove that there is no audible difference between 2 components, then yeah DBTs can be useful.

 

[OverlordXenu]

Quote:

Originally Posted by Wodgy /img/forum/go_quote.gif What do you propose as the alternative? I'm sympathetic to your position in general, but without listening tests, you'll never develop the ability to correlate measurements with sonic characteristics. Taking absolute THD as an exclusive metric is misleading, as is using on-axis speaker measurements exclusively. Ultimately it's your ears that matter, and the way to develop an understanding of what measurements are important and how to read them is to listen.

No offense, but did you read my post?

ABX/Double-blind tests. They remove preconceived notions.

 

[meat01]

After rereading your post, I think I misunderstood you with my last post. You feel that each person who posts a review should double blind test their components rather than just do an A/B test, knowing which one is which. You were not talking about having multiple people DBT different components to detect a winner.

I agree that this would take away a lot of bias toward certain manufacturers and people may actually find bargains.

 

[grawk]

Listening tests are useless if you give a **** what anyone else thinks. Listening tests are the only solution if you care what your ears tell you.

 

[Wodgy]

Quote:

Originally Posted by OverlordXenu /img/forum/go_quote.gif No offense, but did you read my post? ABX/Double-blind tests. They remove preconceived notions.

ABX/Double-blind tests don't provide you with any generalizable insight into reading measurements either. All they do is enable you to compare two items you have on hand. ABX aren't bad though in helping you to clarify when there are no differences.

 

[Sovkiller]

The best way is using paralell comparisons, the brain and the senses are very good on paralell comparisons, but very poor on serial ones, unfortunatelly the hearing is one of those that the tests always have to be conducted in a serial way, as you can not listen two things at the same time...we are always in disadvantage for that reason...what you listen now five minutes after is gone, listening span memory is really really small...

I have a friend that have tried to condct several BDT in the past...Result an absurd...The first thing you have to do while you have two setups is test the measuring instrument, in this case the setups and the ears, so play the same setups a few times to see if people recognize that they are listening the same setup, do it with both, the result should be always the same...later on go to the next, alternating both. But these tests never passed the first step, as the listeners began to hear differences even while listening always the same setup all the time...

Again, I would like to see all those golden ears we have here, in a double blind test, conducted the way they want, using the gear they like, and are familiar with, just to see the results...

 

[OverlordXenu]

Quote:

Originally Posted by grawk /img/forum/go_quote.gif Listening tests are useless if you give a **** what anyone else thinks. Listening tests are the only solution if you care what your ears tell you.

But we can't trust our ears...Our hearing is one of the, if not the most fallible senses. No matter what, our hearing is biased. We will always have preconceived notions.

I do care what my ears tell me...Which is why I plan on ABX/(double-)blind testing any possible upgrades in my system. That way, I won't subconciously make any assumptions. (Eg. "x amp must sound better than an M^3, it costs much more and looks much better!" is what my mind is thinking, so that may be what I hear. To remove that from influencing my opinion on an amp or whatever, I cannot know what I am listening to, otherwise I will be biased for or against it. It's that simple.)
Quote:

Originally Posted by meat01 /img/forum/go_quote.gif After rereading your post, I think I misunderstood you with my last post. You feel that each person who posts a review should double blind test their components rather than just do an A/B test, knowing which one is which. You were not talking about having multiple people DBT different components to detect a winner. I agree that this would take away a lot of bias toward certain manufacturers and people may actually find bargains.

Exactly. A review is useless if the reviewer knows what he is listening to when he's comparing something.

And because our ears lie to us, we have to compare when we are creating our thoughts on equipment, and why we have to do it blind. Otherwise our opinions are null and void, because our bias has influenced our hearing.

I'm not even talking about better. Look at all the people that love tube amps, because they sound different, not better, but different. I'm not saying that we just get the best components, but the ones we like the most...And they should be the ones, that we actually like, not ones that we think we like.

I think from now on, my "overt/subtle" statement should be assumed.

Quote:

Originally Posted by Sovkiller /img/forum/go_quote.gif The best way is using paralell comparisons, the brain and the senses are very good on paralell comparisons, but very poor on serial ones, unfortunatelly the hearing is one of those that the tests always have to be conducted in a serial way, as you can not listen two things at the same time...we are always in disadvantage for that reason...what you listen now five minutes after is gone, listening span memory is really really small... I have a friend that have tried to condct several BDT in the past...Result an absurd...The first thing you have to do while you have two setups is test the measuring instrument, in this case the setups and the ears, so play the same setups a few times to see if people recognize that they are listening the same setup, do it with both, the result should be always the same...later on go to the next, alternating both. But these tests never passed the first step, as the listeners began to hear differences even while listening always the same setup all the time... Again, I would like to see all those golden ears we have here, in a double blind test, conducted the way they want, using the gear they like, and are familiar with, just to see the results...

As to the first part: I couldn't agree more. Not to mention I've read (in one of those evil DBT's) that human short-term hearing memory only lasts for 5 seconds.

As to the last: I agree, I want to see the self-proclaimed golden ears do a double-blind test, because honestly, I don't think they really exist. I just think that they're trying to be condescending elitist a-holes, like some people I will not mention...

 

[grawk]

It may be relevant for science that my ears aren't trustworthy. I'm not a scientist, I'm not doing experiments, I'm not selling drugs. I'm listening to music, and as such, I trust the sensors I was given by God. You can do whatever you want. I'll trust my ears.

 

[fordgtlover]

Quote:

Originally Posted by OverlordXenu /img/forum/go_quote.gif Listening tests are useless. There, I said it. And I will simply say why, if you know what you are using, your bias/preconceived notions are going to influence you, either overtly or subtly. That is to say, you will either lie outright (for whatever reason), or your subconscious will make you think something sounded better or worse. snip

I could not disagree with you more.

If you are suggesting that the difference between my CMoy and my M³ is just something I want to hear then you either haven't heard enough decent equipment or your hearing is so poor that you have no ability to discerne between good and bad equipment.

I am not a believer in the voodoo and the snake oil that surrounds this hobby, but I know that I can tell the difference between the HD580 and the K701, or my Millett Max and my M³. I don't need to do a DBT.

Personally, I have had enough 'wow' moments when listening to new equipment to know that I can hear a difference between many components.

Perhaps if you can't tell the difference in equipment, then this hobby is not for you.

Quote:

So, at least in my opinion, double-blind tests are the only way to actually have any certainty in audio. Headphones, cables, speakers, sources, etc. -- anything.

Who really gives a crap about certainty in audio. I have heard relatively inexpensive systems that I much prefer over far more expensive ones. I listen to the equipment that I like. I don't need someone telling me the results of a DBT to verify whether there's a difference. Subjectivity is fine by me in this hobby.

The only time I would support the 'certainty' view of the world is when people spout on about expensive tweaks and such that might make a marginal difference, but tell newbies that the differences are 'night and day'.

Quote:

Originally Posted by colonelkernel8 /img/forum/go_quote.gif I agree 100%. And you are right, you will get flamed by a bunch of mindless Head-Fi zombies and the same people that think power cords make a difference.

I'm sorry, but this is not a discussion on whether a power cable makes a difference. This goes to the core of whether people can actually hear a difference between key components.

 

[PiccoloNamek]

Xenu is right, our ears do lie. I don't trust my own ears any more than I would trust a known pathological liar, because the whole auditory system is just that. Telling the difference between headphone frequency responses is easy, anything else is spurious at best.