Are headphone reviews essentially useless? - Page 2  

In the contrary I thought the 50 headphones review was great.

For me, a review makes sense only with a comparision to other gear, you can interpret the terms in objective value, for instance a bright headphone will likely have an emphasis on the higher frequencies, etc.

Quote:

Originally Posted by Puranti 

In the contrary I thought the 50 headphones review was great.

For me, a review makes sense only with a comparision to other gear, you can interpret the terms in objective value, for instance a bright headphone will likely have an emphasis on the higher frequencies, etc.

The problem here even with well intentioned reviews is that we have no way of knowing if the subjective comparisons are close to accurate. This is even a problem with things like solid state CD players which should vary less (hopefully) where one person will say A has a more extended upper range than B while another will say the reverse and when you look at them both are razor flat to 20K. The subjective experience is often a long way from accurate.

When Sean Olive describes the differences between sighted and blinded testing can immediately see how much we are biased by all sorts of factors; price, appearance and reputation among them. It is really hard to remove theses biases even when you are perfectly aware of them...

Of course.

Quote:

Originally Posted by chewy4 
...

I also disregard any review that says something sounds too "analytical". If you're saying a headphone catches every detail but still fails to have any musicality it's the song's problem since you're hearing everything that the song has to offer.

I haven't heard the cans which have reputation of overly analytical, but I often wonder if "clean" headphones which present what's in the music with less distortion could even sound better than lesser cans which smudge the source and distorts it ("veiled" sound). Maybe even the majority of recordings can't satisfy the high end cans.

Quote:

Originally Posted by matti620 

I haven't heard the cans which have reputation of overly analytical, but I often wonder if "clean" headphones which present what's in the music with less distortion could even sound better than lesser cans which smudge the source and distorts it ("veiled" sound). Maybe even the majority of recordings can't satisfy the high end cans.

Don't know how and if it's related, but isn't the majority of recordings aimed at speaker use?

Quote:

Originally Posted by MrHeuristic 

In that case, is there any sort of database of purely objective headphone measurements? Maybe a database of frequency response curves (similar to MTF charts for lenses) so that any headphone can be compared?

http://www.innerfidelity.com/resources

http://www.headphone.com/learning-center/build-a-graph.php

http://sonove.angry.jp/

http://rinchoi.blogspot.de/

http://www.geocities.jp/ryumatsuba/review.html

This is all I know of. I would never buy a headphone without looking at as many measurements as possible. This will save you money, time and frustration.

I'd love to see more links

Quote:

Originally Posted by fubar3 

The subjective report of one person should be taken lightly. However, opinions of the crowd can be given more weight. I tried the audio-technicia m50 because it seemed popular here and I am pleased with that choice.

I had so many disappointments when listening to the crowd.

For example, look at this. The Sennheiser IE 8 is the best rated universal IEM! So it has to be good, right??

Wrong. In my opinion it completely sucks without a ton of EQ. The messmates support this.

Quote:

Originally Posted by dancingpotato 

This site uses what they call "scientific' methods to evaluate headphones ... http://www.headphoneinfo.com/

lol, this site doesn't provide enough data to come to any conlusion.

 It's like describing food or wine, a generalization at best.

I definitely lean toward the objective measurements but subjectivity can enhance those imo. If 10 people give a positive review to headphone xxx and all have similar things to say about how it sounds but 1 other reviewer disagrees, you can probably figure the 10 are closer to the mark. Not universally so, but more than likely. What I find amusing/frustrating is when people say something like, "there seems to be a bump in the midrange around the 5000 mHz area and then you look at a frequency graph and voila, there it is just as described. I think they probably saw the graph first and then made the comment (I don't know anyone who has calibrated ears). 

Subjectivity on it's own is almost pointless. I've heard great things about the Audio Technica M50s but when I listened to them I didn't care for them at all. The HD 650's are considered by many to be as good as you can get in the under $500 range but not to my ears/brain. If you rely on measurements and subjective reviews, I think you'll be able to make better decisions.

Quote:

Originally Posted by roadcykler 

I definitely lean toward the objective measurements but subjectivity can enhance those imo. If 10 people give a positive review to headphone xxx and all have similar things to say about how it sounds but 1 other reviewer disagrees, you can probably figure the 10 are closer to the mark. Not universally so, but more than likely. What I find amusing/frustrating is when people say something like, "there seems to be a bump in the midrange around the 5000 mHz area and then you look at a frequency graph and voila, there it is just as described. I think they probably saw the graph first and then made the comment (I don't know anyone who has calibrated ears). 

 

Subjectivity on it's own is almost pointless. I've heard great things about the Audio Technica M50s but when I listened to them I didn't care for them at all. The HD 650's are considered by many to be as good as you can get in the under $500 range but not to my ears/brain. If you rely on measurements and subjective reviews, I think you'll be able to make better decisions.

Just understand that measuring headphones is really hard work and only going off headphone measurements (without first hand listening experiences) can lead to mis-conceptions of how they really sound.

I think this article might help but objective measurements into a slightly better perspective:

http://www.innerfidelity.com/content/expert-tests-innerfidelitys-headphone-measurement-repeatability-and-reproducibility

I don't see how you folks think that measurements are objective..   They are just as flawed as anything else.  How can three different objective measurements of the same headphone show different results?  Each site we go to for these measurements use different amps different tools, everything is different thus giving different results.  

Sorry, I don't buy it.  try to sell me something els.

Quote:

Originally Posted by preproman 

I don't see how you folks think that measurements are objective..   They are just as flawed as anything else.  How can three different objective measurements of the same headphone show different results?  Each site we go to for these measurements use different amps different tools, everything is different thus giving different results.  

 

Sorry, I don't buy it.  try to sell me something els.

I think when used in conjunction with actual first hand experiences, they can help. 

Merry Christmas!

Quote:

Originally Posted by preproman 

I don't see how you folks think that measurements are objective..   They are just as flawed as anything else.  How can three different objective measurements of the same headphone show different results?  Each site we go to for these measurements use different amps different tools, everything is different thus giving different results.  

 

Sorry, I don't buy it.  try to sell me something els.

They're objective in the sense that they're not biased by peoples' thoughts and perceptions, and are generated by the device and the testing regime.  This does not imply immunity to many key issues such as imperfect testing equipment, (varying) level of background noise, differences in sound between what reaches a test dummy's ears and what reaches your ears based on your differing anatomy, etc.  As for different measurement sites producing different results, that's mostly a result of different frequency response compensation (which you can "uncompensate" for yourself, roughly), having different actual headphones to test as there can be significant sample-to-sample variance between headphone samples of a given model, differences in microphone/headphone/dummy ear coupling, headphone positioning, and so on.  That on top of the things already mentioned.  Unless they're using bad amplifiers, that should have minimal impact, but that's there as well, as you mentioned.

Whether you label that as objective or subjective is not really that important.

The important thing is to understand the limitations of any source of information, filter it for the useful stuff, and weight it appropriately.  Just as we shouldn't forget any of the above issues and more with published measurement data, we should't ignore how (even unintentionally) biased user reviews of headphones are by the price, look, feel, brand, their mood, their preferences, etc., and how much of a moving target perception can be.

Giving a zero weight to measurement data and a zero weight to all subjective reviews are both bad ideas.

And a Merry Christmas... to all that celebrate it.  Maybe everyone else too.

Quote:

Originally Posted by nick_charles 

The problem here even with well intentioned reviews is that we have no way of knowing if the subjective comparisons are close to accurate. This is even a problem with things like solid state CD players which should vary less (hopefully) where one person will say A has a more extended upper range than B while another will say the reverse and when you look at them both are razor flat to 20K. The subjective experience is often a long way from accurate.

 

Such perceptions could be related to the presence of small amounts harmonic or inter modulation distortion going by impressions I've read compared with measurements of the same item. However, I've only seen one set of measurements taken of two amps which used actual music, revealing an amp that had great measurements using regular tools was not so linear when it was actually required to play anything.  If science is to be relevant to this hobby, it has to have meaning to people.  As it is, many people seem to treat measurements or anything science related in the same way famous reviewers are treated, eg: "XYZ said it so it is right/true." becomes "XYZ measured it so it is right/true."

Quote:

Originally Posted by Currawong 

Such perceptions could be related to the presence of small amounts harmonic or inter modulation distortion going by impressions I've read compared with measurements of the same item. However, I've only seen one set of measurements taken of two amps which used actual music, revealing an amp that had great measurements using regular tools was not so linear when it was actually required to play anything.  If science is to be relevant to this hobby, it has to have meaning to people.  As it is, many people seem to treat measurements or anything science related in the same way famous reviewers are treated, eg: "XYZ said it so it is right/true." becomes "XYZ measured it so it is right/true."

[emphasis added]

Do you remember anything else about this example, any more details (e.g. which regular tools, testing conditions, etc.), maybe a link?  I've been looking and asking for such a thing for years.

Seems like the proper load testing covering enough test signals should be good enough in practice to uncover problems that would show up with music.  After all, the music's not going to contain frequency content outside of a certain range, not going to exceed a certain level.  It's a well-behaved input into what's hopefully a well-engineered, mostly linear system.  People make the "...but with real music" argument all the time, so how much substance is there?  Because I haven't seen the data, I don't find the argument convincing outside of transducers because the behavior of stuff that tests well is so close to linear to begin with.  I mean, you could make your bench tests include music samples if you wanted, but that would be outside the scope of "regular tools" I guess.