[MrHeuristic]

One of my first experiences before joining Head-Fi was the big "Battle of the Flagships" thread with the 50+ headphone reviews. 
 
I was originally swayed by Solomon's comments, though now I'm fairly certain his review is, frankly, worthless. Keeping in mind that human perception of a headphone is very, very heavily influenced by expectation and emotion, and also keeping in mind that Solomon's reviews of the headphones were not at all blind, wouldn't it be true that most of his descriptors ("hard to drive", "detailed", "analytical", "wide sound stage", "warm", etc) are just subjective to his personal expectation? 
 
Furthermore, aren't ALL headphone reviews in this style absolutely worthless? 
 
I'd propose that the only useful thing a headphone review could provide is an analysis of comfort and build quality of a pair of 'phones, and then a frequency response chart comparing the 'phones to ones in a similar price range (and maybe a few well beyond that price range for a comparison). Frequency response charts would be the only way to objectively convey the properties of the headphones.
 
When you draw comparisons to reviews of other products, headphone reviews seem ridiculous. For instance, if one were reviewing a camera lens, usually pictures are taken with that lens and with other, similar lenses to determine its sharpness. Objectively. Then the build quality and price point are usually touched on. The lens' autofocus performance is measured, usually in low light. If the reviewer has the right equipment, distortion of the lens is measured, or measurements from an external source are mentioned.
There is no need for silly subjective descriptors. You'll never hear a lens review that says "well, I decided not to actually take any pictures for comparison, but just looking through the lens, I really FEEL that it's sharp. I really get an analytical feel from this lens."
 
So, why the hell is this the norm for headphones?

 

[wakibaki]

Quote:

So, why the hell is this the norm for headphones?

 
We don't know.
 
Well, that's not quite true, but the reasons are convoluted, obscure and inflammatory to some and reiterating them is not exactly welcomed on a forum that to a large extent owes its existence to the fact that it is.
 
We have here a ghetto in which discussion of objective means of testing is permitted, whereas if there were any rationality to the forum's layout, discussion of objective means of testing would not only be permitted everywhere, but objective substantiation of expressed views would be insisted on, and, if 'subjective' assessments were protected anywhere, they would be ghettoized.
 
The forum would probably be less popular if that were the case, however, so you just have to accept the status quo if you want to mingle with the crowd, or, as my wife would have it, get down wid da youf.
 
w

 

[MrHeuristic]

In that case, is there any sort of database of purely objective headphone measurements? Maybe a database of frequency response curves (similar to MTF charts for lenses) so that any headphone can be compared?

 

[whitedragon551]

Everything is subjective in a sense. Even with camera lens' someones eyes could be better than anothers and thus a certain lens is more clean than another.
 
Even looking at graphs doesnt indicate exactly how a set of cans will sound. It can be a good indicator, but isnt exact in any sense of the imagination.
 

 

[Strangelove424]

You're comparing apples and oranges with the cameras. You can show someone a file that a camera produces, and if they have a calibrated monitor they see exactly what you do, but you'll never be able to do the same for the sound a particular headphone produces without actually wearing it. The frequency response graphs don't really say much on their own, even in relation to other headphones. If you really want frequency graphs, go here: http://www.headphone.com/learning-center/build-a-graph.php You can deduce what any particular headphone sounds like by looking at these graphs as easily as you can know the flavor of a meal by graphing the flavor palette. This is where the subjective and quasi-metaphorical descriptors come from. And they have the same subjective wording in camera reviews. Canon DSLR images are often considered "colder" than Nikon's, noise/iso ratios need to be examined by eye, and lenses are often judged on the vividness of colors. There is a cross roads here between science and art, but science will never remove the subjectivity from it completely. That's part of the fun. 

 

[fubar3]

Quote:

One of my first experiences before joining Head-Fi was the big "Battle of the Flagships" thread with the 50+ headphone reviews. 
 
I was originally swayed by Solomon's comments, though now I'm fairly certain his review is, frankly, worthless.

The subjective report of one person should be taken lightly. However, opinions of the crowd can be given more weight. I tried the audio-technicia m50 because it seemed popular here and I am pleased with that choice.

 

[wuwhere]

"hard to drive" - you may see this on the impedance curve and efficiency.
"detailed" - you may see some of this in the frequency response but the degree can be subjective.
"analytical" - this is subjective.
"wide sound stage" - subjective.
"warm" - you may see some of this in the frequency response but the degree can be subjective.
 
There are other measurements that can tell how the driver behaves.
 
The sound of the headphone can also be influence by the rest of the system, solid state vs. tube amp, the source (DAC, CDP, turntable) and
the recordings used to evaluate them.
 
IMO, objective is not good enough because it does not tell you how it sounds. It does not tell you how it synergises with the rest of the system.
Subjective should be back up with some measurements.
 
So when you buy a headphone the first thing you will ask is what is a good amp to drive them with. Measurements won't tell you that.

 

[AKG240mkII]

Quote:

The sound of the headphone can also be influence by the rest of the system, solid state vs. tube amp, the source (DAC, CDP, turntable) and
the recordings used to evaluate them.
 
IMO, objective is not good enough because it does not tell you how it sounds. It does not tell you how it synergises with the rest of the system.
Subjective should be back up with some measurements.
 
So when you buy a headphone the first thing you will ask is what is a good amp to drive them with. Measurements won't tell you that.

 
A : No, 'The Sound' of the headphone can not be influenced by those things . The cans sound like they sound, no matter how any other gear colours the output .
B : 'Objective' IS good enough. Measurements, if done correctly, do tell you precisely how things 'synergises' . This is not voodoo !
C : If you have correctly done measurements, you don't even need to ask what amp is 'good for driving' the cans!
(There ARE other perfectly valid reasons for choosing 'A' piece of equipment over 'B' than measurements . For instance : If you hate the way it looks it will NEVER sound right to your ears !!)

 

[Snow_Fox]

With any product YMMV.

It's really hard to review *anything* almost since there will always be discrepancies. That awesome Sprint phone doesn't get coverage where I live, so I can't agree on "best phone ever" title someone else gives it. I can still agree it has amazing tech specs and in areas that it does work it is an awesome phone.

Heck, I can even agree that a certain camera lens is much better just not so much on my poorly calibrated monitor.

No matter what review you are reading, it's very rare that someone else can replicate the exact criteria by which you will use your device.

For this reason in particular, I try to take big threads with lots of headphones (or any product reviews) with a large grain of salt.

I'm sure someone who enjoys iTunes is going to give an iphone a much higher review score than the non-iTunes Lg Optimus G. It would only make sense they consistently rate iproducts easier to use if they aren't a fan of plug and play... Where as a plug and play MP3 person would probably score an idevice lower blah blah blah.

While there is a large degree of subjectivity for any device, it's important to remember most things in life are relative either way.
 

 

[MrHeuristic]

Quote:

Originally Posted by Snow_Fox /img/forum/go_quote.gif
 
While there is a large degree of subjectivity for any device, it's important to remember most things in life are relative either way.

 
I totally agree, but there still seems to be a fundamental difference when it comes to headphone reviews.
 
Going back to my lens analogy. Yes, lens reviews contain subjective opinion (and in fact, I think they should!) as do headphone reviews. Where the difference lies is in the structure; lens reviews have a base of objective measurements (comparison pictures almost always, MTF charts sometimes) with subjective opinion on top of that (something like "while it's a very sharp lens, the bokeh is harsh and if you're using it for portraits, you may want to choose a different lens")
With headphone reviews, those subjective comments become the body of the review. Objective measurements (specs or frequency response curves) are rarely included. 
 
While yes, the subjectivity is acceptable in a review, it shouldn't become the main focus. A headphone is designed to, as accurately as possible in a price range, reproduce a source of audio without too much coloration or distortion. Much like a lens is designed, as much as possible at the price point, to reproduce a visual scene accurately and with minimal distortion or aberration.
So, headphones and lenses should, in theory, be reviewed in a very similar way; they are not.

I just wish headphones were reviewed like lenses. Objectivity may be 'less fun' than subjectivity, but it sure is easy to determine the best lens to buy in a given price range. In the world of lenses, there is almost always one or two "best" lenses in each category and price range. Headphones? Not so much. Given a price range, you can get a swath of recommendations, and opinions of certain headphones are often ridiculously polar. For instance, about half of the reviews of the AKG K702 complain that its' bass is weak, while the other half insist that its' bass response is great, and extends deeply. It's either one or the other; this is why objective measurements and frequency response curves should be the norm. I've never encountered a lens in which half of its reviews claim it has no chromatic aberration and the other half claim it has terrible chromatic aberration; it either does, or it doesn't.

 

[thelostMIDrange]

seems like over time a general concensus forms which is somewhat usefull and somewhat accurate......I'm learning that regarding IEM's, reviews are even less worthwile due to the huge sound changes that are heard with tip changes. A iem can sound like 6 different headphones and there's no way to know what any reviewer is using for tips often.

 

[Snow_Fox]

Yeah. Aside from that if you get 3 tips with one IEM set, no way of telling what will be comfortable and what it will sound like.

 

[thelostMIDrange]

right, and the guy could have another 3 of a different material from another iem he has that he is using/trying. Any one tip has 3 variables as well. Length, material, opening. So the permutations on  a half dozen tips is 18 (check my math). And all 18 scenarios will make the iem sound different in some serious ways often,

 

[Currawong]

Quote:

We have here a ghetto in which discussion of objective means of testing is permitted, whereas if there were any rationality to the forum's layout, discussion of objective means of testing would not only be permitted everywhere, but objective substantiation of expressed views would be insisted on, and, if 'subjective' assessments were protected anywhere, they would be ghettoized.

 
The problem that occurs with this is that you get people making incorrect interpretations and assumptions about the meaning of the results of tests and measurements. Heck, quite a few people thought that science is about abusing anyone who doesn't accept their interpretation of results that they have no qualification to understand! That's really no different to someone dissing on another person for not liking the headphones/music/whatever that they do.
 
What we have here is the ability of anyone to be free to express their opinion without feeling anyone insisting on anything (except that they do so respectfully towards others). This means people can even be wrong, and from that learn something, rather than be looked down upon and abused for not looking at things their way.
 
There are only two things not permitted in all but one forum: ABX and DBT discussion. The reasons they aren't permitted is because people either a: get into huge, thread-crapping arguments about them, or b: use them to attack others. Neither is productive to useful discussion. Where people have been using either to better understand what is going on (or isn't) with what they hear, there is no problem. There are actually quite a few reviews now where the reviewer has done an ABX or DBT during the review.
 
Back to the main topic: Reviews make more sense with experience. A good reviewer that can evaluate equipment with a wide variety of music and explain where the strengths and weaknesses are can write something coherent that can make sense to many people. For a person new to all this though, nothing is going to make up for experience, combined with an easy-to-understand explanation for what they are hearing, in the end.

 

[chewy4]

I don't know exactly what each measurement sounds like, so subjective reviews are nice at least collectively. I mean sure I can make an educated guess with measurements, but I'd still have to hear them to get the big picture especially if there are some variables that I missed.
 
That being said, I do feel like some reviewers try way too hard to differentiate between equipment to the point where it's obvious that it's all in their head sometimes. For example I've seen someone compare the three modes of an amped soundcard, when the truth is that the three modes only change a parameter in the volume control.
 
I also disregard any review that says something sounds too "analytical". If you're saying a headphone catches every detail but still fails to have any musicality it's the song's problem since you're hearing everything that the song has to offer.