I've discussed with peers whether this post belongs in the sound science forum or here, and I decided to post it here for the following reasons:
- This study is really about the subjective experience first. And how objective measurements back up those subjective experiences rather than the reverse.
- This study does not subscribe to the fringes of the objectivist or subjectivist belief systems
- Frequency response and CSD waterfall plots are commonplace in the speaker world. Respected journalists like John Atkinson have been using these kinds of measurements for decades. Such measurements can hardly be considered esoteric science in today's Information Age world.
- Only "popular" headphones were selected for use in the study. The measurements shown pertain directly to these headphones.
That being said, moderators should move this post to the sound science forum if they feel it's more appropriate.
Do Objective Headphone Measurements Correlate to the Audiophile's Subjective Experience?
There are those who feel that any attempt to scientifically correlate headphone measurements to the complexities of the subjective experience is futile. This post is not for those people, as they have already made up their minds.
There should be no tension between subjective and objective evaluation. Those who feel this way are likely on the fringes of their belief systems, or have a misunderstanding of the limitations or unreasonable expectations of objective measurements. Others may feel that measurements are better suited for engineers and too complex for the layman to understand. This is a good argument, but one that I’ve seen fall apart once the layman is gradually exposed to more data and becomes familiar enough with measurement patterns, wiggly lines so to speak, to correlate them with actual experience.
Some people may wholly discount measurements if they feel slighted because their favorite headphones do not measure perfectly, i.e. in a flat straight line. This is yet another misunderstanding of how measurements work. While there are those who prefer a technically perfect measurement, most people prefer a little “sugar and spice”. The flat straight line is simply an artificial reference point. Those who have developed specific tastes will likely prefer certain kinds measurement patterns. Oppositely, they may also be adverse to certain kinds of measurement patterns.
Finally, there are those who have unreasonable expectations, thinking that because measurements cannot tell the entire story, they are of little use. It’s true that frequency response, cumulative spectral decays, and non-linear distortion measurements do not tell the whole story. But they do say a lot and this assertion is best illustrated.
While there’s no guarantee that a good measuring headphone will sound good to every person, there is a strong correlation between good measurements and good sound and bad measurements to bad sound. Of course there is the issue of how to interpret measurements, but we will leave that up to the reader, and in a follow up report.
A pool of thirty headphone hobbyists were asked to rate on a five point scale seven headphones. They were not told of the purpose of this study. The headphones in the study were selected because of their popularity in order to increase the likelihood that the hobbyists were familiar with them, and hence could provide a rating. Each hobbyist was asked to rate the headphone as follows without any consideration to price:
- It’s a good sounding headphone
- It’s good, but with some reservations about certain aspects
- It’s OK. Nothing special.
- I don’t like it, but I can see why others do
- It’s a bad sounding headphone
For now, the models of these seven headphones will not be identified. It’s probable that many readers will be able to guess.
Average Rating: 4.07 (1 = good, 5 = bad)
Standard Deviation 1.14
- glaring, sibilant upper-end. I do like other facets of it
- heard it once at a meet..walked away thinking "people actually paid nearly XX for this?? Maybe they will buy my dog poop for 100 bucks per pound as a statement piece"
- the XX was your sacrifice to the Headphone Gods to give you a good YY
As a side note, to illustrate why measurements are important - they keep us honest. A reviewer once said of this headphone:
The XX’s are the flattest sounding, most extended headphones I have ever heard. No headphone I know of will equal them from top to bottom for evenness of performance... I have never heard or thought I would hear headphones this extended and flat in both directions in my lifetime, but here they are.
Average Rating: 1.78 (1 = good, 5 = bad)
Standard Deviation 0.83
- i wanted to love this HP when i owned it years ago, but ultimately found it a bit dull and lifeless.
- and despite the XX’s flaws, I still like them quite a bit. Switching to them coming from sterile sounding cans makes music fun to listen to again.
- I think it's not the best headphone in every aspect but still a very capable headphone with a pleasant sound signature.
Average Rating: 2.00 (1 = good, 5 = bad)
Standard Deviation 0.74
- very airy and clear, wide soundstage,
- It's a damn good-sounding headphone. It's not perfect, but as AA said, EQ works well to correct its flaws.
- I can appreciate it from a technical standpoint but don't really want it near me. Like a professional weightlifter.
Average Rating: 2.95 (1 = good, 5 = bad)
Standard Deviation 0.94
- love the instrument separation and bass - especially the double bass as i find most recordings or HPs bloat this instrument. requires EQing and/or careful amp matching
- I like the clarity and separation from it, and I'm actually okay with the overall FR. Don't like the weird bump in the mids though, and the tonality is definitely artificial.
- properly amped, this is a good one. I gave it to a good friend when I decided it's not for me, and he's still happy with it, and I wouldn't give bad headphones as presents
Average Rating: 3.20 (1 = good, 5 = bad)
Standard Deviation 0.89
- bad sounding headphone. Pain = bad. I think it's about as simple as that.
- The XX is like a beautiful slut with a disease. I know there may very well be repercussions, but I just don't care to think that far ahead. Plus I know I've been bad, and I need to be punished.
- It was my main headphone for years, so I'm quite used to its sound signature.
- I feel they impart a certain body to music, but it's kind of weightless and not planted... kind of like an elephant in zero gravity. Aside from that I found them to be pretty thoroughly "meh."
- I bought these after the usual hype from the likes of YY, ZZ, and a few others kept harping on them. I do not consider them in the same class in the Audeze and HD800. They are thoroughly middling to my ears.
- I've heard several and they were all warm, bright, and metallic. All at the same time. Kinda like when my Dremel's cut off wheel shoots a stream of aluminum particles in my face.
Average Rating: 1.65 (1 = good, 5 = bad)
Standard Deviation 0.49
- I hate to nitpick this headphone because I found it so appealing. For the price I was expecting an ortho succubus that would be everything I wanted it in headphone.
- I'd buy it for $500-$600
Hopefully it should be rather clear by now that headphones with less high frequency anomalies and cleaner waterfall plots tend to sound better to most people. Ultimately readers should decide for themselves if these measurements mean anything to them personally. It should noted that a number of participants stated that they did could not vote for rating #1 (It’s a good sounding headphone) because they did not consider any headphone, no matter how good, to to absolutely free of any issues.
Edited by purrin - 6/17/12 at 8:22pm