[samvafaei]

Hello Head-Fiers!

We have made a video explaining some of the common issues in making frequency response measurements, and the solutions that we have come up with. These include bass leakage/coupling issues, treble inconsistencies, and target response curves.

The purpose of this video is to share our findings, and to hear your comments and suggestions on how to improve our measurements. So please share your opinions and ask questions about how we do things. Some of the stuff I explain with video are a bit simplified to keep the video from getting too long, but we can go in more detail here if you guys are interested.



Sam

 

[castleofargh]

maybe you'd be able to settle the battle in the Z1R topic with your warnings ^_^

 

[aleksanderp]

Sam, the video is very informative and enjoyable. May I suggest that you post the uncompensated graph in each headphone's measurement as well? As people's taste change and you may be revising your curve in the future. Thanks and please keep up the good work.

 

[samvafaei]

maybe you'd be able to settle the battle in the Z1R topic with your warnings ^_^

Well, I'm trying not to get in the middle of that discussion at the moment and this video wasn't meant as a response to that. But yes, hopefully it will provide some perspective on the state of currently available measurements.

Sam, the video is very informative and enjoyable. May I suggest that you post the uncompensated graph in each headphone's measurement as well? As people's taste change and you may be revising your curve in the future. Thanks and please keep up the good work.

Thanks for your comment. Providing raw responses of our headphones measurements has been on our to do list for a while now, but they are not our highest priority task at the moment. However, rest assured it will be done sooner than later.

 

[bigshot]

Not knowing how my own particular noggin might affect readings, variations in test subjects is interesting, but there's no way for me to know how it applies. And target curves are fine, but only if they're unchanging. If I look at one chart comparing to one kind kind of target curve, it's pointless to compare it to another done with a different target curve. Most people use the Harman curve. I would just use that as a standard so your results are comparable to other people's.

When I look at headphone measurements, I realize it isn't precise. I don't sweat a couple of dB, I just want to know I'm comparing apples to apples. If It can get in the ballpark of the Harman curve, I can EQ them to polish it. I just want to avoid grossly imbalanced cans that require drastic adjustments to sound good.

The thing that irks me the most about response charts is the way they are graphically laid out to emphasize the extreme frequencies that play a smaller part in perceived sound quality than the core frequencies. But I guess that is the way someone with no graphic sense designed it and we're stuck with it.

 

[castleofargh]

you're wiser than I am. ^_^
I'm guessing you've calibrated the in ear couplers by placing them on the dummy and trying to match a headphone response vs the response on the dummy alone? did it return significant variations compared to the default calibration provided with the mics? I'm fairly curious about such devices and would have bought some already if my massively dominant listening practice wasn't IEMs.

what I really like with your propositions is showing how much a headphone is user/placement dependent. it's a notion that I really like. Tyll (innerfidelity) has an interesting approach to this by moving the headphone around on the dummy head and provides all the RAWs for us to feast on, but you add the notion of different people finding their own comfy position which is relevant too.

now the insecure objectivist in me has this to say: what about "dramatic" problems to this method like how our ears keep growing all our life, how do you explain the "drastic" changes after 2 years of time when your ear is a full 2.5mm bigger? and what if your glass user changes his glasses? and what if someone is fired? the insecurity is killing me!!!!
seriously I like the ideas of mixing things up to use objectivity in most methods while still picking portions of results you guys feel subjectively nicer/closer to your perceived balance. I wouldn't take that road for actual research(I'm not a scientist so it doesn't matter^_^), but to share with the public it's very nice IMO.

 

[samvafaei]

Not knowing how my own particular noggin might affect readings, variations in test subjects is interesting, but there's no way for me to know how it applies. And target curves are fine, but only if they're unchanging. If I look at one chart comparing to one kind kind of target curve, it's pointless to compare it to another done with a different target curve. Most people use the Harman curve. I would just use that as a standard so your results are comparable to other people's.

As I say in the video, one of the reasons the Harman curve didn't work for us, was that it was based on measurements on a different head. Attached is the frequency response of the HD 600 (albeit not the same unit) on our rig (Green) vs the Harman's rig (Red), and as you can see there's no way that a single curve would work on both. So I think in the end the Headphones Target Response will be more of a "standard" than a fixed curve, since each head and torso simulator (HATS) should have its own target curve.

you're wiser than I am. ^_^
I'm guessing you've calibrated the in ear couplers by placing them on the dummy and trying to match a headphone response vs the response on the dummy alone? did it return significant variations compared to the default calibration provided with the mics? I'm fairly curious about such devices and would have bought some already if my massively dominant listening practice wasn't IEMs.

what I really like with your propositions is showing how much a headphone is user/placement dependent. it's a notion that I really like. Tyll (innerfidelity) has an interesting approach to this by moving the headphone around on the dummy head and provides all the RAWs for us to feast on, but you add the notion of different people finding their own comfy position which is relevant too.

now the insecure objectivist in me has this to say: what about "dramatic" problems to this method like how our ears keep growing all our life, how do you explain the "drastic" changes after 2 years of time when your ear is a full 2.5mm bigger? and what if your glass user changes his glasses? and what if someone is fired? the insecurity is killing me!!!!
seriously I like the ideas of mixing things up to use objectivity in most methods while still picking portions of results you guys feel subjectively nicer/closer to your perceived balance. I wouldn't take that road for actual research(I'm not a scientist so it doesn't matter^_^), but to share with the public it's very nice IMO.

Yes, the ear microphones had to be calibrated to match the response of the ear simulator in the HMS. There difference in response was both in the Bass and Treble ranges.
I share your concerns, probably to a lesser extent though but it's true that if our optically challenged colleague changes his glasses it could add a bit of unwanted variables to our testing procedure. There's not much we can do about these things but to do more averaging, and be transparent about the shortcomings in headphones measurements. In the end, there doesn't some to be an absolute frequency response for headphones (unless it's for and measured on a specific individual), but an average frequency response. This makes the need for self-calibrating headphones and scoring the consistency of the headphones more important.

 

[bigshot]

How do you know that it's the head making the difference? Couldn't it also be sample variation? I heard a headphone designer say that his manufacturing tolerances allowed for +/- 1dB, but most other headphone manufacturers allow 2-3 dB. I would imagine that above 14kHz or so, they just take whatever they get.

 

[samvafaei]

How do you know that it's the head making the difference? Couldn't it also be sample variation? I heard a headphone designer say that his manufacturing tolerances allowed for +/- 1dB, but most other headphone manufacturers allow 2-3 dB. I would imagine that above 14kHz or so, they just take whatever they get.

That of course is a possibility, but a highly unlikely one for a couple of a reasons. Harman sent me measurements of 6 different headphones to compare and not just the HD 600, and you see a relatively similar offset in all of them. Additionally, you can compare the measurement of our HD 600 to Tyll's and there's not much difference there. Finally, looking at the amount of difference in the two measurements, one of our units would have be defective, with a nearly identical defect in the L/R channels.

 

[bigshot]

If everyone's getting the same measurements, then it seems like there's no problem at all. The next thing to do then is to work on educating people about what those charts represent. I see people in forums comparing wiggles on the far right above 10KHz to see which can has "better treble" and extrapolating that variations of 1 or 2dB roll offs at the bottom mean "weak bass". Too much judging by what doesn't matter and not enough focus on the core frequencies that count. That's why I don't like the way frequency response charts are laid out. They emphasize the first and last octave visually and collapse the range people should really be looking at. A little bump of +3dB at 1-2kHz looks like very little on a graph, but it means a lot more to the sound than all the mountain ranges in the highest octave.

 

[samvafaei]

If everyone's getting the same measurements, then it seems like there's no problem at all. The next thing to do then is to work on educating people about what those charts represent. I see people in forums comparing wiggles on the far right above 10KHz to see which can has "better treble" and extrapolating that variations of 1 or 2dB roll offs at the bottom mean "weak bass". Too much judging by what doesn't matter and not enough focus on the core frequencies that count. That's why I don't like the way frequency response charts are laid out. They emphasize the first and last octave visually and collapse the range people should really be looking at. A little bump of +3dB at 1-2kHz looks like very little on a graph, but it means a lot more to the sound than all the mountain ranges in the highest octave.

Agreed. That's what we're trying to do with our scoring algorithm. It has less weight in low-bass and high-treble regions, and we have even more plans to make the scoring system more perceptually relevant in the near future.

 

[Strangelove424]

The thing that irks me the most about response charts is the way they are graphically laid out to emphasize the extreme frequencies that play a smaller part in perceived sound quality than the core frequencies. But I guess that is the way someone with no graphic sense designed it and we're stuck with it.

That can be fixed easily. This is a quick solution, but just changing the geometry of the representation has startling results, without changing the data.

To be honest, I never quite understood the Harman curve stuff. The justification seems to be based on statistical percentage of listener preferences, but if preference defines the ideal EQ curve I might as well make up my own. Am I missing something?

 

[castleofargh]

if I had to give one advice to people it would be not to look at the upper trebles much. but at the same time it's still nice to have a graph to get an idea if there is "something" or "nothing" in the area we can still hear. so I don't have an easy solution.

Harman simply hoped to do what they have done for speakers, push the industry to agree about a preferred standard. of course customization is always better for the individual. but the headphone industry has been making headphones with the weirdest responses while still claiming it was amazballz high fidelity. and we can't really tell them to piss off because there is no consensus on how bad their signatures are. if we all agreed on Harman's target as the right standard for preferred signature, new headphones would try to avoid going too far off and headphones that are absolute crap wouldn't get away as easily as before with trying to sell for 5000$. so it's important to have "a" standard as it sets a general direction for the industry. it was IMO a fairly massive event for the speaker world and really helped making the all market much better.
for the headphone market I do feel like many brands took the series of Harman papers seriously and that it might have participated in getting many rather cheap headphones/IEMs with a fairly nice sound. but I really have no evidence of that and could be full of crap right now ^_^. gut feeling 110%.
of course if that direction the standard sets isn't actually a consensus on preferred signature or perceived neutral, then we would be aiming at the wrong target like idiots and that would be bad for us all. the same way we have been for years with the diffuse field target that almost nobody finds to sound neutral or the most enjoyable. yet most graphs are shown with that compensation as reference. and way too many people still think that flat on those graph is the legendary "neutral" that lets us join an elite group of audiophiles.
I personally fall between diffuse field and the Harman target so to me it doesn't really matter, I just accept that I'm not standard ^_^. but if I could only pick one of the 2, I would definitely go for Harman. a little too warm is dull, a little too bright is pain.
I know many people who feel that the target on the Golden Ears website sounds to them more aligned than the classic diffuse field. and they too go for a boost in the low end. so maybe there really is something there? but we'd need a little more than my private conversations to decide what is most often perceived as nice or neutral.
as a listener I will of course keep EQing everything to my very own preferences. but that has nothing to do with standards. I also feel that chairs are too low and wish there were more left handed mice ^_^.

 

[bigshot]

Music is mixed with human ears and human ears don't perceive all frequencies evenly. An arbitrary concept of "flat" needs to be compensated for what our ears perceive as balanced. It makes total sense, but it's just an approximation because everyone has a slightly different noggin. I think the Harman curve is well thought out, and I think it's fine for what it is.

 

[Strangelove424]

I like the idea of a standard, and think Harman has done interesting work, but if the aim is to find a curve that mimics the natural perceptions of sound, it would be nice to see something more objective, like a weighing of frequencies based on perceived loudness vs. actual SPL. I guess the problem with that is measuring perception. Maybe a listening test prior to EQing. I like the idea of delineating differences between "flat" for a mic transducer vs human ears, but if people are deciding this just based on preference stats, it seems like the target will keep moving. Even mastering decisions will effect our concept of what we expect "neutral" to sound like.

The purpose of this video is to share our findings, and to hear your comments and suggestions on how to improve our measurements. So please share your opinions and ask questions about how we do things. Some of the stuff I explain with video are a bit simplified to keep the video from getting too long, but we can go in more detail here if you guys are interested.

Sam

Sam, I really like how you guys quantify as much as you can. Your work on anatomical differences is very interesting. I have a large tragus and anti-tragus and wondered how that might effect the perception of sound. Maybe we each need our own custom ear molds to do proper EQ. Your meticulous measurements I noticed even includes clamping pressure. 1.4lbs on my HD600s when new was enough to give me a headache, and it’s good to know the exact force now that could cause that.

If I had one suggestion it would be to include a bit more subjective observation. When I checked the “best of” suggested headphones, I couldn’t always see the reasoning for the choice and felt like there must be a guiding factor or a weighing of features that wasn’t explicitly stated. Nobody can avoid having a preference for one headphone’s sound over another, so it helps to know the reviewer’s preference in addition to objective performance metrics.

Great work!