Building a Headphone Measurement Lab

[arnaud]

 
Quote:

...

So, the experiment I did the other day was to position the headphones as best I could, take a measurement, and then change the position slightly. Five measurements were taken: centered, up-forward, up-behind, low-forward, and low behind. The data looks like this:

You'll notice two things: it looks somewhat like the various compensation curves mentioned above, and once you get above 8kHz the data moves all over the place.
 

 
Tyll, just found out about this thread today (I should visit this section of the forum more often) and wanted to congratulate you for once again stirring us in the right direction.
 
Your studies on sensitivity of measurements to placement and the proper way to digest the test data into something meaningful is particularly helpful! I have suspected for a long time that headphone measurements at higher frequencies may be a crap shoot at best if not massaged properly and am glad to see some rigorous demonstration...
 
I deal with high frequency vibro-acoustics (both simulation and testing) in my job and one thing that is very clear: anyone who think they can reproduce measurements without spatial / frequency averaging at higher frequencies are fooling themselves... It's nature of the physics of sound and vibration at high frequencies (google Statistical Energy Analysis if you want to know more...) and the reason why your pragmatic approach is a very good one (averaging measurement over slight variation in positioning and doing some frequency smoothing).
 
I would be happy to support you if I can help in any way although I realize I am coming a little late here . One thing you mentioned early in the thread is how to get better diffuseness of sound field for isolation measurements:

• I think the results will not be very good below 3-400Hz because there are not sufficient number of modes in the cavity to support diffuse sound field. However, from looking at your curves, looks like it's a moot point because most headphone don't provide any isolation below 1kHz or so?
• As you mentioned, one way to increase reverberation is to remove the sound absorbers so maybe you could glue the foam pads on a thin board which you could then slide in and out of the test chamber for specific measurement?
• Another which you already know also it seems: use several speakers at different locations.
• Last recommendation is to add rigid panels placed randomly in the chamber to increase reflections (google reverberant chamber ) although I doubt it's practical in your small box...

 
For increase the sound isolation from the outside, it depends on the frequency range, but generally speaking:

1. At low frequency, you typically need to add mass to your panels, one way is to add lead panels (you might not be happy about that if you're planning to take it in/out of your truck often . I think you can buy some similar stuff of adding to car door panels (for bass heads how have issues with subwoofer rattling the whole car...)
2. At mid frequencies, you typically need to add mechanical damping to the enclosure panels. One way is to user rubber like material between the main panel and the inner board (well it's too late for that now though). Incidentally, the lead barrier above can also provide some mechanical damping to the enclosure panels.
3. At higher frequencies, you typically need to control leakage. Any tiny slit or hole will kill your hopes of isolation over 20dB... You've got this one under control it seems.

 
Lastly, I will say just the waterfall plot for headphone might also be relevant. Although we won't see the strong panel / cavity resonances of speaker enclosure I would believe that some resonances in ear cup cavities and diaphragms might become visible (taking longer to decay than the rest of the spectrum).
 
Quote:

Originally Posted by xnor
Noooo, (too much) smoothing is evil!
Imo, not only does it get rid of the spikes that could be accounted for driver resonances (admittedly CSD would be more useful here) or construction imperfections ... but the smoothing could make any headphone look good. (and maybe sell well? That's not what this should be about, right?)
Looking forward to those measurements. Thank you for doing this.

 
I think there is no other way around, see above...

 

[Currawong]

Tyll, nice work on the measurements.  Some of the info is over my head, but it's fascinating how much we have taken for granted with headphone FR measurements or beliefs that are already being challenged. 

 

[xnor]

 
 
Quote:

I think there is no other way around, see above...

@arnaud: A bit of smoothing is needed and fine. I was referring more to that kind of extreme smoothing that some manufacturers do to hide problem areas.
IIRC, back then it sounded a bit like all you can read out of FR graphs is the overall balance (bass, mids, highs) and so the graphs should be smoothed "accordingly".

 

[xnor]

It's so quiet around here, let's hope Tyll didn't lock himself up in that box. 

 

[Tyll Hertsens]

 ... mmmph ... mmmph mmmum ... MMMMMMPH!!!

 

[dBs]

Just noticed this thread and I read all of it (though quickly in some places). An idea occurred to me. At one point the transfer function of the human ear was mentioned. I think I know a [very tedious] way of testing what this is for the average person so that it can be applied to the headphone transfer function for the net end result (remember: two transfer functions in series are multiplied together =D). Whats more, this method would find the perfect environment for testing at CanJam.
 
The idea is this; using human guinea pigs with headphones strapped on (with an experimentally derived known transfer function), play 50 to 100 different frequencies--one at a time--at -100dB while slowly increasing their volume. When the hapless victim can first hear the sound he signals to you. You make note of the dB level at which he heard that tone at. You continue this for the 50-100 different frequencies. Average these values among the sample set. Now you will get a pretty grainy set of points across the spectrum, but they can be cleaned up into a clean[er] transfer function. Take those frequencies with their respective dB levels and turning them into their corresponding Fourier representation. Sum them together into one massive 50-100 term Fourier equation (probably a program to do this) and viola you have a transfer function. Now you will still have to subtract out the known headphone transfer function and with that end result you should have the average human ear transfer function.
 
EDIT: Now that I think about it, that wont completely work, summing them will just give you the same data points. It wouldn't be a perfect average this way but if you shift the sample points (ie 50hz for the first person then 51Hz for the second person etc). Unless there are some amazing algorithms for composite best fit curving. Then take that as your transfer function.

 

[xnor]

Actually I wrote a little java app some time ago ... plays sine waves at a fixed set of frequencies and the user has to turn the volume up/down for each frequency until he reaches the hearing threshold. The same should be repeated with a second headphone and the results can then be submitted to an online database, for further processing. Though, nobody seemed to care. Maybe I'm a bit crazy or maybe I should have posted it in this subforum, idk.
 
One problem, however, is that the headphone(s) might behave quite differently at such low volume levels. That's one of the reasons why I was asking for a test measurement of the FR at lower levels, like 50 dB.

 

[Tyll Hertsens]

 
Quote:Arnuad

I deal with high frequency vibro-acoustics (both simulation and testing) in my job and one thing that is very clear: anyone who think they can reproduce measurements without spatial / frequency averaging at higher frequencies are fooling themselves...

 
Ooo! May I ask what you do exactly?
 
 
Quote:Arnoud

and the reason why your pragmatic approach is a very good one (averaging measurement over slight variation in positioning and doing some frequency smoothing).

 
Thanks, that's very reassuring. High frequency sound waves are kinda like laser speckle, there's significant spacial granularity with all the modal interference. I don't see anyway around heving to move stuff araound a bit in order to get a curve of averages that reasonably represents the energy output from the cans.
 
In the long run, if the FR measurements are accurate, I believe it would be reasonable to smooth the hell out of the curve (or even redraw the line with a curve fit) for display to n00bs and the buying public.  So many of them don't even know what a FR is; just getting curves that show generally if the bass, treble, mids are different between cans would be enough.
 
The good news is that can all be done on the fly by the graph server, so folks will be able to look at everything from raw data to highly smoothed data.
 
 
 
Quote:

I would be happy to support you if I can help in any way although I realize I am coming a little late here .

 
You're not late at all; I've just begun to fight.  It's likely the best help you can be is to just keep an eye on this thread and make suggestions.
 
 
Quote:

 One thing you mentioned early in the thread is how to get better diffuseness of sound field for isolation measurements:

• I think the results will not be very good below 3-400Hz because there are not sufficient number of modes in the cavity to support diffuse sound field. However, from looking at your curves, looks like it's a moot point because most headphone don't provide any isolation below 1kHz or so?

 
The diffuse sound field is mostly needed so as to most mimic ambient noise coming from many directions.  Previously the noise source was in an anechoic chamber from a single speaker, and I was worried that a noise source coming from one direction would create an uncharacteristic isolation reading because the sound might only energize certain modes in the ear cup.  I figured if sound came from many directions, the all available acoustic modes of the earpiece cavity would be available for stimulus. Sooooooo, the fact that low frequency sounds can't be diffuse per se in the box, that's okay, because in essence they already are coming from every direction as the 1/2 wavelength becomes bigger than the box. 
 
I was worried more about the mid frequencies --- maybe 300Hz (~18" 1/2 wave) to 2kHz (~3" 1/2 wave) --- where I'm most likely to get modal interaction with the box. In the long run though, because I have a background subtraction curve from the head without any cans on, nice relative measures should pop out.
 
Quote:

For increase the sound isolation from the outside, it depends on the frequency range, but generally speaking:

1. At low frequency, you typically need to add mass to your panels, one way is to add lead panels (you might not be happy about that if you're planning to take it in/out of your truck often . I think you can buy some similar stuff of adding to car door panels (for bass heads how have issues with subwoofer rattling the whole car...)
2. At mid frequencies, you typically need to add mechanical damping to the enclosure panels. One way is to user rubber like material between the main panel and the inner board (well it's too late for that now though). Incidentally, the lead barrier above can also provide some mechanical damping to the enclosure panels.
3. At higher frequencies, you typically need to control leakage. Any tiny slit or hole will kill your hopes of isolation over 20dB... You've got this one under control it seems.

 
That one should be a sticky --- exactly right.  Fortunately I'm doing the measurements in a quiet neighborhood home in small town Montana, so the ubiquitous low frequency noise of the city isn't much of a problem. I thought of all your suggestions, but the evils of having a slightly poorer isolating box just didn't seem as evil as having a 500 lb box that I had to lug around a few times a year. Anyway, I'm pleased it seems to work better than the room at HeadRoom, and the harmonic series are clearly above the noise floor. Probably got in the way a bit with those low signal level THD+noise Vs F the other day though.
 
 
Quote:

Lastly, I will say just the waterfall plot for headphone might also be relevant. Although we won't see the strong panel / cavity resonances of speaker enclosure I would believe that some resonances in ear cup cavities and diaphragms might become visible (taking longer to decay than the rest of the spectrum).
 

 
Sadly the AP doesn't do waterfall plots. At this point I'm convinced I should look at achieving a Version one of this thing with some good solid but relatively simple data set. I simply believe it's imperative to get a great graphing tool out there that has ALL the important cans in it. Then develop a killer test routine over the next couple of years, and then re-measure all the cans. Nothing like Job security.
 

 

[Tyll Hertsens]

Quote:

Originally Posted by dBs /img/forum/go_quote.gif
 
The idea is this; using human guinea pigs with headphones strapped on (with an experimentally derived known transfer function), play 50 to 100 different frequencies--one at a time--at -100dB while slowly increasing their volume. When the hapless victim can first hear the sound he signals to you. You make note of the dB level at which he heard that tone at. You continue this for the 50-100 different frequencies. Average these values among the sample set. Now you will get a pretty grainy set of points across the spectrum, but they can be cleaned up into a clean[er] transfer function

 
Hiya dBs.  That's quite similar to one of the specified method for measureing headphone FR in the IEC spec ... but not quite.
 
The flaw in your reasoning is the assumption that the audibility threashold is a flat line, and it's not. If you look at the Fletcher-Munson equal loudness contour curves it will show you that audibility of highs and lows is significantly reduced at low volume levels. To compensate for low listening levels you have to turn up the bass and treble. The Fletcher-Munson curves show how much you'd have to boost the lows and highs to compensate.
 

 
So, your idea would result in a curve looking somewhat like the lowest one in the curves above called "threashold."
 
The method described in the IEC spec is to put the subject in a room with some speakers. They play a calibrated tone series in the room on the speakers and the subject has a volume control for his headphones, and a switch to switch between headphone sound and speaker sound.  Then the subject adjusts the volume of the headphones to match the volume of the speakers heard in the room over a series of tones through the spectrum.
 
The unfortunate thing about this method is that it doesn't directly correlate with objective measurements ... and they don't really know why.  There are some papers on this phenomina, and it is speculated that the human perception system somehow knows how to correlate the loudness heard and the distance to the source such that when the actual loudness on the headphones and speakers is technically the same, the listener percieve the speaker as louder because the brain knows it's farther away and therefor must be a more powerful source.  Or something like that.
 
Bottom line, they haven't figure out why they can't get the subjective and objective measures to line up.
 
Weird, eh?
 

 

[Tyll Hertsens]

Quote:

One problem, however, is that the headphone(s) might behave quite differently at such low volume levels. That's one of the reasons why I was asking for a test measurement of the FR at lower levels, like 50 dB.

 
I'll do that at some point for ya, but I don't think it will change much, and as I said above, it's your ears, not the cans, that are the major contributor to changes in the perceived FR with volume.
 
Okie dokie, back to work here, I know I haven't posted in quite a few days, but I have been busy.
 
First up, I did manage to get the "Independant of direction" compensation curve onto the spreadsheets so that we could look at the compensation.
 
As you know, my goal at he moment is to ready myself for raw data gathering at CanJam.  But because there won't be a way to make the measurements public until I've managed to get a working graph display tool up on the web, I feel the need to publish the measurements I gather as PDFs so people can get some value out of them while we wait for a proper on-line display tool. So, I think it's important to have the compensated FR available in the PDF. I had a nice long chat with Wade at Head Acoustice (he's their science guy here in the US) and after a bit of discussion we both came away feeling like the "independant of direction" (ID) compensating curve is probably more appropriate than the "diffuse field"  (DF) curve we have been using. There's not too much difference in the curve, but it does seem to me that the good headphones measure fairly flat in the raw data below 1kHz. When I apply the DF curve to compensate, I get a long gentle upward slope from 1kHz to about 200 Hz.
 
In the graph below, the top to curves are the two compensation curves (ID orange, DF red).  You can see that the ID is flat between 200Hz and 1k, and the DF rises slowly over that area.
 

 
The light blue curve is the raw data from the HD800. The green curve is the ID compensated result, and the dark blue curve is the DF compensated curve.
 
Looking at the raw HD800 FR (orange) it's pretty easy to see that it most resembles the ID comp curve (orange) more than the DF curve (red) through the 200Hz-1kHz region. I guess I just have to believe that the Sennheiser folks are likely to have gotten that part right (it's the 2kHz to 8KHz bit that most struggle with, I bet), and that fact that they look similar encourages me to believe the ID curve is correct. And seeing the hollow roughly between 1kHz and 6kHz on the HD800 seems to jive with what I hear on the 800s.
 
At any rate, I wanted to get the ID curve compensation in on the spreadsheet earlier rather than later.  I will show both compensated FRs on the spreadsheets from CanJam.
 

 

[inarc]

 
Quote:

In the graph below, the top to curves are the two compensation curves (ID orange, DF red).  You can see that the ID is flat between 200Hz and 1k, and the DF rises slowly over that area.

 
Why are your graph and the one from the brochure so different? For example, in the brochure the DF curve falls faster and deeper than the the ID curve, but in your graph it is the other way around (and inverted, of course). Why do you think is this the case? Have the curves been updated at some point? Which one is more to trust?

 

[dBs]

Quote:

 
Hiya dBs.  That's quite similar to one of the specified method for measureing headphone FR in the IEC spec ... but not quite.
 
The flaw in your reasoning is the assumption that the audibility threashold is a flat line, and it's not. If you look at the Fletcher-Munson equal loudness contour curves it will show you that audibility of highs and lows is significantly reduced at low volume levels. To compensate for low listening levels you have to turn up the bass and treble. The Fletcher-Munson curves show how much you'd have to boost the lows and highs to compensate.
 

 
So, your idea would result in a curve looking somewhat like the lowest one in the curves above called "threashold."
 
The method described in the IEC spec is to put the subject in a room with some speakers. They play a calibrated tone series in the room on the speakers and the subject has a volume control for his headphones, and a switch to switch between headphone sound and speaker sound.  Then the subject adjusts the volume of the headphones to match the volume of the speakers heard in the room over a series of tones through the spectrum.
 
The unfortunate thing about this method is that it doesn't directly correlate with objective measurements ... and they don't really know why.  There are some papers on this phenomina, and it is speculated that the human perception system somehow knows how to correlate the loudness heard and the distance to the source such that when the actual loudness on the headphones and speakers is technically the same, the listener percieve the speaker as louder because the brain knows it's farther away and therefor must be a more powerful source.  Or something like that.
 
Bottom line, they haven't figure out why they can't get the subjective and objective measures to line up.
 
Weird, eh?
 

I am assuming that the headphone attenuation of the outside speaker sound was accounted for? I figure that that would have a significant influence unless you are using something like K1000s.
 
If the distance idea is an issue, then I would aim to minimize that distance element. Start the speakers farther out and move them to closer positions. If you graph the differences you might be able to extrapolate an equation that would result in a convergence point (theoretically it would be zero) and possibly create a compensation curve that you could apply to the matching headphone-speaker dB level. Kind of how a solid state device curves if extrapolated out (negatively) converge on a single point.
 

 

[Tyll Hertsens]

Quote:

I am assuming that the headphone attenuation of the outside speaker sound was accounted for? I figure that that would have a significant influence unless you are using something like K1000s.

 

 Yeah, in Sennheisers set-up they have a foot switch that switches between speakers and headphones, so as you take the cans on and off you can easily switch between.
 
It's an awful tedious way to test headphones, and as I say, the numbers come out different than the objective measurements.  I think that's why we see Sennheiser (for example) deviating from technically flat in the HD800 measurements above --- I say that because I'd bet a six-pack they could make them measure flat if they wanted to ... well, except in the low bass and the noise from reflections and stuff in the high highs. I bet they could have made them pretty damn flat between 100Hz and 7kHz if they really wanted to.
 
 
Quote: dBs

If the distance idea is an issue,

Like I said, I'd have to read the paper again, but as I recall it did seem like another one of those perceptual things that's hard to get around.
 
Like everything, I suppose, measuring headphones is a lot harder than it looks. Also, remember, there are a whole lot of people smarter than us who have been working at it a long time in relatively well funded labs who are still struggling quite hard to get this thing figured out. We have to be careful not to fool ourselve into thinking we're doing something novel here. There is a LOT of information buried in the AES library on headphone stuff; I've been reading, but have just begun to scratch the surface.

 

[SP Wild]

 
Quote:

 I think that's why we see Sennheiser (for example) deviating from technically flat in the HD800 measurements above --- I say that because I'd bet a six-pack they could make them measure flat if they wanted to ... well, except in the low bass and the noise from reflections and stuff in the high highs. I bet they could have made them pretty damn flat between 100Hz and 7kHz if they really wanted to.

Of course they could - but then who would buy the HD850? ....if the HD800 is perfect....

 

[Tyll Hertsens]

Quote:

 
Why are your graph and the one from the brochure so different? For example, in the brochure the DF curve falls faster and deeper than the the ID curve, but in your graph it is the other way around (and inverted, of course). Why do you think is this the case? Have the curves been updated at some point? Which one is more to trust?

Sorry man, just missed your post somehow yesterday.
 
In my graph the two curve are offset from each other a bit.  You can see that if you were to lower the ID curve about three dB so that it lines up in the bass, it would pop below between 100hZ and 3kHz. Also my curve is an actual measurement from this head. The curves you show are just generalizations of course.  So both are right, but my curve is exact for this head.