@Brooko, you are correct that our headphone measurement rig is not like most others you're seeing measurements from on the web. As I said in the previous post, I think some of the measurement commonalities you'll see between different enthusiast measurements on the web is due to commonly shared opinions and recommendations on how to make and tune do-it-yourself measurement rigs. Also, most of the DIY measurement rigs we've seen make no attempt to simulate the interaction of the headphone/earphone and the human that wears it -- not in terms of external influencing anatomy, nor in terms of attempting to load the headphone or earphone in the same way as the human ear.
Also, in many cases, we have no idea how a web measurement we're looking at was generated (how was position over the mic determined, for example?), what gear was used, what kind of coupler or mic was used, whether or not the system is calibrated, etc. For example, from one of your previous messages:
...Another interesting thing is that there were more measurements showing a 10 kHz peak around the net with that pair of headphones - in fact I think Jude's may have been the only measurement that wasn't. So when something like this comes up, I'd be holding off any judgement.
The person that had the measurement showing the highest level 10 kHz peak with that headphone (the Sony MDR-Z1R) posted this about that measurement:
I made the last measurement. That spike is much bigger than others. I would not necessarily trust mine completely. My measurement rig is a piece of leather and some felt tapped to a broken lamp. But it does seem relatively accurate for most measurements.
That was a candid description of what he used, but, best I could find, it wasn't known until that comment was posted. Because so many are looking to find quick, definitive answers in numbers and lines, then the graphs and specs can have an outsize influence on the perception of a product. While measurements can certainly be useful (otherwise I wouldn't be wasting time and money doing them), I do think it's important to discuss how they're useful, considerations that need to be weighed, their limitations, and simply checking for yourself by listening whenever you can.
...Pretty much all measurements on the internet including Tyll's measurements of various IE800 units show a peak at 10kHz. Now if the 800S follows a similar response, it makes us stop and think when a measurement of the 800S doesn't show a peak there. Even if the measurement is from a industrial level rig. May be Jude's measurement is correct and the IE800S doesn't have the peak. If that is the case, we need to investigate why there are differences instead of discrediting other measurements....if Tyll's measurements of IE800 all show a peak at 10kHz and if the IE800S possibly has the same peak (as stated by some trained listeners), would you start calling Jude's measurement garbage, because his doesn't show a peak there? You can't. We would have to investigate the possible reason for the discrepancy and understand why a certain measurement has the peak or doesn't.
"10 kHz" will forever be a hot topic 'round here! Again, while some may hear a 10 kHz peak of the magnitude shown in some of the IE800S measurements, I do not. There are some differences in our measurement setup here (versus most of the web measurement setups) that may explain some of the differences in some of the measured results. Again, Tyll discussed this briefly in a post on InnerFidelity (and in the video interview with Paul Barton that accompanies the post):
Getting back to the reliance on numbers and lines, and a headphone measurement example that's
not about frequency response:
In a recent discussion about sensitivity of the new Sennheiser HD660S, there was some concern expressed when our sensitivity numbers varied a bit from Sennheiser's. Questions about the accuracy or integrity of their sensitivity spec arose (despite the fact that their lower sensitivity number actually would be viewed by most as disadvantageous not advantageous). It was clear to me at that point that there was probably not a clear understanding about how sensitivity measurements are done, and how variability is inevitable, affected by
many different factors (mic position, earpad conditions, how tight or loose the headphone is, etc.). I never anticipated taking an in-depth look at headphone sensitivity, but, since it came up, we performed sensitivity measurements of the headphone, and then posted about it in some detail. You can read my post about this at the following link:
Again, I think in our pursuit of easy, definitive answers, we sometimes fixate too much on numbers and lines, and finding out that there's usually variability and many other factors to consider -- that you can't just say that line or number is the gospel truth for all -- can be dissatisfying.
Getting back to web headphone measurements, primarily frequency response (and distortion):
Because I did not feel that a lot of the headphone measurements I was seeing on the web reflected what I was hearing (both based on normal subjective music listening impressions, as well as doing frequency sweeps to see if I was hearing the displayed peaks and troughs), we wanted to take a look at what was available -- and what was coming down the pike -- in terms of audio measurement technology, thinking that perhaps we could assemble a headphone measurement setup with the hope of providing measured results that make sense to most people, independent of preference.
Which brings me to this:
In my opinion, then, the best that one can currently do for the kind of headphone measurements we’re seeking as a community is to approximate the headphone on (or
in, as they sometimes are) an average human -- how it wears on the human, simulating the anatomy and the working load that the headphone is subject to
in situ. Since we can’t reasonably get a precise measurement microphone down a bunch of humans’ canals to the eardrum, the best we can currently do is a fixture or manikin that offers some amount of human dimensional simulation on the outside (and inside, where possible), and to simulate the working load the headphone is faced with when worn.
So...is there a fixture or manikin that can perfectly simulate the average human in all these respects? Do we at Head-Fi have systems that perfectly simulate the average human in all these respects? No. And no. What we are doing is employing the best currently commercially-available technology with that goal in mind. It's an asymptotic goal, perhaps, because it's one we can get closer and closer to, but probably never perfectly reach.
Now -- and this very important -- if
hypothetically someone developed a measurement system that perfectly represented the human average for these purposes, what would be the best we could expect from it? The best we could expect from such a system would be measurements that, independent of preference, would make sense to most of those who have heard the headphones that were measured on it. Notice I said
"most" -- not
"all." I didn’t say “all” because there are always going to be those who deviate enough from the average that even a hypothetically perfectly-executed headphone measurement on the hypothetically perfect human average may not represent what those people (who deviate far enough from the norm) are hearing.
Even if you had this hypothetically perfect human average headphone measurement system, would the measurements from it -- even if on-the-whole they make sense to
most -- be representative of every peak and dip for every one of these people? No. Absolutely not, and perhaps especially as you move higher up in frequency, and the wavelengths shorten, where positional and other physical variables can effect increasingly substantial differences. Meaningful headphone measurements above 8 kHz have been a big challenge for decades, and I'll say more on that shortly.
So what do we use in our measurement systems? For headphone measurements, we're working with a head or fixture with more or less fixed dimensions representing average human dimensions. Right now, those apparatuses are the GRAS 45CA and the GRAS 45BB-12 KEMAR manikin.
(In the above photo, KEMAR is face-forward.)
The pinnae / ear canals we use on the GRAS 45BB-12 KEMAR are anthropometric, based on 260 three-dimensional scans of human ear canals. These pinnae include the first bend and the second bend of the canals, with "flesh" all the way to the mics. Because they're more anatomically representative than traditional measurement pinnae, they have (among other features) a more realistic, more oval-shaped entrance point. Here's a photo of our current GRAS 45CA ears (which currently use standard measurement pinnae):
Here are a photo showing the new anthropometric pinna / canal:
Another advantage of the new pinnae is their increased realism
externally. If you've ever felt measurement pinnae, they're typically stiffer than human pinnae and don't readily compress against the head, which is why many who measure headphones often have difficulty measuring supra-aural (on-the-ear) headphones with them. (They can also present problems with measuring shallow-cup circumaurals, too.) The new pinnae feel and move much more like human pinnae and compress against the head much more like (most) people's pinnae do. Here is a photo I took of the supra-aural Audeze Sine on the standard measurement pinnae on a GRAS 45CA:
Here's the same headphone on the new anthropometric pinnae on the GRAS 45BB-12 KEMAR:
Where in-ears are concerned, we've found the new pinnae/canals to help tremendously with more realistic and consistent placement, as the pinnae/canals are definitely more human-like now. With this improved realism, for example, we've found that characterizing the differences between eartip types via measurements (relative to our subjective experiences) is improved.
Am I asserting that we've accomplished the perfect average human with our measurement rigs? Again, no. But we're trying to move in this direction as much as we reasonably can.
The GRAS 43BB ear simulators we're using in the GRAS 45BB-12 KEMAR are also quite different than standard 60318-4 couplers. While they still meet the IEC 60318-4 tolerances, the single high-Q resonance above 10 kHz is replaced by two more balanced, more damped resonances. The splitting of the one high-Q resonance into two low-Q resonances may present an advantage in decreasing the uncertainty in the measurements around the resonance (above 10 kHz). Also, the GRAS 43BB is highly sensitive, and
very low-noise, and extends the lower dynamic range
below the threshold of human hearing. Given its extremely low-noise nature, the 43BB can be used to measure and characterize things like the self-noise of an active headphone (both with and without active noise canceling), which is something we'll be increasingly interested in with the growing prominence of high-fidelity wireless headphones and earphones. It can also help in measuring low-level distortion in headphones and earphones. NOTE: One thing to consider with this low-noise simulator is that it's not suited to very-high-SPL measurements, with an upper limit of the dynamic range to about 110 dBSPL. This hasn't been an issue for us, though, as most of our measurements are set at 90 dBSPL (at 1 kHz).
A little aside:
Speaking of wireless headphones, another update we're making to our measurement lab very soon is the addition of
Audio Precision's new Bluetooth Duo Module to our Audio Precision APx555 audio analyzer. This new Bluetooth module can act as source and sink for AAC, aptX, aptX-HD, aptX-LL, and SBC. Last year we had two aptX-HD-enabled wireless headphones in our office. Now we have many more, and it will be exciting to be able to measure these headphones at their wireless best. We'll be covering this module more over time.
Okay, back to ear simulators:
A few weeks ago, GRAS announced still another coupler designed specifically for measuring high-resolution headphones. On Friday (two days ago) we took delivery of the new GRAS RA0401 High Resolution Ear Simulators, and we'll be installing them on our GRAS 45CA, along with the new anthropometric pinnae for the GRAS 45CA.
This is still another very exciting development for headphone measurements, as obtaining meaningful measurements above 8 kHz with most systems can be enormously challenging. This new GRAS RA0401 High Resolution Ear Simulator also meets the IEC 60318-4 tolerances, but GRAS was able to design it so that its performance from 10 kHz to 20 kHz is substantially improved, that range through which it has a tolerance of +/- 2.2 dB. Here is a graph showing the RA0401's response (including the IEC 60318-4 tolerances) compared to a standard 60318-4 coupler:
Again, meaningful headphone measurements above 8 kHz or 10 kHz have been a major pain point for decades, so I think these new GRAS High Resolution Ear Simulators may prove an important development in the world of headphone testing.
Unfortunately, we haven't had a chance to run our own measurements with the new RA0401 High Resolution Ear Simulators yet, as we shipped our measurement mic preamp power supply back to GRAS for a check-up and any necessary calibration (as it's now nearly three years old, and has been jostled around quite a bit). We should have that (GRAS 12AQ) back in the next few days, and we'll fire up these newest ear simulators just as soon as we do.
...Nexdt thing to note - Alex is using my compensation curve for my Veritas on my set-up. It's been calibrated as well as I can to mimic an IEC711 raw measurement, and I already acknowledge that at 9-10 kHz on my rig, my calibration measures lower than it should. Above 10 kHz on my set-up its a crap shoot - the Veritas is pretty inaccurate. However - up to around 8 kHz my measurements on my rig seem to be pretty good. How do I know? Because I had help (from Ken) in getting some accuracy. We both measured the same IEMs (we mailed them back and forth) and I could check and recheck. He also gave me his raw data and I used that to build the curves. Again - from 9 kHz onward I wouldn't call mine accurate - but I have already had 2 manufacturers (other than Ken) tell me that my rig is pretty close to what they are seeing on their own professional rigs at 8 kHz and under...
I've not yet used a Vibro Veritas, and I'm not sure one can simply EQ it into something approximating a 60318-4 simulator, as the 60318-4 (or 60711) ear simulator was designed to mimic the input and transfer impedance of a human ear. It consists of a main volume and side volumes connected to the main volume by thin slits, and is a pretty precise piece of kit. Here are a couple of cut-through views of a standard 60318-4 coupler:
I'll order a Vibro Veritas to play with it and compare with it, as it's only $79 with mic.
IMHO, it's much more likely that deviations among listener's ear canal anatomies are the reason behind these discrepancies. Have you ever compared ear impressions for custom IEMs?...
Actually, here's a link to a short discussion about ear canals (and eardrum impedance) that I had with someone at InnerFidelity:
...The statement about KEMAR having "anatomically correct ear canals" is a bit misleading in my book. Of course they're closer to human ear canals than perfectly cylindrical ones, but (like HRTF compensation targets) they're still based on averaged data, collected from individuals that may vary significantly....
They state openly that their anthropometric pinnae / ear canals are based on 260 three-dimensional scans of human ear canals, with some adjustments made to adapt the ear canal to interface with the ear simulator. These pinnae include the first bend and the second bend of the canals, with "flesh" all the way to the mics. And of course they're averaged.
And, as you stated, they're closer to human ear canals than perfectly cylindrical metal ones. If you've ever tried to put an anatomically angled IEM into a standard measurement pinnae, you'll know it can be challenging, and, with some in-ears, almost impossible. With the anthropometric pinnae, most in-ears (whether straight or anatomically angled) fit into the ear very much like they would with most real human ears. Just from the standpoint of fit, it helps with more realistic placement relative to the "eardrum," as well as more realistic deformation of the eartips upon insert (which is most obvious when you pull foam-tipped IEMs out of them). Like I said earlier in this post, I think that the best one might reasonably do for the purpose is to approximate an average human, and these more realistic pinnae/canal assemblies seem like a step in the right direction that's
long overdue.
In a LinkedIn discussion about headphone measurements, where the discussion of testing transducers apart from the headphone came up,
Christopher J. Struck of CJS Labs (who is a living library and a wealth of knowledge, especially if you have audio and electroacoustics challenges you need solved) said this:
“All of these points are well taken. Dimensionally, the standard manikin offers median dimensions. In any commercial deign, one must also be concerned with fitting a reasonably wide range of head and ear sizes in order to have a sellable product. With respect to acoustic response, the same can be said: The median ear simulator, pinna and head size in the manikin is intended to provide a response corresponding to the theoretical median listener. This should be in the center of your 'fit range'. If that is not right, everything else is biased accordingly. As Flannigan points out, testing the transducers apart from the system is a separate issue. Ultimately, the transducer performance in-situ is what matters, so any preliminary transducer testing should correlate to the application, i.e., how the transducer will be used in the system or product: e.g., circum-aural, supra-aural, intra-concha, insert, open (low impedance), sealed (high impedance), etc.”
Again, we're doing our reasonable best to find and use the best of what's currently available to provide the most meaningful audio measurements we can. You'll be seeing more posts and videos about this from us, as there is so much to discuss. Yes we've been posting measurements periodically for a couple of years, but, as far as I'm concerned, this is just the start, and with some key developments in the last year or two, and with more coming, it's an exciting time to ramp this up.