Personally, don't take the measurement so serious. The placement of the HP on the H.A.T.S. will vary the result. The hardness of the artificial ears will vary the result. The equipment variation will affect the result. There're many many factors.......
You're absolutely right that there will be differences in results between different measurement rigs. You're also right that the hardness (and other aspects) of the artificial ears may vary the result. For couplers, we started off with the
G.R.A.S. 45CA, which uses the type of pinnae and ear simulators common to most lab-type heads used for headphone measurements. Last year, we switched to the G.R.A.S. 45BB-12 for a number of reasons, which I'll be posting about (both here on the forums and on videos) in greater detail later. (We still have the 45CA, and will keep it for further studies and comparisons.)
However, specific to your comment, among the updates that came with the 45BB-12 were G.R.A.S.'s new anthropometric pinnae that are designed to represent a mean human ear canal, based on a project from Denmark's Technical University (DTU) that mapped the scope of the human ear canal using over 300 three-dimensional scans of human ear canals. This results in some key changes versus standard measurement ears. Starting with the canals, these measurement pinnae represent characteristics of the human ear canal, including the canal's 1st and 2nd bend, the oval interface with the concha, and "flesh" all the way to the eardrum--of course, there were tweaks that were necessary to adapt the mean human ear canal to the ear simulator microphone, but the overall characteristics were preserved.
Here's a cut-through view of one of the anthropometric pinnae:
By comparison, other measurement pinnae have ear canal extensions that are cylindrical or conical and that do not mimic the complexity of the human ear canal. These pinnae/canals were initially designed for testing hearing aids, so the cylindrical and conical canal extensions worked well for those applications.
Immediately below are photos showing one of our original standard pinnae. Note that the "flesh" in the canal ends quickly, coupling to the cylindrical metal extension that leads to the microphone (ear simulator). Also, note the canal shape to the microphone is a straight cylinder. Again, this is the most common type of configuration found in lab-type headphone measurement heads.
Here are a couple of photos (below) showing one of our anthropometric pinnae, its canal (with bends and features more consistent with human canals), and "flesh" all the way to the eardrum.
Additionally, the outer portions of the anthropometric pinnae are designed to be softer, with structural changes that make them more realistic, more flexible, more like actual human ears. Here are two photos (below), the one on the left showing a supra-aural headphone on one of our original pinnae, and the one on the right showing that same headphone resting on the new anthropometric pinnae.
Versus standard measurement pinnae, the more human-like softness of these anthropometric pinnae don't only behave and compress differently with supra-aural (on-the-ear) headphones, but also with circumaural (around-the-ear) headphones, especially when they're shallow enough that the inside of the earcup makes contact with the pinnae, as we've found most circumaurals will do. (We've also found these new pinnae/canals also help a lot with most IEM measurements, but that's another topic.)
The updates to the pinnae/canals alone may contribute to differences in the measurements when compared to other systems.
Another key difference with the 45BB-12 is the inclusion of the new G.R.A.S. 43BB low-noise hearing simulators. I won't go into great detail about these new simulators here, but I do want to mention that the 43BB introduces updates versus the IEC 60318-4 (60711) ear simulators commonly used. The low-noise microphones have very high sensitivity and low inherent damping, which results in key differences between a standard 60318-4 (60711) ear simulator and a 60318-4 (60711) with the low-noise microphone. Up to 10 kHz, the low-noise microphone is similar to a standard 60318-4 (60711) simulator. However, beyond 10 kHz, the differences are substantial--the standard simulator has a high-Q resonance at around 13.5 kHz. With the low-noise simulator, that single high-Q resonance is replaced by two smaller resonances. A filter unit controls the mechanical resonance of the low-noise microphone. The combination of the filter and the low damping of the diaphragm cancels out the high peak of the resonance in the simulator. Of course, among the goals for this is more meaningful measurement data beyond 10 kHz.
The updates to the ear simulators may also contribute to differences in the measurements when compared to other systems.
Again, we
are seeing differences between some of our measurements and others. I'm not going to discuss those differences now, but we've been doing informal surveying to examine subjective correlation with the measurements, particularly when we're seeing differences from other measurements, and it's been an interesting enough activity that we'll likely be more formal with this exercise down the road.
There are
so many different measurement rigs.
I have prevously discussed many of the different systems we've seen measurements from here on Head-Fi, but that post certainly doesn't come close to covering
all of them. As described in that post, some are more industry standard lab-type setups, like InnerFidelity's, Brent Butterworth's, and ours. Most are probably DIY measurement setups. For example, one of the MDR-Z1R measurements recently posted in this thread (and in InnerFidelity's recent article about this thread) was made with a setup described by its user as
"...a piece of leather and some felt tapped to a broken lamp. But it does seem relatively accurate for most measurements."
As Tyll has done (and which we used and learned from when we started with this a couple of years ago), we will be shooting some
Head-Fi TV videos that go into more detail about our audio measurement setups here, as well as some of our audio measurement procedures (especially where electro-acoustic measurements are concerned, as they can be more nuanced than solely electronic measurements). If you’re using headphone measurements to help you make buying decisions--or to guide your headphone DIY modifications--it would be tremendously helpful to know more about the various differences in systems, methods and procedures used by your preferred source(s) of headphone measurement data.
Long story long, as you've mentioned,
@Music818, the various systems, procedures, and even the hardness and shape of the pinnae/canals will impact the measurements. Again, this is something we'll be examining further, on an ongoing basis.
NOTE: Much of the technical information about the anthropometric pinnae and the low-noise ear simulators mentioned above comes from
this whitepaper by Peter Wulf-Andersen & Morten Wille. Also, Mike Klasco from audioXpress wrote
an excellent article about these new systems.