OK, this sort of opens Pandora's Box of long winded forum posts, but I'll fight the urge to ramble on and try to keep it succinct. I will qualify my response by noting that I am opinionated on this matter and my response reflects only my take on it; it's not any official company line.
For years, we've used the accuracy score for evaluation in development. Long before I got here, actually. It's a tremendously useful metric when designing and it is good for knowing how close we have come to the target curve. I don't imagine we will ever stop using this as a development tool nor is our target curve likely to change.
That said, we're probably not going to put it on the main pages of our website anymore. Why stop after all these years? Well, basically, it's can be a bit confusing at times to some folks, especially when it's simply a percentage listed without reference. To really understand what and why, it's better within the context of something like a white paper. FWIW, I've been tasked with writing a white paper on exactly this; hopefully it won't take too long, although I need to fit it in among other responsibilities.
I mentioned it briefly earlier in the thread, but I'll expound on it a bit more here. There are a couple issues with publishing the accuracy score in general:
1. While Etymotic has used it for years, nobody else does. It's our own metric and not an industry standard, so many customers don't know what it means relative to what else is out there. For example, I've measured many many earphones over the years and the majority of them were sub 70% (and many significantly lower). So if I take an HF5 with an average accuracy score of 85%, I could argue that it's higher fidelity than other, more expensive earphones. By our metric, that would be a factual statement (note fidelity does not reflect preference at all, just pure accuracy). I've actually wondered if publishing 85% accuracy for the HF5 hurt us or helped us. 85% sounds like a midrange "B" average, even if it is quite accurate compared to most of the field.
2. The other thing is that (I believe) the accuracy score tells you more and more the closer you get to 100%. I feel very comfortable that I would like any earphone that had a score of 100%. On the other hand, I may or may not like an earphone with an 80% accuracy score. It really depends on where the inaccuracies lie. A good example of this is the ER4XR. It's accuracy score would be the same as the ER4SR except that we added a bit of a bass lift, which hurts it mathematically, even though I still think it sounds quite accurate. I think it sounds more accurate than the 4P because of where the earphone is accurate (the midrange and the high frequencies) but its accuracy score is lower. The accuracy score is weighted but doesn't totally account for my preference. I'd trade very low frequency accuracy for high frequency accuracy (which is a personal preference). OTOH, midbass inaccuracy often drives me nuts, so I won't usually accept that.
Another issue we run into is when two dissimilar earphones have the same accuracy score. The HF5 has an average accuracy score of 85%. The MC5 also has an accuracy score of 85% on average. This exact thing caused confusion with a friend of mine went to our site. He called me and asked: "Can you explain to me why I would buy the HF5 instead of the MC5 when they have the same accuracy score and the HF5 is significantly more expensive?" I was able to explain it to him, but he would've been confused if he hadn't had access to me. The HF5 is more accurate in the midband than the MC5. The peak is better centered at 2.7kHz and I think it sounds more natural. The MC is flatter in other areas but the scores end up being about the same. The accuracy score is useful in that it tells you that they both are very accurate earphones, but as a standalone number, it isn't enough in this particular case.
So, after all that rambling about *why* we didn't publish the accuracy scores of the new version, I'll actually attempt to answer the question. But first, some historical context will help. Keep in mind that I wasn't here during the ER4S development, but I'll recount my understanding of the origins. The original accuracy score of the ER4S was based on testing a number of units, averaging the responses and calculating the accuracy score, which yielded between 91-92% accuracy for that average curve. Individual units vary, with some being in the low 90's and some being in the very high 80's. Little differences can easily knock a point or two off here or there, so there is going to be a little variation. With the new ER4SR, the accuracy score comes out between 92-93% accuracy. Thus far, most of them seem to come in around 92% and with a much tighter grouping. Now, if it was the exact same earphone design, can you hear a 1% difference in accuracy? I don't expect so. In fact, I'd bet a lot of money against it. The ER4SR is a little bit more accurate than the ER4S. But I think the shape of the curve and the differences are what make it better. I've been pretty careful not to oversell it compared to the ER4S, but I do feel it's an improvement. It's not wildly more accurate (or even wildly different), but it is a bit different and I do prefer the new shape of the response (what else did you expect me to say?).
*Whew* that took a while.
h34r:
