How to compare headphones?
Aug 21, 2020 at 7:02 PM Post #16 of 58
  1. If I hear a difference between two headphones: Is there really a difference?
  2. After I convinced myself that two headphones sound different: Which sounds "better", i.e. which shall I get?
Ad 1: There are some less controversial differences, e.g. 2 $ earbuds that sound like a tin can or the difference between a Meze Empyrean and a Stax SR-009s when playing a bass heavy pop song. But then there is a large middle ground where I would not be so sure of my hearing.

Ad 2: Chances for a scientific answer on 2. are slim, even lower than for 1.
@RickInHouston even pointed out that chosing the headphone which has the "sound I want" may not be wise.

There is one headphone for which I am inclinded to think that 1. there is a difference and 2. it actually sounds "better" than the others is the Stax SR-009s - if bass heavy pop songs are excluded for the moment.
(I exclude also the Sennheiser HE-90 and HE-1 here because they are too pricey.)

Which still leaves the amplifier choice for the Stax open. I will have to be careful to not fall for amplifiers that are expensive without audible benefits.

Nevertheless, if you have further good advice on choosing/comparing headphones and avoiding expectation bias, I am looking forward to read it here.
After all, the 009s is a significant investment, so I will not buy it in the next few days.
Your first question can be only answered by an objective test, which is pretty hard to do. Double blinds with different headphones are pretty hard for many reasons, and this is the reason we leave this to preference. The second one, better in terms of audio perception is subjective, thus, there won't be any consensus that can represent a statistical majority. You evaluate your priorities and choose with them in mind.

If you want one of the most objective ways to compare headphones is to read measurements. They're usually not perfect due to problems in compensation curves and equipment, but, they are a starting point. You can define better by getting transducers with low distortion and a good FR that correlates with what you're looking for.
 
Aug 22, 2020 at 8:05 AM Post #17 of 58
The FR of headphones is something that to a listener is subjective and cannot be correctly objectively categorized.

It is subjective at the individual level, but not so much at the statistical one (or at least waaaaay less so than FR curves disparity among headphones would suggests).

There's a pretty solid argument to make that if headphones adhered a bit better to the Harman curve or its derivatives sound quality would be statistically perceived to be better across the whole industry. It isn't a perfect target and individual preferences will vary, but a pair of $1500 headphones with 20 dBs peaks and dips in the upper midrange / lower trebles should rather be seen in my opinion as a piece of crap that largely exceeds these variations in personal preferences, instead of a tasteful variation around the usual target curves.
 
Aug 22, 2020 at 8:06 AM Post #18 of 58
If you want one of the most objective ways to compare headphones is to read measurements. They're usually not perfect due to problems in compensation curves and equipment, but, they are a starting point. You can define better by getting transducers with low distortion and a good FR that correlates with what you're looking for.

I very much aim to have a better understanding of the connection of measurements and my audio perception of a headphone.

What measurement shows that a Stax 009s sounds "more detailed" than other headphones, say dynamic headphones?
(I am pretty sure that the 009s actually sounds "more detailed", even though there is still a chance that I am biased and only tricked myself into believing it.)
My hypothesis is that it is not only the frequency response.

Can the "more detailed" music playback be identified by the low distortion of the headdphone?
Or the impulse response?


One issue I have with Stax is that I do not find detailed informations on the headphones and amplifiers.
I would wish that Stax published more measurements, also given the price point of their products. But then again there a third-party Stax amplifier manufacturers who publish no measurements at all - instead people exchange in forums about the huge (subjectively perceived) differences between amplifiers.

Stax international states on their website:
  • SR-009s: Frequency response:5 – 42,000Hz. [But what is the + - dB range?]
  • Amplifier SRM-700s:
    • Harmonic distortion: 0.01% or less (1kHz/100Vrms output) [Well, few amplifier manufacturers publish THD across the frequency range.]
    • Maximum output voltage: 450Vrms (1kHz). [The specification @ 1 kHz makes me wonder about behaviour at other frequencies.]
    • "The output of J-FET excellent in low-noise amplification as well as large current emitter follower circuit brings out the full performance of earspeakers. Even the music source with much high-frequency ingredients can easily be driven." [This makes me wonder about the performance of other Stax amplifiers concerning music with "much" high-frequency components. I would also like to learn how to quantify and measure the "ease to drive a headphone".]

Would you have any further insights or can direct me to places where I can find more about measurements in particular for Stax equipment?

I am looking forward to your feedback. Thank you!
 
Aug 22, 2020 at 8:14 AM Post #19 of 58
What measurement shows that a Stax 009s sounds "more detailed" than other headphones, say dynamic headphones?
My hypothesis is that it is not only the frequency response.

Can the "more detailed" music playback be identified by the low distortion of the headdphone?
Or the impulse response?

Good answer here :
https://old.reddit.com/r/oratory199...beats_solo_pro_is_the_best_headphone/fpay3b5/
To make it short : frequency response and impulse response are intrinsically linked.

Most headphones still vary by a lot more than 5 dBs relative to the various FR targets that have seen some empirical validation by Sean Olive's studies, and that includes Stax headphones. Add to that individual variations as explained by Oratory's post.
Most headphones have pretty low levels of distortion that most people would struggle to identify.
So it's likely that yep indeed it's mostly FR to look for.
 
Aug 22, 2020 at 9:36 AM Post #20 of 58
statistical one
You're only considering the sampling method and value assignation of subjective data. I mean, you are using objective methods to analyze preference (what Harman did). It does not make the underlying data objective (preference), but it lets you draw some correlations and trends out of it.
 
Aug 22, 2020 at 9:48 AM Post #21 of 58
What measurement shows that a Stax 009s sounds "more detailed" than other headphones, say dynamic headphones?
There isn't. You can draw some insights out of headphone measurements (such as an elevated or depressed part indicating some effect most listeners will experience). Having good measurements in general is a good start.


One issue I have with Stax is that I do not find detailed informations on the headphones and amplifiers.
I would wish that Stax published more measurements, also given the price point of their products. But then again there a third-party Stax amplifier manufacturers who publish no measurements at all - instead people exchange in forums about the huge (subjectively perceived) differences between amplifiers.
There is two reasons you don't see many measurements of electrostatic equipment: the demand doesn't require it and it is pretty hard to measure. People buying Stax aren't measurement-based, and usually do not care about the result, thus, nobody will do it. Technically, you have to place an attenuator in series with the output of the amplifier since the voltage is too high for many if not all audio analyzers.
 
Aug 23, 2020 at 4:41 AM Post #22 of 58
It's hard to compare measurements made by two different people too. Most places online apply compensation curves that mean comparing two different people's measurements is apples and oranges.
 
Aug 23, 2020 at 5:23 AM Post #23 of 58
The FR of headphones is something that to a listener is subjective and cannot be correctly objectively categorized.
This sounds interesting, but I am not sure how to understand this. Do you mean:
  1. The FR of a headphone as perceived at the eardrum of a person is dependent on the individual person (because of geometry of the ear canal, placement of headphones, etc.), or
  2. Given two persons that experience the same FR of a headphone at the eardrum, they might still perceive the played sounds different in their brains (because of different sensitivities of hair cells, neural deficits, different processing in the brain, etc.).
 
Aug 23, 2020 at 5:40 AM Post #24 of 58
As I understand it, there are two opinions on my question "What measurement shows that a Stax 009s sounds "more detailed" than other headphones, say dynamic headphones?"
So it's likely that yep indeed it's mostly FR to look for.

There isn't.

If the explanations referenced by @MayaTlab are correct, this would basically mean that any two headphones could be transformed into each other by DSP (EQ) - unless there is a fundamental flaw with one headphone (e.g. frequency response is zero for some frequencies or unusually high distortion).

If @MayaTlab was true, it would have a massive impact on the audio industry.
(Well, maybe not - given the ongoing discussions on amp and cable sound.)

I have not read/watched through all these references, yet. So, maybe I misunderstood the posts above.
I look forward to your feedback.
 
Aug 23, 2020 at 6:28 AM Post #25 of 58
You're only considering the sampling method and value assignation of subjective data. I mean, you are using objective methods to analyze preference (what Harman did). It does not make the underlying data objective (preference), but it lets you draw some correlations and trends out of it.

You're right.

In my view though, it's dangerous not to take into account that research as it leaves the door wide open to the sort of crappy headphones we frequently see popping up throughout the price range. There is a world of difference in my opinion between reasonable variations around these target curves (let's say, HD650 or Beats Solo Pro or Focal Clear) and simply poorly engineered / tuned POS (Ultrasone).

Besides, there is a nugget of objectivity in the underlying preference : the Harman target was based on the assumption that people would have a preference for a FR curve that attempts to reproduce in the headphones format what speakers that measure flat in an anechoic chamber produce in a decent average room. That leaves a lot of leeway in terms of what the curves look like, but not enough to justify awful frequency response curves such as the ones from the Parrot Ziks or the B&W PX7.

Let's just say that I'm not particularly inclined to use the subjectivity argument as an excuse for this :

Screenshot 2020-08-23 at 11.55.26.png

If the explanations referenced by @MayaTlab are correct, this would basically mean that any two headphones could be transformed into each other by DSP (EQ) - unless there is a fundamental flaw with one headphone (e.g. frequency response is zero for some frequencies or unusually high distortion).

Note that I'm just referencing Oratory1990's post on reddit. I am not an audio engineer by any stretch of the imagination.

Going by his post, the laconic answer would be "if". It appears that it's a huge challenge that is nowhere near solved, particularly when taking into account variations in humans' anatomy.

Also, I'm not sure I understand this well or not, but it appears that to ensure that two headphones produce the same FR curve at listener's A eardrum, they may have to produce a different one on a particular testing system if their physical format will interact differently with listener's A anatomy relative to that particular test system. So to achieve an ideal FR curve target, we'd need to have, for a specific user, either individual measurements (with a mic in one's ear), or enough anatomical data, and have the headphones specifically tailor their FR curve to that user's anatomy (similar to personalised HRTF profiles I guess ?).
As it's impractical to ask every single user to have their personalised profile measured, or even have their head scanned in 3D, what companies are starting to do is to gather rough anatomical data and have an algorithm produce a personalised profile based on a data set of measured HRTF vs. anatomical data (neural networks for example, but there may be others, you'll find plenty of papers on the web on that subject, one video here for example : ). The big challenge being gathering sufficiently good anatomical data. Sony has started to try its hand at it with their 360 Reality Audio system : you can send them a picture of your ear and have a personalised profile made for you. It's unclear that this is enough to produce something convincing and it's very cumbersome for users.
Apple is in a very favourable position here as they already have the necessary technology to make it a lab experiment (think Face ID but for your ears), but the most detailed patents they've released so far are limited to using much rougher data using capacitance sensors inside the ear cup to simply detect left vs. right ear and the rotation of the ears within the ear cups (example here : https://pdfpiw.uspto.gov/.piw?PageNum=0&docid=10555066&IDKey=27D294743960 &HomeUrl=http://patft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1%26Sect2=HITOFF%26d=PALL%26p=1%26u=%252Fnetahtml%252FPTO%252Fsrchnum.htm%26r=1%26f=G%26l=50%26s1=10555066.PN.%26OS=PN/10555066%26RS=PN/10555066)
Some of their patents mention using other sensors, such as IR structured light ones (not too dissimilar from Face ID) which may provide higher resolution anatomical data so it's entirely possible that they're testing this in their labs as well. It's highly unclear to me that this can be turned into a shipping product soon.

This all probably means that you should take all measurements with a pinch of salt and look for tendencies and trends instead of nitpicking on very small variations.
But IMO when the departure from ideal is so severe (cf. Parrot Zik above) I believe that you can confidently claim that no amount of variation in personal preferences or variation in human anatomy can justify that (we're taking about sharp 10-20 dB peculiarly placed peaks or dips relative to Rting's target here - I doubt that there is a human being in existence whose anatomy will compensate for that. It's just bad engineering as the distortion plots suggest anyway).
 
Last edited:
Aug 23, 2020 at 6:29 AM Post #26 of 58
If you can define a target curve, you can usually EQ decent cans into sounding like that. The Harman curve isn’t a hard and fast thing, it’s an average based on a range of target curves. Frequency response is the biggest difference between models of headphones. There can also be significant differences between different copies of the same model, due to manufacturing tolerances. It’s hard to measure how headphones sound in a way that has meaning to multiple people in a specific way. People have different preferences, and it’s notoriously difficult to measure cans accurately in the first place.
 
Last edited:
Aug 23, 2020 at 6:31 AM Post #27 of 58
This sounds interesting, but I am not sure how to understand this. Do you mean:
  1. The FR of a headphone as perceived at the eardrum of a person is dependent on the individual person (because of geometry of the ear canal, placement of headphones, etc.), or
  2. Given two persons that experience the same FR of a headphone at the eardrum, they might still perceive the played sounds different in their brains (because of different sensitivities of hair cells, neural deficits, different processing in the brain, etc.).

https://www.head-fi.org/threads/hea...w-standard-part-1.937301/page-3#post-15764945
But at the same time we're still humans so as the guys at Harman demonstrated, there are big FR trends that will appeal to most of us. It's always at least an interesting starting point IMO, as chances are that what you need is closer to that target than to some arbitrarily picked headphone.

As I understand it, there are two opinions on my question "What measurement shows that a Stax 009s sounds "more detailed" than other headphones, say dynamic headphones?"




If the explanations referenced by @MayaTlab are correct, this would basically mean that any two headphones could be transformed into each other by DSP (EQ) - unless there is a fundamental flaw with one headphone (e.g. frequency response is zero for some frequencies or unusually high distortion).

If @MayaTlab was true, it would have a massive impact on the audio industry.
(Well, maybe not - given the ongoing discussions on amp and cable sound.)

I have not read/watched through all these references, yet. So, maybe I misunderstood the posts above.
I look forward to your feedback.
There is a problem with practicality. I don't think there is a doubt that FR is a major variable for the subjective impressions of a listener, and while many headphones have audible distortions somewhere, there are also many with reasonably low distos where the FR isn't just a leading subjective variable but the variable that will make you chose one or the other.
When that happens, it doesn't seem too crazy to expect that a DSP could make us prefer a headphone we would like better than with its original tuning. But as I mentioned, copying one headphone into the other one is not simple. Mainly because of what I wrote in the post I linked. If you measure a headphone on a dummy head, even if you can perfectly control the placement each time, even if the headphone responds well to EQ, even if both headphones have outstandingly low distos, you will have made a nice copy to convince the dummy head... on your head both headphones would give different responses at the eardrum, so we would need individual customisation and ideally the ability to measure at our eardrum. Not exactly an easy task on a living human. The next best thing is using binaural mics that will measure the entrance of the ear canal. Not as good, but also, most people who tried that will tell you how the slightest change in insertion and placement yield a noticeably different result. It can be really good, the Realiser A16 uses such in ear mics for its calibration, and we do get really convincing results. But for all the reasons mentioned here and in the post linked, doing it on 2 different headphones usually gives 2 noticeably different results. Both might work great, but usually there remain some audible differences. So in practice, coming close is very feasible, fooling you into mistaking both when you can do a direct comparison, that's a different story. But IMO, if we knew how to improve consistency in the measurements, the trick should work on at least a bunch of headphones.
Then there might be issues like huge planar diaphragm against small dynamic driver, heavy headphones against light one, and other things that might still make us feel different event if the sound at the eardrum is the same. A subjective experience isn't limited to one sense. It's possible that just the knowledge of using another phone with EQ would cause a significant amount of the population to just be biased against it to the point of having a bad experience no matter the sound.
 
Aug 23, 2020 at 7:32 AM Post #28 of 58
Comparison between IEMs is difficult as the distance from the transducer to the eardrum is different for each person. Same as full-size headphones will be of a different sound to each person due to head shape and outer ear and ear canal shape.

There are also psychological and neurological differences as well as fully fledged cognitive differences between listeners.

IEMs
With IEMs the actual shape of the ear canal varies too, with some tapering in, some tapering out. Some physical shape of their ear canals allows an IEM to be pushed in, and some folks have zero fitting-play. Depending on the shape of the tip, for some an IEM is only going to lodge in one specific place. Thus no way to be able to adjust frequency response due to distance to ear drum. The reality is IEM tip rolling and IEM adjustments take more time than typically provided at a retail setting.

General sound personalities can be obtained in a store, but typically there is a range of bias that is difficult to overcome. Amazingly prejudice can come from what we read, or what we have been told. The assumption of gear actually being too expensive can create emotional bias. Though typically much of our diminished objectivity is simply due to what we are used to. Everyone has either a frequency response they call correct or ideas of what must actually be wrong. If that new sound doesn’t correlate with known ideas the brain looses it’s ability to gain perception of music.

Interestingly some famous products choose to depart from known and acceptable frequency response characteristics. Meaning even outside of your own tried and true ideas to frequency response characteristics, some manufacturers put in character.

Typically on a short listen that character gets a red flag as to being wrong. When in some ways it really is wrong, yet only a subtle departure from neutral. There are the super difficult products as after an extended time, the listener could not only find the response to be correct but even preferred and ultimately exciting.

Much of the time non-correct guilty pleasures can be found with a transducer which is designed by creation to follow an unorthodox departure from the tried and true and accepted idea of audiophile.

Strangely though these fun and different creations walk a fine line of holding full world class technicality along with an artistic statement in response character. It’s in these exact times that even days at a retail demo area will never be enough. The elephant in the room with this thread is that fact that happiness is not always a science but a long and tedious learning curve from trial and error.

What’s fascinating is this becomes and active and very involving hobby with even renowned and expensive gear sounding different from day to day. There are two variables here; one the listener and one the headphone. As we know from history our ability to have objective test hearing can change or move slightly from being critical in one place.

Time and trial and error are the only way to learn. There is also the changes of upstream equipment, that while every attempt is made to restrict this variation, it’s a difficulty. File quality, amps and digital sources will ultimately add or subtract character which could either move the desired sound signature closer or farther away.

And finally there is the enigma of detail. Is it synthetic detail from a forward treble? Is it better imaging due to damping factor or a blacker background? Does the transient response create a avenue in which the listener forgets his normal set of parameters? It’s safe to say that leaning to hear detail may be a leaned ability. Still this question of detail seems to be the most perplexing concept of all. “Really we are not hearing things that always have been there?” Such concepts border on the unbelievable yet in visual circumstances are a regular affair.

Bring an accomplished cabinet maker to see a set of newly made cabinets and they will show you twenty imperfections. Are those imperfections deal breakers? Normally not, but the years of building cabinets have given the carpenter eyes to see the faults. A 16 year old dedicated to the discovery will only notice so much detail as they don’t know what to look for. Typically these inexperienced opinions are honest but lacking judgement. This judgment can be learned but it takes time. The youthful viewer sees that the cabinets work and generally look nice, but careful experienced inspection reveals the faults which are there and always will be.
 
Last edited:
Aug 23, 2020 at 9:26 AM Post #29 of 58
Thank you @castleofargh for explanation and the linked post, @MayaTlab for the further explanation, and @bigshot for the comment.

Here is the status of my understanding/beliefs:
Based on my previous headphone listening sessions and the above arguments, I think it is plausible that audible differences among normal (i.e. not faulty) headphones are determined by FR alone. The differences can be very well neutralized by EQ.
This applies to most headphones I heard e.g. Sennheiser HD 800s, Hifiman HE-1000 V2, Meze Empyrean.

The one remaining issue for me remains the "detail" I heard with the Stax SR-009s.
It seems highly unlikely to me that this is only due to a different FR.
[Here is an (admittedly subjective since no DBT) argument: I heard the detail for a range of music instruments and notes played.
If the additional "detail" was only due to a FR-push in, say, treble, the additional detail would not be heard for notes from the complimentary range, e.g. lower midrange.]

Still, I might be biased and the "detail" is only in my imagination.
 

Users who are viewing this thread

Back
Top