Quote:
I really can't rate things based on technical measurements - for many reasons, but partly because I don't have the equipment available to me and partly because I wouldn't need to be in the equation at all if all that mattered was the response curve and THD. An algorithm could take care of all the "reviewing" if that were the case.
I think I can get the read on a signature pretty well at this point but the overall score, at least approximately, comes from the overall feel a phone gives as compared to one of my reference sets. I take my time with reviews to ward off any 'new toy syndrome' and typically go back after a while and see if my residual impressions of a phone match the score given. I don't claim to be 100% consistent and if I had to do it over again certain things would probably be done differently.
Oh, but an algorithm can rate IEMs based on frequency response and e.g. impulse response. (using waterfall graphs)
You'd have to put some weighting to the frequency response differences from 0 dB line - and this is a matter of taste, as well as rate various kinds of ringing.
It's pretty easy to classify bright/dark spectrum and describe the "kind" of brightness from the frequency response once you know what you're looking for.
Examples: "bell-like" peak at 4 kHz (Hifiman RE-2xx series and ATH-CK10), hot 6.5 kHz peak (GR07 - combined with resonance? - and lesser Sennheiser IE7), sparkly peak at 10-11kHz (ER-4S/P, possibly FI-BA-SS); forward or recessed sound due to 2kHz or 3kHz emphasis/cut, boxy sound due to high bass emphasis (200-300 Hz, common with low end IEMs).
Soundstaging is in fact directly related to the frequency response as well, esp. the highest registers, however this range is highly personal, so neither review nor measurement will tell the whole story, although I find that Joker's reviews match my own experiences 1:1.
Also, ergonomics are pretty much make or break of IEMs. It doesn't matter really if it sounds perfect if the cable breaks after a few months.. (RE272 - they should ban silicone cable sheaths) or if you can't actually fit it in the ears without causing pain. (e.g. Monoprice 9320 or Radius DDM for some.) If the IEM loses seal every 30s, that's also pretty bad, but possibly fixable.
It might be also useful to rate the impedance to give a recommendation like "needs an amp", "needs low impedance output" and perhaps measure the frequency response with various sources, simulating a portable player, a portable headphone amplifier and one a desktop amplifier.
--
The only people I know of who can actually rate IEMs and headphones based on measurements right now are Tyll of InnerFidelity/Headroom fame and especially people running Golden Ears site.
Both actually provide subjective reviews as well.
GE does so in a "dry" numeric form (maybe the text reviews are in Korean?), while Tyll has long winded reviews of the Stereophile kind with added measurements.
Joker's occupy a happy medium, but the "quality" thing is quite open to interpretation (it's the "like/dislike" scale) - however he does grade various ergonomic features accurately and in a numeric form too.