For these reasons stated, I've always found that the introduction of signature, into an IEM ranking, is a flawed system due to the variances between perception of what constitutes an enjoyable signature, what is a sufficient bass, treble and so on. Objectivity, needs to remove personal preferences in a ranking system. For example, the SE5U has really amazing authority in its bass and its decay, but that in turn drowns out its technical performance because all details are masked behind that bass presence, the same way the vega masks details when compared to the dream. If a person likes overwhelming bass, I wouldn't be surprised that he would rate the SE 5U highly, because the glaring flaws of the IEMs simply dont matter much. For for a person with a much more balanced POV, the SE5U (compared to modern offerings) is going to repulse with its glaring muddiness and lack of a treble presence.
As such, the only objective way to rank IEMS, should purely be on a technical performance level. We know that we each perceive colours (and sound) differently, and hence we may disagree on what constitutes sufficient or insufficient treble/mids/bass and so on. However, what we can all agree on, is how well an IEM is, to use an analogy, able to present and contrast the different colours, or even shades within a single colour spectrum.
Sounding realistic, how musical or enjoyable or not, are irrelevant factors, because that differs from person to person, and hence a subjective reference point cannot by its definition be used in an objective ranking without compromising the very premise of the ranking. To use an analogy, an artist that paints all the leaves of a tree in a single monotone that is accurate to the general shade of the tree, is inferior to another artist that is able to accurately portray the difference in shading between every leaf in that tree, but may however have consistently used a shade of colouring that is way too dark or light for the entire tree. The former artist is realistic, but technically inferior to the later artist. You can extrapolate the sound signature of a song that is off pitch, but you cant extrapolate data that wasn't there to begin with.
So, to bite the bullet and to state an extreme example, a higher ranking IEM should be the most revealing IEM, even if it sports a ridiculously boring signature, than one that might be perceived to be true to life, and fun but has obscured details. A high ranking IEM, should allow you to, figuratively, read every part the music you hear, even if you dont actually get to see the music for what it is.
In my opinion, it should be a healthy balance of both; technical performance is pointless without naturality, and an organic signature needs technical prowess to back it up. Although compromises are inevitable, higher points (again, IMO) should be awarded to those that can portray what sounds, for lack of a better word, "right" with the least amount of compromises possible.
Although deviations of tuning surely exist, engineers who can pull off said deviations without sacrificing other aspects of the sound should be rewarded, and the same applies to engineers who can push details, speed, PRaT, imaging, etc. out the wazoo, but still maintain a feeling of reality and naturality to the music.
Similarly, in music, what's the point in listening to a pristine recording of a piano, if it's the same note played over and over again for 6 minutes? And what's the point in listening to a brillianty arranged and painstakingly rehearsed 200-piece orchestra if it's recorded through a mobile phone? Alluding to your analogy as well, what's the point in seeing all the details if none of it looks real? And what's the point of getting all of the colours perfect to the human eye, if it's all a blurry mess?
If I were to review these IEMs myself, the more accurate the IEM could replicate what instruments sound like in real life (or how faithfully the instrument was captured via mic, mixed, mastered etc.), the higher the IEM ranks in terms of tonality. Although many don't share that opinion, @flinkenick does, as he explained in his preamble preceding his review(s). I'm not saying this makes my opinion more valid than anyone else's, what I'm saying is judging tonality is easier to standardize once you understand the reviewer's bias (as was mentioned a few posts ago), and it then becomes an integral part of the IEM's evaluation.
If, say, your personal views on how an IEM/instruments should sound do not align with the reviewer's, then it is important to read their description of the IEM's signature, and see which sounds like the one for you. A score should never be taken at face value, but the relevance of a score should never be undermined either. Every facet of Nick's scoring is of equal weight IMO, and I thank his efforts in making them as clear and level as possible; each taking part in painting the mental picture of an IEM's sound that, to many, is as good as it gets without a direct physical demo.
At the end of the day, we listen to music with our gear; not the other way around
Last edited: