Reading reviews with a bag of salt?
May 5, 2010 at 1:41 AM Thread Starter Post #1 of 56

khaos974

Headphoneus Supremus
Joined
Mar 19, 2008
Posts
2,085
Likes
117
As audiophiles, or at least people interested in audio equipment, each of us have attempted to objectively judge gear. It is inevitable that we managed to find our own conclusion about what works and what does not.

For example, some of us found out that expansive cables can bring tremendous improvements, others thinks that the 2-5% improvement they bring is not worth the price, finally some have attempted in vain to recognize cable in DBT condition and failed repeatedly, concluding that they don't change the sound at all. The same could be said with HiRez vs 16/44.1, different dacs, different amps, ...

Now, assuming that reviewers are honest, if a reviewer who has shown themselves as hearing differently from you (they hear differences where you do not, or the reverse) reviews a headphone (where the most skeptical of us acknowledge there are differences between model), how much do you trust the review?

There are two cases, one in which the reviewer hears differences you don't hear (due to better hearing or placebo) and one in which the reviewer does not hear the differences you hear. So tick two choices or one if you always hear differences in the cases above.

PS: this is assuming that while your ears are the best judge, you trust to some extent other reviewers.
 
May 5, 2010 at 2:23 AM Post #2 of 56

Czilla9000

10 Year Member. Still no custom title.
Joined
Feb 26, 2002
Posts
2,238
Likes
12
I assume this thread will generate copious replies, so I'll be short:

In general I think we tend to exaggerate differences in equipment from what we're used to. When something is tad bit better than what where used to, we exaggerate it as huge. Similarly, when something is a bit worse than what we're used to, we exaggerate it as being worse than it is.

Part of the reason for this hyperbolic tendency is that during reviews you are intentionally screening for deviations from the norm...deviations you probably wouldn't noticed if you were simply enjoying the music.

Most of us, I think, have experienced the phenomenom of buying a phone we think is great, writing great things about it, only to grow bored with it later on. We get used to sound, it becomes our new reference, so if no longer sees as phenomenomal. Then when you try your previous favorite phone, it no longer sounds that great in comparison. You wonder how you could have ever liked it.

Similarly, if you have bought a phone that you think you hate, you may find yourself growing used to the sound and no longer having as ill thoughts about it after getting used to it. Then when you hear a phone you previously liked, that phone will sound really really fantasic, untill you grow used to it.

What I'm trying to get at is that we notice deviations from what we're used to. This extrasensitivity to small deviations causes us to unfairly praise or pan certain headphones.

This is where I get opinionated: I've been a member here for over 8 years. I've kept silent on a certain opinion which I will now reveal. In general I think HeadFi members spend too much on equipment, chasing ever smaller and smaller deviations from the norm. Of course you grow used to the deviations, so you buy more and more equipment to get your next fix.

I own a Stax system and a JVC DX1000. They now gather dust (at least metaphorically, I keep them clean). My day to day headphone is the Sennheiser HD20, which I bought for $20 at B&H. It fits my lifestyle better, and it gives me all the enjoyment of my more expensive phones. This wasn't my first downgrade - I first downgraded from the Stax to the $169 Ultrasone DJ1s. Again, I didn't really miss anything. In general, I think $20 is all you really need. Ya, the HD20 isn't perfect, but unless you are constantly listening to better phones you won't miss anything in the long run. And better phones will still sound special to you when you occasionally come across them.

I think HeadFi does new members a disservice by encouraging them to buy gear they don't really need to enjoy there music on a day to day basis. Really folks, a $20 pair of HD20s is all you need. If you want to buy more expensive gear because you love the hobby, or love the novelty, then sure buy it...but don't make it sound to newbies that they can't hear their music "correctly" without an amp or more expensive gear. The difference between a pair of stock headphones and an HD20 is 10,000x greater than the difference between an HD20 and an HD600 with proper amp. Besides, you'll simply grow used to the HD600 and a proper amp and it will cease to be special.

Which reminds me, I really need to review the HD20s....
 
May 5, 2010 at 2:42 AM Post #3 of 56

khaos974

Headphoneus Supremus
Joined
Mar 19, 2008
Posts
2,085
Likes
117
While you are getting hyperbolic with the 10000x difference, I couldn't agree more, unless it's a case of overload, clipping or simply bad design, any 20/30€ headphone out of a standard jack (no amp) is far enough to enjoy music.

On the other hand, that's why audiophilia (is that a word?) is a hobby by itself.
 
May 5, 2010 at 5:14 AM Post #5 of 56
Joined
Jan 4, 2008
Posts
17,478
Likes
11,818
Location
Fukuoka, Japan
The last time I reviewed a pair of headphones, as it was, I specifically wrote it in a manner where I attempted to compensate for factors such as musical taste and the effect of the gear on the performance of them as I knew that, while I did love the headphones, not all people will, nor will they experience exactly what I did. I think though that it's taken me quite a lot of experience since I joined to get to the point I felt I had a good enough understanding of what I was hearing and was caused it to be able to write that though. I'm always discovering new aspects to what I know, or thought I knew.
 
May 5, 2010 at 5:24 AM Post #6 of 56

d.g

100+ Head-Fier
Joined
Mar 7, 2010
Posts
101
Likes
11
Quote:

Originally Posted by Czilla9000 /img/forum/go_quote.gif
Really folks, a $20 pair of HD20s is all you need..... .
......Besides, you'll simply grow used to the HD600 and a proper amp and it will cease to be special.

Which reminds me, I really need to review the HD20s....



All you need in the same way that a small basic low powered car is all you need to get from a-b too...but life isn't that straightforward is it.

I disagree that decent phones and amp will cease to be special, whilst that might be the case for you, I come across songs each time I fire my system up that make me smile and remember why I seek better equipment and hence better sound.
 
May 5, 2010 at 5:33 AM Post #7 of 56

Landis

Headphoneus Supremus
Joined
Apr 8, 2009
Posts
1,656
Likes
24
Location
Toronto, ON
This thread has two long posts in it.

This has been an honest Landis thread review.


A lot of trusting reviewers, in my opinion, has been trial and error. Thankfully I decided to put my trust in a few stand up HeadFiers like Headphone Addict and HiFlight when I first joined and they helped me out quite a bit and I found their reviews to match my own impressions. On the other hand, I've read reviews and felt very different about the product, but one has to always consider differences in hearing, slight equipment variations and other x-factors at play. If you can find a few people who you can trust on here and feel that they have similar tastes as you do, then that really is your best option... Thankfully guys like Larry (HPA) have a crap-ton of gear.
 
May 5, 2010 at 5:34 AM Post #8 of 56

gknix

500+ Head-Fier
Joined
Jan 27, 2010
Posts
659
Likes
15
I have to agree with a lot of what has been said so far.

I think it also depends on the environment/scenario of how you listen to your music. I think many of us here listen to music while working/studying/and among other things. In this case, I think these subtle differences really don't matter as much. If we are preoccupied with something else and we use music as a source of creating a different atmosphere/environment, different gears provide minimal difference especially if it wasnt used as its primary purpose.

I do, however, believe that all these gear do have its differences if we do in fact sit down, enjoy, and listen to music as a primary activity. This is when we do our A/B tests etc. I think by involving music as its primary activity, some of us do feel that all these expensive gear is necessary. Not to mention, everyone's ears are different. We simply can't compare whether one's ears are better because we can't A/B ears. The preception of sound is unique to everyone, and I feel everyone is entitled to opinion, even if exaggerations are involved.

A part of being on head-fi, in my opinion, is discovering new amps/dacs/headphones and reading about the exageratted differences. But again, for reviews, in order to emphasize the changes, we would have to underscore changes that we hear. This is the nature of human beings to better understand differences, and to accept the alterations as a certainty. Thereby, logging on to head-fi everyday and realizing that you find new headphones that you want to try induces our brain to spend money to halt a stop to our curiosity. Only do we know that the premium we pay have huge diminishing returns. I think all this is part of a hobby, constantly changing gear and feeling to shed the old for a new swift feeling of owning something new and perhaps luxirious/exclusive (compared to most people earbuds/headphones in the mass consumer market). This is the similar notion that is described above by Czilla.

I do not, however, disagree that large sonic differences exist. Whether we're willing to pay an extremely large premium for a much less return in better SQ is what matters. Reviewers are doing their job, and describe the differences in what they hear, and hence exaggerations occur. But I think sometimes people do not take the exaggeration in context and get caught up in the moment when they start reading multiple opinions and reviews from other fellows on head-fi.

Though I have not been in here for long, I have learned quite a lot so far. It's been an interesting ride, but refraining oneself from purchasing and trying out new rigs can be quite tough, particularly when some reviews are very well written.

Again, take my thoughts with a grain of salt, but I just thought I would comment on what I'm thinking at the moment.
 
May 5, 2010 at 6:17 AM Post #9 of 56

jax

Didn't do the flip
Joined
Jan 19, 2005
Posts
1,555
Likes
83
Trust a review??? Reviews, no matter by whom, or with how much experience behind them, are at best a point of reference which may become a point of departure. This hobby is highly subjective...objectivity has little to do with it in my opinion. The idea of an absolute sound, or of fidelity to some finite set of numbers that you can graph and shake your slide-rule at while expounding on how right you are because these scientific representations of truthfulness have validated your attachment to the objective...well, this just has nothing at all to do with the enjoyment of music anymore. Music is emotions... heart and soul... it is primal...it is not cold and detached and objective. The idea of an absolute is the proverbial carrot on a stick attached to your head. Which X is the best X? Do wires make a difference and how can I save the world from selling their souls to the wire salesman? Headfi is making me poor by convincing me I need to spend a months salary on these headphones, and that I've got to have a different pair of headphones for every day of the week, and lets not forget the IEM's for special occasions and holidays. C'mon...give me a break!! So reviewer Bob likes Grado's while reviewer Syd hates Grados. Are Grados therefore bad or good? Do you trust Bob or Syd? Bob likes Pink Floyd, so I trust Bob. Wait, Syd's with that magazine, so did he get a kickback? Is Bob a jerk? What about Syd...isn't he getting kind of old to be reviewing headphones? Who wants gum?

For reviews there is only one person I really, consistently rely upon. I suggest you do the same: Use your own ears, your own music, your own system (and in your own room if you are using speakers).

Rant over.
 
May 9, 2010 at 3:11 AM Post #11 of 56

krmathis

Head-Fi's Most Prolific Poster
Joined
Jan 13, 2004
Posts
34,761
Likes
76
Indeed!
In the end only trust your own ears.
 
May 9, 2010 at 3:14 AM Post #12 of 56

Mad Max

Headphoneus Supremus
Joined
Nov 12, 2009
Posts
7,556
Likes
337
Location
USA
Who said other reviewers might not try to troll you?
 
May 9, 2010 at 2:49 PM Post #14 of 56

Mad Max

Headphoneus Supremus
Joined
Nov 12, 2009
Posts
7,556
Likes
337
Location
USA
LOL, I know this isn't 4chan, but I do not feel it is sufficiently free enough of trolls.
 
Seriously though - yes, there are some vast differences in opinions, but some people purposely overblow it. Then there are more contrarians than I initially suspected.
 
May 9, 2010 at 3:18 PM Post #15 of 56

Prog Rock Man

Headphoneus Supremus
Joined
Jul 2, 2009
Posts
3,806
Likes
163
I have changed my view on hifi a lot recently. The purchase of £11 headphones that sound just as good (but different) as my £180 ones and making my own cables has seen to that. 
 
I now think burn in, reports of cables sounding different etc are down to mood, time of day, placebo and we have different ears and brains. I no longer believe in the highly unsatisfactory pseudo science that many spout. We buy our kit depending on budget and looks and want reviews to act as some sort of prop. Hence we side with the favourable reviews and disagree with or ignore the unfavourable ones.
 
Those who claim obvious and night and day differences should try ABX testing. My reading on that subject strongly suggests that many reviews are deeply flawed, since if the reviewer was blind to the items being tested, they would not be able to tell the difference.
 

Users who are viewing this thread

Top