Outstanding article Jason. One of my favorite to date. And also, as usual, one that i would assume would scare and/or be counter intuitive to someone like yourself who owns a company that could be negatively affected by such a proposal. Kudos sir for seeing the greater good in your musings!
The concept of reviewer bias is one ive thought about constantly since getting into higher end audio gear. Its very real. Its very prevalent. And its darn near impossible to avoid...with one and only one exception....blind listening.
Here's just a few of the ways i think our community can be 'reviewer biased':
- Trusted Reviews. This is a double edge sword. I find trusted reviews (ie: tyll's wall of fame, sniff, r.i.p..) to be invaluable in helping guide solid purchases HOWEVER they also give us a perception that the gear is good because someone told us it is.
- Misc Reviews: question for y'all...how often to you buy some gear and THEN go out and read a bunch of random review sites about the gear...especially looking for good reviews? If you are like me, you do this alot. At then end of the day i think its because we want validation that we are right when we say something is good OR we want to the warm fuzzies that come from saying "A HA! Others like this too!"
- Personal Reviews: no disrespect to head-fi, but getting truly objective reviews from individual personal reviewers reviewing their own gear seems tricky to me. Namely in the fact that if we spend alot of money on gear that we like, we're bound to review it well. Thats not to say that the individual is saying something untrue..far from it...but truly objective non biased reviews are tough from dedicated reviewers....and way tougher from individuals owning specific gear. (also there's the factor of our own ability to compare is limited by what we have access too).
- Cost Bias: This ones huge. As we invest more and move up higher in a companies product line, the assumption is that we are gettting better sounding gear. And alot of the time its true. But its also not always clearly obvious. Nor is it also linear. Just as Jason mentioned...moving from a $100 DAC to a $1000 DAC may...MAY...improve sound quality but it also may...MAY..do so marginallly.
- Audiophile Collective Hive Mind: no disrespect to our community, but there does seem to be some general stereotypes out there. Before jumping on me, let me give you an example. What do you think about the recently released Beats model XXXX headphones? Is your knee jerk reaction that Beats do not make audiophile headphones? Im not saying that reaction may or may not be warranted by personal general overall experience with their products, but lets just pretend that they literally create one of the greatest sounding headphones ever. How many people would never give them a listen purely based on the audiophile collective history of bashing their cans?
- Audiophile Collective Hive Mind part 2: Lazy Blogging: Ive noticed another trend. Some headphones that get picked by a few trusted sites as 'best in class' end up showing up on many many other sites 'best of' lists. Example...if i ask you what the best noise cancelling headphones are out there to buy today. Ill be you dollars to donuts that a lot...a WHOLE lot...of you thought of the Sony WH-1000XM3s. Its certainly a great pick....i happen to dig them alot. But i also notice that they show up on countless sites, many of which i dont really think have a focus on gear analysis (my own opinion) and that creates an overall larger-than-reality opinion of them. Again, it may be warrented, but it does impose bias.
So...in the end...after all my rambling....what is the one and only way to remove every single bias above? Blind listening. Its the one way to find out what sounds the best to each individual AND help prove/disprove our own opinions of gear and variances between specific elements of it. Thanks Jason!!!
Peace n Living in Stereo
3ToF