Let's collect and discuss some evidence that show how subjective, sighted and (therefore) usually biased reviews fail.
Note: This thread is not about intolerance or ridicule towards subjectivity per se. See #51.
The following is in no particular order. The reviews are chosen completely randomly.
Sighted listening tests
http://seanolive.blogspot.co.at/2009/04/dishonesty-of-sighted-audio-product.html ("Audio Musings by Sean Olive: The Dishonesty of Sighted Listening Tests")
Bit-perfect audiophile audio players (software)
most "player A sounds better than B" posts in computer-audio
(Software induced) Jitter, audibility
see some of the audiophile audio player reviews or product pages
http://www.head-fi.org/t/629729/xonar-d1-measurements (see loaded measurement results)
... discusses how the Wadia 121 24-bit DAC (MSRP $1299) received very positive reviews, but Stereophile's measurements concluded only AES/EBU works "correctly" achieving about 17 - 18 bits of performance, S/PDIF 24-bit data was truncated to 16 bits and USB measurements showed a noise floor around the 15-bit level.
http://www.head-fi.org/products/earsonics-sm64 (1 review, 4.5/5 stars)
http://rinchoi.blogspot.co.at/2013/05/earsonics-sm64-electroacoustic.html ("An electroacoustic abomination born")
Heir Audio 3.Ai
http://www.head-fi.org/products/heir-audio-3-ai (3 reviews, 4.67/5 stars)
http://rinchoi.blogspot.co.at/2012/10/heir-audio-3ai.html (huge dip from 2 to 4 kHz, bad channel balance)
Heir Audio 4.Ai
http://www.head-fi.org/products/heir-audio-4-ai (9 reviews, 4.72/5 stars)
http://rinchoi.blogspot.co.at/2012/10/heir-audio-4ai.html (broader dip than the 3.ai, better matching in the lows though)
Beyerdynamic DT48 E (25 Ohm)
(maybe other versions too)
http://www.head-fi.org/products/beyerdynamic-dt-48-e-25-ohm (4 reviews, 4.62/5 stars)
Ultrasone Edition 10
http://www.head-fi.org/products/ultrasone-edition-10 (4 reviews, 4.5/5 stars)
Shure SRH1840 (and SRH1440)
http://www.head-fi.org/products/shure-srh1840-professional-open-back-headphones-black (3 reviews, 4.5/5 stars)
... more to come ...
Edited by xnor - 10/9/13 at 9:18am