Interesting thread - and I can see both sides. I do think it important to realise that the OR (original reviewer - referred to by ab Initio), whilst being a member far longer than I have, has only posted his reviews (5) all in the last week or so. All 5 stars - so that would probably raise alarm bells for me - and prompt me toward further research into his experience if I was researching a product. YMMV. I've been following Jason and Schiit for a long time, and I know at some stage I want to try a combo of their gear (Bifrost, Asgard 2, and Valhalla). That will be a while away though - perfectly happy with my current set-up for the time being.
So Id' guess the OR may not be very experienced in reviewing - and real appreciation of what it takes to write a good review only comes with practise and time IMO. My thanks to ab initio for using one of my recent ones as an idea of what a review can be / should be - I do spend a lot of time putting them together - but I should also point out how bad I was originally when I first started. My issues as a beginner were over enthusiasm, not knowing the scientific process, and relying purely on flawed subjective indicators - rather than trying to have a good mix of objectivity and subjectivity.
Don't get me wrong though - any review here relies on the subjective. But we can make that subjectivity a lot more understandable and relatable by adopting a few simple rules with reviewing.
- State what we know about ourselves as a reviewer - eg age (can relate to hearing ability + also the enthusiasm factor), genre preferences, past and present gear, what any known preferences are (I am not treble sensitive - some are). By stating them - we allow the reader to form a baseline of the reviewers personal bias.
- State any known bias - and why. Those that know me will understand that I have performed abx tests many times on different music formats (lossy vs lossless), and I know exactly where my threshold to transparency is (I can't distinguish aac256 from lossless if transcoded properly from the same source). I also don't believe in massive audible changes from burn-in of headphones (speakers are different) - so I state it.
- When doing direct comparisons - always volume match using an SPL meter - and if you can't get the matching correct, state it.
- Listen first - form an opinion, but also where possible try to obtain graphs etc so that you can learn, see , maybe even explain what you''re hearing and why.
- Spend time and try to cover as many genres as possible so you can account for a lot of differing tastes.
- And probably most important - I try to review in a manner I would personally like to read if I was researching a product for possible purchase. Limit the enthusiasm. Point out both good points and faults. Try to be fair.
To give you and idea of how bad I was - here was my review of the E7 3 years ago - http://www.head-fi.org/products/fiio-e7-usb-dac-and-portable-headphone-amplifier/reviews/4826 (it was actually worse than this originally, I must have tidied it up a little later).
* subjective
* no real hints at what my personal bias was
* no real experience with a lot of gear to be able to form a meaningful opinion
* no volume matching
etc.
So I think we should give the OR a bit of a break. Hopefully he'll see the critique, and modify his reviewing style over time. We all have to start from somewhere.
Lastly -
I do really like the fact that Head-Fi is regularly adding front page content, and I do like it when my reviews are chosen. But it's not my call - nor do I email anyone to let them know I've written one. That is the Admins here only. Their call, and I respect that. If this thread does serve to encourage all reviewers to lift their game though - that would also be beneficial IMO. I am very aware that buying decisions can be made on what I've written. It;s a responsibility no-one should take lightly.
Cheers
Paul