[Brooko]

I’ve been on Head-Fi now for almost exactly 5 years – make it 6 years if you take into account the first 12 months where I wasn’t registered, and simply spent my time reading a lot of posts. I’ve always loved music, and I couldn’t think of a time when it hasn’t played a huge part of my life – from spinning vinyl with my friends back in my teens, through to the present day where I’d easily listen to music in some form or other for 5-6 hours every day.
 
I can still remember my first days on the forum – armed with a Cowon iAudio7 (marvellous little DAP with a whole 16 Gb of available space!) a pair of Sennheiser EH250 headphones, and my pride and joy – a pair of Shure SE420 dual driver earphones.
 
Back then I was in awe of many of the regulars with multiple headphones, and discussing headphones and amps in the hundreds of dollars – if not the thousands. My aim back them was to expand my knowledge with personal audio, and hopefully find the easiest path to expanding my set-up. But I was nervous with my puny set-up getting into discussions, or even asking simple questions of some of the regulars. I now know that was a mistake. For any of the newcomers with modest gear – I started just like you, and I still remember what it was like being overawed by the regulars. But we’re here to help – and most guys I know on the forums are happy to talk about their experiences – so please don’t be shy asking for help. The only stupid question in my book – is the one you don’t ask.
 
I’m not sure if the forum software still does it – but way back then, I used to get semi-regular automated messages (either PM or email – I can’t remember) suggesting that I post a review about the gear I own. After some time, it started to get annoying, so I thought what the heck – I’ll write something so I could stop the reminders.  Funny how something small can snowball into what it has become today.  I’d just bought an Apple iPod Touch Gen 4 32 Gb with a rewards voucher – so it seemed as good a place as any to start.  So I wrote a short review on the iPod Touch Gen 4 – clearly not my best effort, but the reminders about reviewing my gear continued.
 
Frustration!  How do I get this to stop?
 
OK I’ll review some of the other gear – so next up was the Fiio E7 (my very first portable amp/dac) – and the review is quite different today than it was when I first posted it.  I’ll leave that particular story for another chapter (probably called “mistakes I’ve made”, or “the gushy noob reviewer”).
 
By the time I got to my 3^rd review (the SE425), I was starting to actually enjoy the process, and I’d finally twigged that the real secret to writing a review was to consider what I’d like to read if I was researching a product. So in came the more informative sections on accessories, build, and the inclusion of photos – and of course they began to get a little longer.
 
Fast forward almost exactly a year, and some 16 product reviews later, and I’d posted reviews for the HD600 and HM5 and my reviewing style was beginning to take shape.  I have to stress at this point though – everything I had reviewed at this stage was my own gear. And if there is one thing I need to stress to any potential new reviewer, and especially those looking to build rep to be able to review manufacturer supplied review samples eventually, start with your own gear, and develop your style (we’ll also cover that in a later chapter).
 
By now my reviews were just starting to get noticed and I had the first manufacturers approach me and ask if I’d like to listen to some of their gear and review it. So I started with HiSoundAudio’s range of IEMs and earbuds, and this eventually led me to buying and reviewing their TOTL DAP at the time - the Studio Anniversary 3. But in the meantime, I was still buying and selling my own gear, and still reviewing my own purchases along the way.
 
In late 2013 I started a relationship with DUNU (and my thanks for this relationship go to djvkool and H20fidelity for the introductions), and has led over the past three years to a long term relationship with DUNU built on feedback, mutual respect, and honest appraisal. Shortly afterwards came contacts and reviews for Brainwavz, Fiio, Fidue, T-Peos, Rockjaw, and many others.  Some were as a result of direct contact from the manufacturer.  Some were as a result of manufacturers advertising on Head-Fi looking for reviewers ( a case of being lucky enough to be chosen), and some were the result of local tours being organised by my Australian mates – which incidentally is a great way to gain experience reviewing without the outlay. With many of the reviews, I was still buying off the manufacturers (admittedly at cheaper or introductory rates), but as my style progressed and matured, more often than not the approaches were coming to me – often out of the blue.
 
I’ll finish this section with just a little insight to my position at the moment.  I deal with roughly a dozen different manufacturers at this time, some are regulars, some are relatively new, and some have dropped away. I don’t actively solicit reviews, apart from replying to threads where manufacturers are on Head-Fi actively looking for reviewers.  And I have a very set policy with the manufacturers I deal with. Anything they send me is theirs, unless I pay for it – and yes I still buy some of the gear I really like.  If I don’t buy it, they often choose to leave it with me – for follow up reviews, comparisons, assistance on the forums etc.  But it remains their property, and I’m ready to return it at any stage.  If manufacturers want to send me a review sample as a short term loaner – I’m totally fine with that too – all that I ask is that they assist with freight costs in sending it back to them.
 
Now why do I have the above policy? It is simple really.  If I’m not getting the gear for free – then it removes (or at least diminishes) a possible bias toward writing a positive review to the manufacturer that may not necessarily reflect a truly honest opinion.  And for me – the most important thing with writing a review is my own integrity. I’ll make mistakes along the way (Savant review is one I really butchered – I’ll cover that in a few chapters) – but it is important to me that people accept my reviews as being completely honest. If they don’t – then they become worthless – to me, to the manufacturer, and to the reader.
And to conclude this first chapter – here are some simple dos & don’ts for those who are interested in becoming a recognised contributing reviewer:
 

• Start with your own gear – earn your stripes – learn along the way.  Practise with reviewing does help. A manufacturer won’t touch you if you can’t be bothered to review the gear you have already.
   
• Be completely honest in your appraisal of products.  Review it for yourself as the reader.  Write what you’d like to read.  Include information that you’d like available when doing research.
   
• Be prepared to learn, and be prepared to be wrong.  I’ll cover some of the mistakes I’ve made along the way in chapters to come.  Some will surprise you.  And for some, I hope they are a revelation and a guide, and can assist not making the same stuff ups I’ve made.

 
In the next chapter – I’ll start getting a little more informative.  I’ll give you some insight into the gear I use for reviewing and why.
 
The one after I’ll look at the anatomy of my reviews – what I include and why.
 
Please feel free to leave feedback – what you like, what you don’t, and I’ll try and answer any questions you may have along the way.
 
Enjoy the music!
Paul

 

[d marc0]

This is very nice Paul! Thanks for sharing.

 

[Brooko]

Thanks Mark - a bit boring for an initial blog post, but I have in my head a series that should help those looking to take reviewing further, and with a bit of luck should be entertaining enough to give some of you a laugh at my expense 

 down the track a bit.

 

[iancraig10]

Hi Paul,

With reviews, I really get troubled over how many stars to award. Whether to take cost into mind or review it as it is.......

How good for example is a five star £200 headphone in comparison to a five star £30 one.

That's where reviews can be difficult for me. I personally take cost into consideration but it's a hard one.

Ian

 

[Hisoundfi]

Subbed

 

[twister6]

Thank you for sharing, Paul, and look forward to more chapters!!!

 

[twister6]

I'm still troubled by star rating   I feel like star rating is part of a relative comparison.  When you starting off, you don't have too many references and if this is the best thing you ever heard - it will get a high rating.  Moving on, as you test more gear, you realize other products are better and deserve a higher rating.  You can keep a list of everything you reviewed and constantly re-adjust the rating, but it's a hassle.
 
I completely relate to Paul's experience in terms of looking back at my early reviews and scratching my head.  But that's how we grow, from our mistakes, from our experience, from everything we learned and continue learning in this amazing community.
 
Also, I would like to add:
 
I absolutely agree, you have to start somewhere and your own gear is the way to go.  Or maybe going to your local audio store and taking notes while listening.  Your review collection is like a resume of your work.  Having a few behind your belt can help with a next step of having an opportunity to participate in various product tours offered on Head-fi where they usually ask you to provide a few links to your previous work.  From there you can take it as high as you want to keep yourself busy.  Also, add pictures to your review/impression.  It breaks the monotony of starring at all text and also gives another dimension to your review.  And with text, don't write super long paragraphs or avoid punctuation.  Break it down to make it easier to read, partitioned in specific topics.  And so on...  And the MOST important thing - reviews are not about getting a free sample.  It's a hard work, it's a real commitment, it's a privilege given by a manufacturer who entrust you with their product in exchange for your honest opinion.  I still get occasional PMs with people asking me "how to score a free pair of headphones"   But anyway...
 
Paul, you are off to a great start with this Blog, and looking forward to more chapters mate!
 
Alex.

 

[castleofargh]

so you got bullied into reviewing by a bot, pushing you into getting your first hit, and now look at the review junky you've become. there are places where you can get help to fight your addiction.

 
 
very curious to see how this will go, thanks for sharing.

 

[iancraig10]

Well, I've been on here for ten years now and I blame you lot for making me spend loads of money!!! You made me do it........

Damn, just done it again.

 

[jynxed]

Great idea for a thread, as someone who is new to the hobby I am looking forward to seeing how someone who has been around for a few years progressed.

I have seen a few posters here and there trying to get in to reviewing or just starting out and just saying how wonderful a product is. I sometimes wonder if it is owner bias or a hope to keep the freebees. I like to see both sides of the coin for a review to be usefull and appreciate the level of details I find on your reviews.

So yeah subbed too.

 

[iancraig10]

Another problem with reviews apart from pricing and star awarding is that different headphones for instance, do different things better. So it might be a question of 'different' but not necessarily better.

I'm not sure that positive reviews in order to keep gear is such a great influence as some might think. If you write a misleading review, then others read it but might well not take too much notice of it your reviews at a later date. So you have no clout any longer as a result perhaps.

I have certain reviewers that I read more than others, basically because I've compared what I hear with what they say, so over my ten years, I've started to pick and choose who I trust more than others.

Each reviewer also might have quite different preferences for sound and neither is 'correct' really. Reviewing is quite difficult in that you have to be fair to the maker as well as critical for the reader.

Also, human error and mood come into play. All kinds of things.

Also, some guys get so damned heated about it. I've never forgot a fella who was mad keen on the dt880 when it came out. I had one and said that I felt that it was a bit bright for me. The insults that came back were awful and we fell out big time. He refused to speak to me..... Over a comment!! Funny thing that he was driving it with one of Norman's Go Vibe amps which also tended to be a tad shimmery in the top, so the combo (which I tried at home) just didn't work for me.

With your reviews Paul, I really like the subjective stuff, followed by measurements as well. They don't always tally and I think the sensible position is in the centre of the two edges of beliefs about hi fi.

Some interesting thoughts here though.

 

[twister6]

^ exactly, your reputation as a reviewer will crumble when people buy the product based on your recommendation and it sounds opposite of what they expected.  Also, when you are working on too many reviews at once, the quality will go down because it's hard to keep up.  I learned that the hard way...  Going through my own 12-step program now lol!!!

 

[Brooko]

Thanks gents - and I really appreciate the feedback.  I'm hoping that with the discussions on some of these blog posts, I'll learn a lot and ultimately it'll help myself (and others) become a better reviewer.
 
Sorry for the long post coming back - but I'll try and answer the main thoughts:
 
 

With reviews, I really get troubled over how many stars to award. Whether to take cost into mind or review it as it is.......
How good for example is a five star £200 headphone in comparison to a five star £30 one.
That's where reviews can be difficult for me. I personally take cost into consideration but it's a hard one.

 
Really good question Ian - and as long as you can clearly state your reasons as to why you have ranked them - and the ranking is consistent with your other reviews - then IMO you are doing OK. Ranking will always be subjective, and should always be taken into account along with your stated bias.  I always take  value into account - personally I can't see how you can rank without it.  And I usually look at things (for Head-Fi rating) in terms of % - 3/5 stars = 60%, 3.5/5 = 70% etc
 
Classic example is my T3 review vs the T5.  I rated T5 as 5 star (performance to price), and T3 as 4 star (performance to price).  I've already have Avitron ask my why (and given me advice 

) based on his own views.  I'm comfortable with how I've ranked it.  For my stated bias, I'm very comfortable with my scoring.  I see the bass as lacking rather than balanced.  Others have different opinions - and that is cool. As long as each of us provides reasons for our scoring then it is up to the reader to draw conclusions based on the information in the review.  I think most people over time will tend to follow a few reviewers more than others - simply because their opinions gel with our own.  I know I do.
 
 

  I'm still troubled by star rating   I feel like star rating is part of a relative comparison.  When you starting off, you don't have too many references and if this is the best thing you ever heard - it will get a high rating.  Moving on, as you test more gear, you realize other products are better and deserve a higher rating.  You can keep a list of everything you reviewed and constantly re-adjust the rating, but it's a hassle.
 
I completely relate to Paul's experience in terms of looking back at my early reviews and scratching my head.  But that's how we grow, from our mistakes, from our experience, from everything we learned and continue learning in this amazing community.
 
Also, I would like to add:
 
I absolutely agree, you have to start somewhere and your own gear is the way to go.  Or maybe going to your local audio store and taking notes while listening.  Your review collection is like a resume of your work.  Having a few behind your belt can help with a next step of having an opportunity to participate in various product tours offered on Head-fi where they usually ask you to provide a few links to your previous work.  From there you can take it as high as you want to keep yourself busy.  Also, add pictures to your review/impression.  It breaks the monotony of starring at all text and also gives another dimension to your review.  And with text, don't write super long paragraphs or avoid punctuation.  Break it down to make it easier to read, partitioned in specific topics.  And so on...  And the MOST important thing - reviews are not about getting a free sample.  It's a hard work, it's a real commitment, it's a privilege given by a manufacturer who entrust you with their product in exchange for your honest opinion.  I still get occasional PMs with people asking me "how to score a free pair of headphones"  

 
Thanks Alex - although we often have different overall views on how a product sounds relative to our own preferences - because I know yours, I can generally relate to your reviews easily.  And the consistency of your reviews make them great because I know what to expect.  Totally agree on the experience thing too - the refinements to style and accuracy come over time. One of the upcoming chapters in the blog will be "anatomy of a review" - or basically a breakdown on what I do and why. I'd be very appreciative if you could contribute your own methodology when we get to that one.
 
Quote:

  so you got bullied into reviewing by a bot, pushing you into getting your first hit, and now look at the review junky you've become. there are places where you can get help to fight your addiction.

 
very curious to see how this will go, thanks for sharing.

 
Guilty as charged!
 
 

Great idea for a thread, as someone who is new to the hobby I am looking forward to seeing how someone who has been around for a few years progressed.

I have seen a few posters here and there trying to get in to reviewing or just starting out and just saying how wonderful a product is. I sometimes wonder if it is owner bias or a hope to keep the freebees. I like to see both sides of the coin for a review to be usefull and appreciate the level of details I find on your reviews.

So yeah subbed too.

 
And that is one of the big dangers once you start getting sent a lot of gear.  I'll cover that in later chapters too - but I'd agree that maintaining objectivity and stopping manufacturer bias is one of the biggest issues for any reviewer.  I'd like to think I am a lot better now - but I would readily admit that in my early days I was probably clouded by wanting to "please" manufacturers with a positive review.  Nowadays - my personal rep as an honest reviewer is far more important than any individual relationship with a company.  But it took me a while to get there.  Thanks for the input - and that will definitely be a topic for another future chapter.'
 
 

Another problem with reviews apart from pricing and star awarding is that different headphones for instance, do different things better. So it might be a question of 'different' but not necessarily better.

I'm not sure that positive reviews in order to keep gear is such a great influence as some might think. If you write a misleading review, then others read it but might well not take too much notice of it your reviews at a later date. So you have no clout any longer as a result perhaps.

I have certain reviewers that I read more than others, basically because I've compared what I hear with what they say, so over my ten years, I've started to pick and choose who I trust more than others.

Each reviewer also might have quite different preferences for sound and neither is 'correct' really. Reviewing is quite difficult in that you have to be fair to the maker as well as critical for the reader.

Also, human error and mood come into play. All kinds of things.

Also, some guys get so damned heated about it. I've never forgot a fella who was mad keen on the dt880 when it came out. I had one and said that I felt that it was a bit bright for me. The insults that came back were awful and we fell out big time. He refused to speak to me..... Over a comment!! Funny thing that he was driving it with one of Norman's Go Vibe amps which also tended to be a tad shimmery in the top, so the combo (which I tried at home) just didn't work for me.

With your reviews Paul, I really like the subjective stuff, followed by measurements as well. They don't always tally and I think the sensible position is in the centre of the two edges of beliefs about hi fi.

 
Another excellent observance  - a lot of which I've covered already above.  I totally agree with you on mixing the objective with the subjective.  One without the other will slant things.  Relyi9ng on one too much can slant things.  That will definitely be covered when I discuss the traps, and highlight my own mistakes.  Thanks Ian - good insights.

 

[62ohm]

I can't imagine how you coped with the myriads of gear you're sent to review simultaneously. As you know, I received 4 IEMs from CustomArt and a FiiO DAP not too long ago from review tours, and having 5 of them at once overwhelms me already.

You, on the other hand, must be used having more than 5 products to review simultaneously

 

[Hisoundfi]

I noticed you guys were discussing star ratings earlier. I'd like to add my own two cents on it.
 
I personally feel the star rating is crap. Let me explain...
 
WE ALL suffer from new toy syndrome from time to time, and we all have different ears. There are instances where I've gotten something in and thought it was the greatest thing ever. A year later I sit down and listen to it and think it's not very good at all. There are things I've reviewed and love, while others who's opinion I highly respect have felt the opposite. The star ranking is a crap shoot, and should be seen as nothing more than an indicator of whether the reviewer likes it based on their preference. That goes for every review that is on Head-Fi for the most part. 
 
If I really like something, and feel that they perform beyond their asking price it should get five stars. If we think it's good, why not make sure we tell everyone it is? I think all too often people make their purchase based just on how many stars it gets. Truth be told, purchases should be made based on understanding their own preferences, then looking for a product that matches them. 
 
Star rating opens up a can of worms that consists of many variables, all of which can't be factored into a single star rating. If we are going to do stars, there should be a star ranking based on several separate criteria, and the final star ranking should be a mathematical average of all of them. Still, if it's done this way it's factoring every criteria equally. That isn't fair to people who hold particular criteria in higher regard than others. 
 
Another thing, I don't get how some reviewers feel so inclined to be super critical with their star ranking like they're holding out for the mothership of earphones to beam them of into an alternative dimension of music we have yet to experience in order to give it five stars. Let's not be arrogant in how we rank things, but more informative to the community. Three stars is good, four stars is better, and five stars is best, knowhatamsayin?
 
The way I see it, for me and what I write, I want to keep things this simple:
 
5 stars = An awesome product that I highly recommend, arguably the best at its price range. Very few if any customers will be disappointed if they read my review and feel that the product matches their preference and purchases it.
 
4.5 stars = A really good product that performs very well, but I can think of a couple products that might possibly be a better option at that price range.
 
4 stars = A solid piece of gear that has some minor flaws that prevent it from being elite. 
 
3.5 stars = A middle of the pack performer that doesn't match what the better products in it's price range can do. Not the worst but definitely not the best.
 
3 stars = A middle of the pack performer for it's asking price. Not a total waste of money, but they could have probably spent their money on something else and had better results.
 
2.5 stars = This is the lowest rating I will probably give a product before I tell a manufacturer that I can't review it. This is a product that is on the lower half of what I would consider satisfactory for it's price. There are many things to improve on, yet it still has some positives to point out.
 
Anything below 2.5 stars, I am offering to ship it back at my own expense. I don't want a company thinking I'm bailing on a review. I also offer constructive criticism of why I feel this way as well in hopes that they will use it to improve future products.
 
The point of reviewing is helping customers maximize their dollars, and without degrading manufacturers.