[Brooko]

So far in this little journey, I’ve looked at how I started (reviewing), the equipment I use, and the methodology / time-line.  The next logical step is to look at the different parts of a review, why I include them, and what I think is important.
 
As in the past few chapters, you may not agree with the content I suggest.  Some of you may in fact have completely opposite views.  Again – that is great – let’s discuss them further as I’m always open to improvement in what I’m doing.  I still have a long way to go to meet the standards of some I hold in high esteem as reviewers, and I continue to learn all the time.  The great thing about Head-Fi is that so many people are willing to help.
 
So I have my review equipment, I’ve casually listened, critically listened, compared, measured, written my notes – now how do I put it all together and decide what to actually post?
 
The Mandatory "Head-Fi Stuff"
Let’s start right at the beginning, the Title, Pros, Cons and that pesky ranking thing. I know many people look at the Title and try to start with something eye-catching.  I tend to try something descriptive (yes I know – boring old me).  Really speaking there is no right or wrong.
 
With the Pros & Cons – try to just generally list the main advantages and disadvantages of the product – especially if you were a buyer. For IEMs and Headphones, I always like to look at fit, comfort, build, features, tuning, accessories, and value.  For amps or sources, you’re probably concentrating more on performance than sonics.  Look at it as a very quick summary of what people can expect.
 
For the rating – my personal way of going it is simply to equate each half star as 10%.  5 = 100%, 2.5 = 50%.  I try to take into account features, tuning, ease of use – and also the position of the product against its competition.  I always take value into account (some don’t and that is okay too).  For me – an IEM that punches far above its price point can get to a 4.5 or 5 star status, and similarly a reasonably good IEM which is grossly overpriced can slip to a 3 star.  The biggest things to remember are to be fair, and also to be consistent.  A sure sign of if you are doing it wrong is if every review is 4.5-5.0 stars.  I see a lot of this among newer reviewers.  Remember – you’re not working for the manufacturer - you’re supposed to be providing a fair assessment of what you see in the product being reviewed.
 
The Introduction Section
OK – we’ve now moved onto the main body of the review – so let’s look at the Introduction, Disclaimer, and Biography (about me section).  All of these are of course optional – but here are my views.
 
Introduction
I include one with every review.  It helps set the tone and allows the reader to see a little bit about why you’re reviewing the product, what you’re looking for, maybe past experiences.  If done correctly it can draw readers in – and isn’t that what we really want to do?  I also often include a little blurb about the company – because it genuinely interests me, and because some of the companies may be totally new to some of the readers.
 
Disclaimer
Readers nowadays want to know if you have any affiliation, and if it is a free review sample, or if you own it (bought and paid for), or if it is a loaner. Although a good reviewer should not let ownership bias their viewpoint – it is best to be completely up front. For me personally, I want to know this information.  I always state it.
 
Biography (or “about me”)
This is one of the most important parts of the whole review.  It is the key by which another reader will be able to relate your thoughts to their expectation.  Without it – your review is essentially useless.  And yes – I feel strongly about this one. Imagine a situation where you read one of my reviews, you have no knowledge of my tastes, and you are mildly treble sensitive.  I’m not – I clearly state it, but if there is no section you can’t know that.  You buy based on my recommendation only to find out it is way too hot for your tastes.  This section is like the key on the outside of a map – which holds the scale, the abbreviations, and all the little indicators.  Without it, you may get a feel for where you are going but you get no accuracy. It will also give the reader an idea of how much experience you have and any known bias you have.  I don’t believe in (or haven’t encountered anyway) burn-in making a huge difference.  I’m not a believer in major sonic cable changes.  To some that will be important information.
 
One last thing you can include in the preface sections is what you look for in the particular item you are reviewing.  This is completely optional, but I do find that particularly for amps, dacs and sources, listing this helps me keep focussed on what I would be looking for as a buyer.
 
The Review - Basics

1. Packaging/accessories
2. Technical specs – both the review unit and sometimes it’s close competition for comparison
3. A frequency graph and commentary

 
All of these I regard as being pretty mandatory – except maybe the graphing.  I know for packaging and accessories I could look up their product page, or a youtube unboxing.  But with the proper pictures this can be extremely informative, and quite frankly if I’m reading your review I don’t want to go anywhere else.  Likewise for specs – if you haven’t taken the time to list them, chances are I won’t read your review.  Why?  Because they are important.  If the manufacturer takes the time to list them – then you should be too. If it is a headphone, I want to know its sensitivity and impedance (that is important for any buyer). For an amp I need to know the output power and impedance from the headphone out. But go beyond that.  I weigh every IEM.  I measure cable length (if it isn’t stated).  Look at what could be important to a buyer – and list it.
 
I include frequency information because I think it is important – your choice if you choose to or not. I do find it adds value to a review, and it is something I do look for in a review if it is available. As important as the graphs is adding commentary to help people with your interpretation of what the graphs mean.  It is vitally important if you are including that the graphs are there to support the review – and not the other way around.
 
The Review - Product Features

1. Design
2. Build
3. Features
4. Use

​
All of these are pretty mandatory nowadays.  The design and build section is vital because people want to know if the product is built to last and if it is built for purpose.  Try to be constructive with your photos, and use them to illustrate the points you are listing.  And for me – photos are vital here.  I want to be able to see padding and earcups.  I want to know if a hinged mechanism is sturdy and will last. If there is good (or bad) strain relief, it is important.  If I’m reviewing a source, I want to know all the inputs and outputs.  I also need to know about things like rear outputs (for speakers) – is it fixed or variable for example.
 
Hand in hand with that is comfort (headphones / earphones).  How is the long term weight? Is weight distribution even? Are the IEMs a shallow or deep fit?  Are tips easy to fit? Is the housing a good or bad shape?  It’s amazing how many people gloss over these features when they are as important as the sonics.  I don’t care what the headphone or IEM is, if it doesn’t have long term comfort, people won’t use them.
 
And the same goes for sources etc.  How easy are they to use?  How good is the GUI?  If it has serious shortfalls, then you need to articulate that clearly.  The best way to do that is short commentary, and plenty of photos.  If it’s a wall of text I’ll lose interest.  It takes time to really cover a GUI well, but the end results are worth it.
 
The important thing for this entire section is that the photos shouldn't be artistic. You're not marketing the product, you're explaining it's use.  If in this section you're trying to frame something perfectly to get depth, or worried about hanging it in a tree to get the right colour, then you've missed the point of the whole exercise. This is all about explaining a feature, and then giving the reader the best possible idea of what you're explaining.
 
The Sonics
Probably the biggest section and the part most of your readers will gravitate to.  I like to preface it with a general statement about the sonics, and then go into detail. There are no hard and fast rules here – but what I would suggest is to be methodical.  List your test tracks, and describe what you’re hearing.  Try to imagine what you’d like to read if this was someone else’s review and you were a reader.  That still remains my biggest guide – and has served me well over the last 5 years. I write primarily what I’d like to read if I was researching the gear.
 
A few things to be careful with in this area.
 

1. Don’t rely on graphs as an indicator of sound.  The impressions should be what you are hearing, and use the graphs to support it. One of my biggest mistakes was in my review of the Noble Savant.  I was influenced by a graph. I will never make that mistake again.  Always listen before measuring (the exception of course is that you measure to volume match).
    
2. Be realistic. If I read about a source, and you start talking about night and day differences with other gear, then chances are you’re exaggerating.  Nothing turns me off more.  When you actually start comparing similar gear volume matched – the differences are often very small.  So don’t overstate it.  You’re not selling the product – you’re reviewing it.
    
3. Also be realistic about any glaring faults.  If there is bass bleed, you should have a test for it, and be noting it.  If there is sibilance, you should be noting it.  You should also have consistent tests for staging and imaging.  This allows you to be consistent in your descriptors over whether the stage is wide or narrow, deep or shallow, expansive or intimate.

 
Comparisons
This is another of those vital parts of a review, and I feel a bit short changed if you haven’t at least included something similar to compare to.  It’s time consuming and tedious to set-up, and it can take hours – especially with volume matching, checking, and rechecking.  But it pays off.  You get a much better idea of how the product compares to similar products, and that is what your readers want to know too.  I’ve even been at a point where I’ve really liked something, got to the comparison stage, and realised (after close comparisons) that I was being over enthusiastic (new toy syndrome).  Comparisons are a great way to bring you down to earth sometimes, and get right in your own mind where this particular product sits amongst its peers.
 
For sources or amps I do the same – compare it to similar sources or amps.  How do different headphones perform with them?  Are there noticeable differences? How important and how well implemented are the features.
 
The Summary and Recommendations
This essentially should be a section where you take what you’ve just written a full page about, and boil it down to a few lines.  What you’re doing is listing the good points, any flaws, and establishing your reasoning for scoring.  Somebody should be able to skip down to this section and get a reasonable idea of what the entire review is about.  What I like to do here is also include my thoughts on who the product would suit, and who would be better to be cautious about buying.
 
Recommendations are the easiest part. It’s your chance to list (for the manufacturer) changes which may be able to help them make a better product.
 
So that (in a nutshell) is my thoughts on what to include, and a little bit about the why. Before I give you your chance to comment – one final bit (warning very personal opinions ahead).
 
So What Do I Dislike About Some Reviews (what I consider poor)
Some of this I’ve already covered – but I’ll list it anyway.  These are some of my pet peeves.
 

1. Reviews with no disclaimer
2. Reviews with no personal information about the reviewers bias
3. Reviews which concentrate solely on pictures and specs, with very little actual information (i.e. they could be mistaken for advertisements)
4. Reviews with no comparisons (this is especially so for those who have a lot of review samples available).  If you’re not comparing, you’re being lazy.
5. Reviews without pictures.  You don’t have to be excellent with a camera, and even a smartphone will take decent pictures nowadays.
6. Reviews where it is obvious the reviewer hasn’t put a spell check or grammar check through the review.  I miss stuff, but I always do this before I post.
   ​

Oh and if you’re a reader and get value out of a review, click the little “Was This Helpful” button at the bottom if it was genuinely helpful.  The higher the ranking, the higher the review will be placed in the list of reviews.  This lets the best reviews rise to the top.  But please DON’T just click the button just because it is a forum friend. It has to add value. Likewise, just because someone has repped your review doesn't mean you automatically rep theirs.  Rep on merit and only on merit.  I've had plenty of reviews that haven't been repped - it just means I need to make sure I am giving the reader value.
 
So – those are my thoughts – what are yours?

 

[twister6]

Excellent outline for people to get started with their own reviews!!!

 

[h1f1add1cted]

Nice write up as usual, but can't fully agree with the separate part "Biography (or “about me”)". To me I never read this kind of separate information, I always skip this part simply, because I want to read about the gear. Don't get me wrong, this kind of information like you said with i.e. 'treble sensitive' can be very fine fit in the part if you describe the treble from headphones / earphones. To me I write a lot reviews (not in English) and up to now nobody asked me ever for my personal experience, this will be automatically clear for the readers after the entire review are done.
 
I fully agree with your part about sonics. If the review is about headphones / earphones the sound signature needs to fit the personal taste and only the most possible 'objective' description is interesting to read, it's easy to check if the reviewer is totally in "love" (blind love) with the headphones / earphones, or if he really enjoy them but highlight the "not so great" part too, because no headphones / earphones are perfect.
 
I also fully agree with the other gear sonics, like DAPs / DAC/AMPs this kind of devices don't play a big part of sonic influence to headphones / earphones (I know a lot people will disagree, but that's IMHO) if you don't use the worst gear. As better the used gear, as less sonic differences are happen, if really volume matching and blind ABX tests happen.
 

 
 

 

[SillyChili]

1. Reviews where it is obvious the reviewer hasn’t put a spell check or grammar check though the review.  I miss stuff, but I always do this before I post.
    

You missed one right on the very sentence where you tried to make that point

 
jokes aside, always find helpful information from your reviews and do please keep up the good work!

 

[Brooko]

Thanks for picking that up. And that I guess is the issue that a spell check won't pick up - when the alternate word actually exists. I'll change it now. I can assure you though that I read through the article 3-4 times and spell checked twice before I posted. Goes to show you how familiarity can aid in missing minutiae

 

[Brooko]

  Nice write up as usual, but can't fully agree with the separate part "Biography (or “about me”)". To me I never read this kind of separate information, I always skip this part simply, because I want to read about the gear. Don't get me wrong, this kind of information like you said with i.e. 'treble sensitive' can be very fine fit in the part if you describe the treble from headphones / earphones. To me I write a lot reviews (not in English) and up to now nobody asked me ever for my personal experience, this will be automatically clear for the readers after the entire review are done.

 
Thanks.  I always look for it. My own personal experience is that in my early days I often relied on reviews to make buying decisions.  I learnt early that I had to follow guys who have similar tastes to me.  But if I don't know those tastes then how can I read a review properly? Once you've been burnt a couple of times by reviewers talking about clear extended treble, and clarity through the mid-range, only to find later that their personal preferences are very dark, very warm headphones (and you've just spent $2-300 finding that out), you realise how important it can be. Purchasing the Mr Speakers Mad Dogs was a wake up call for me.  They sounded nothing like some of the reviews that were out at the time I bought them (it was quite some time ago now).  If the reviews had the authors blurb I would have missed the experience, but would have saved the money 

 

[d marc0]

Excellent guide Paul. I'm going to take a lot of them into my reviews. Maybe will start with the "about me" section.

What's the best way to approach this? if like me for example, I like the Savant which you've already heard. But I use the SD2 as my reference which is the total opposite of the Savant in the upper midrange and upper treble with a more neutral bass presentation.

 

[Brooko]

Oh I like a myriad of different signatures too - so start with extremes.  What areas are you not sensitive to - and which areas are you sensitive to?  Do you have a low tolerance for sibilance?  Do you have particularly sensitive ears and identify hiss easily?  But most of all, if listening purely for relaxation, what is your preference.  If it is for warmth and bass - tell your readers.  People who value those things also will get more out of your reviews than someone like me. If you can explain it via frequency this can help too.  For example, I know now that I tend to prefer headphones with a slight dip at 1 kHz and a slight bump in the 2-3 kHz area (must add that to my bio actually).
 
Also - other stuff I've disclosed:

1. I suffer from permanent tinnitus, and it masks my high freq recognition (It's above 14-15 kHz so it doesn't really matter)
2. I've never encountered large (if any) burn-in effects with head or earphones
3. I'm yet to find major differences with cables.

 
Basically anything that might give the reader more of a steer on where you are coming from.  Wrap it up in a spoiler tag so it doesn't clutter things, but at least it is there.  Then if someone like me comes along, the information is there, and I can make a buying choice based on it

 

[d marc0]

Thanks Paul, will keep that in mind.

 

[audionewbi]

The current obstacle for me is to get used to other type of music. For me the hardest part of evaluating a product is selecting the right music. I have found out for the sake of putting a number lets say over 80% of IEM I've come across for various testing/feedback for various MOT by nature suit a particular type of music compared to others, so selecting the right music, or at least music track diverse enough to test most kind of music genre is sometime problematic. Naturally I have a particular bias towards certain music type and that music genre might not be suitable for that IEM. Now one might ask isn't that than an indicator that perhaps that this IEM should be avoided due to it indication it possibly a flaw in tuning? To me the answer is well not really, perhaps that IEM is the best thing for an individual who only exclusively listen to that kind of music. 
I also tend to avoid using only well recorded music as they always sound good, I like to mix different type of recording. I select my music quality by first listening to the track and verify my judgement about the recording using MusicScope.
 
The second obstacle is the listening condition, should I listen early in the morning, late afternoon, should I actually do it methodically as in always doing a rapid A/B testing or should I just use the product normally just like I do for any IEM I buy for personal use? We tend to be more hyper critical to certain kind of sound tuning. How do we counter for any bias that might influence in a fair evaluation of the gears. 

Third obstacles that I like to mention is finding product to compare against the product I am testing. Should I factor in the price? If I am focusing on a review that is aiming to indicate how good a product is sonically Am I not meant to first have an idea of that "perfection" is? More importantly what if my idea of perfection is actually a product that I currently own in a particular price range, do I allow ourself to rate the product compared to that standard of perfection (which could happen to be much higher than the product I am testing) or Am I meant to compare that product to its price bracket?
 
There are a lot of other factors that is in my mind, and that is in large why I avoid writing reviews as I find myself perplex when writing the review as I do not really have a way to deal with those question in my mind. What I like to do is post periodic impression of a product, and I ensure that I have purchased that product to do that. I tend to find that when I have a short trial period I tend to be more forgiving to tuning I like and visa versa to those I don't. It is not intentional, I simply realise my bias after purchasing a product and using it with time. It is a bias that I tend to forget when music takes over. 

 

[Brooko]

Thanks for that insight. I guess the genre issue hasn't been one I've ever come across, simply because I listen to such a wide variety of mucin. I can see how it could be poblematic though. And I do agree on the recording quality. Again my test tracks probably cover a good mix - from reference recordings through to very budget Indie.

As far as listening conditions go - most of my critical listening is always early evening, but the general listening can be any time during the day or night. I think the previous chapter on time line covers a lot regarding how I listen.

And on the price, I always try to compare something similar - although if the product is punching far above its weight, then putting it up against a more expensive tier can be illuminating as well.

 

[sledgeharvy]

Brooko.. You are one of the top reviewers for a reason. All of your reviews are and have always been very well thought out, descriptive, and informative. Between you and Twister6, I have the utmost respect for you two. Any product as buy I usually read reviews posted by your two and once I buy the product, I generally find myself agreeing with yours and his reviews completely. I greatly appreciate this basic template you have posted, as I try to model my reviews like your and Twisters style of reviews.. But usually never comes close.. Lol.. But at least I have a better idea as to how to format my reviews better to make the content more structured.

 

[Brooko]

Thanks. But I think every reviewer ultimately has to find their own style and while my blog so far hopefully gives insight to what I do (and why), it's still only one example of what you can do. The secret is to read a lot from a number of different reviewers, look at the points that give you most value, then incorporate them in your own style. For example, I think possibly both HiFiChris and Earfonia have influenced me in the way they have cleverly blended the objective with the subjective - and because of it my reviews now have a more common-sense approach.

 

[serman005]

Brooko--you've outdone yourself. This is all incredibly helpful for those of us doing reviews or wishing to do reviews. Thanks for all your effort!

 

[twister6]

 

 

Brooko.. You are one of the top reviewers for a reason. All of your reviews are and have always been very well thought out, descriptive, and informative. Between you and Twister6, I have the utmost respect for you two. Any product as buy I usually read reviews posted by your two and once I buy the product, I generally find myself agreeing with yours and his reviews completely. I greatly appreciate this basic template you have posted, as I try to model my reviews like your and Twisters style of reviews.. But usually never comes close.. Lol.. But at least I have a better idea as to how to format my reviews better to make the content more structured.

 
Thank you for the kind words!  And as Paul mentioned, the template is a good starting point while it's always important to find your unique style to add your "signature" to the review
 
And one more thing, probably a bit off topic.  I'm sure Paul can related to this as well, so many times people ask in PMs about products reviewed in the past and it used to take me forever to go back searching for the review to refresh my memory until I created the index page.  Every profile page on head-fi has a link to all of your reviews (from the list on the left side), but those are listed 10 per page and not easy to find a specific one.  My Index page is: HERE, and I know Paul has one as well HERE, and those threads are typically updated whenever new review is posted so you can always stay up to date if you subscribe to those threads, and also makes it easy to find all the previous reviews.