[Shaffer]

InnerFidelity is part of the same corporate entity that owns Stereophile and Sound & Vision.

...along with Car and Driver, among others.

As for TAS, well, quite frankly, I'm actually quite happy those guys aren't really part of the head-fi community. If I want to learn new adjectives and poetic phrases that have no basis in science, I read TAS...

This has zero relevance to the initial point.

 

[taffy2207]

Funny that most of the comments I get from many, many people regarding my reviews is mostly "thanks".

Personally I find your comments regarding all non-paid for gear reviewers (generalisations) completely contrary to what I see from most reviewers.

  The word 'thanks' to reviews is the point I am getting at. Reviews are overwhelmingly positive and are therefore promoting products and influencing purchases. If pre-production units were loaned out for peoples' opinions like in a Consumer Focus Group fair enough, there's nothing wrong with consumer feedback if it makes a product potentially better to the consumer. But when that feedback is in the form of a review (again, which are usually overwhelmingly positive and at times fanatical) reviewers are basically doing the marketing for these companies. When forum members ask for peoples advice, usually other members recommend from the 'Head Gear' lists.
 
Also, a point you raised earlier in the thread is that maybe some reviewers are not writing bad reviews because they are likely to not get any gear from that company to test again. Most of the disclaimers I see point out that equiipment was loaned but how do we know that discounts are not going on behind the scenes or that loaner equipment is not being discounted? or even free samples being given out I suspect in some cases.
 
Giving feedback to the companies and testing out new equipment is all well and good and necessary but when it is in the form of a review (yet again, generally overwhelmingly positive) it is promotion, pure and simple, especially when that equipment has not been purchased.
 
I don't think disclaimers go far enough, the majority I see are with regards to loaners. Maybe a step in the right direction would be if goods were discounted or free then that should be mentioned as well. Either way reviews are tainted to my eyes. Many reviewers are not active on the forum and could be employees of these companies, I think there should be a forum post count benchmark before reviews are allowed.
 
When a product is rated as number 1 before it is even released and recommended all over HeadFi, it's just wrong to me. I'd rather gauge opinions from people that actually own the gear and have been through several firmware updates and have used that gear every day than a 'short stay' review.
 
I'm not going to change your opinion and you're not going to change mine. I can live with that. To me, there's blurred lines and transparency is needed.
 
Debate is healthy but I'm going to back out of this thread, I've said my piece on reviews.

 

[Brooko]

I understand your point - I really do.  There are some reviews - where I've reviewed the same IEM - and I'm left scratching my head.  Yes there are differences in preference, but when you're talking about bass so bad it bleeds into the mid-range, and things like poor isolation and microphonic cables - then I can't see how some people give such high ratings either.
 
But that is the nature of Head-Fi.  You have to find 2-3 Head-Fier reviewers who match your own experience - then follow them.
 
What I object to is you stating that all reviewers have bias towards the manufacturers.  Some of us make it very clear to try and stay as neutral as possible.
 
I would suggest you try at some stage to write an objective review of some of your own gear.  You'll them see how difficult it is - how much research you have to do, and little return you get for writing a review - especially when you do it for a limited listening session with a loaner unit.  But in my case, I don't do it for any "rewards".  If I receive them - I disclose it, and it's nice - but my long term listening gear is all the gear I've purchased anyway.  I do it for the enjoyment of the hobby.
I just think you have to realise that we're (reviewers) not all painted with the same brush

 

 

[taffy2207]

What I object to is you stating that all reviewers have bias towards the manufacturers.  Some of us make it very clear to try and stay as neutral as possible.  

I've never said ALL reviewers. I said its tainted because inevitably there a few that are. Stop dragging me back into this thread

 

[billybob_jcv]

There is one way to read a review that you will know you can trust with 100% certainty: Write your own.

 

[mutabor]

   
Reviews on loaner units appear to be overwhelmingly positive. Yes, there are negative reviews but they do seem to be in the minority. As I stated earlier, everytime I see a negative review it shocks me. Maybe you've inadvertently hit the nail on the head with your post. If someone writes a negative review on a companys product maybe they won't get asked to review by that company again, maybe that's another reason.

 
Loaner units usually go to experienced reviewers who are trying to make balanced reviews. What does it mean a balanced review? It means they weigh pros and cons and are trying to be careful and neutral which leads to a fact that they simply can't make a negative review. For example,  a piece of gear can sound average and nothing special but it is super comfortable or it is super compact or it is very cheap or it looks cool - you can't give a negative review. There is always a feature in a product that won't allow to bash it completely.
 
A negative review would come out of a person who is not interested to be objective. For example, a person can't tolerate bad ergonomics. He will bash every product with bad ergonomics and call it crap even if it sounds great.
 
Let's say that any experienced reviewer is politically correct by default. He is very afraid to say strong opinions because of fear to sound prejudiced or rude.

 

[Brooko]

   
Loaner units usually go to experienced reviewers who are trying to make balanced reviews. What does it mean a balanced review? It means they weigh pros and cons and are trying to be careful and neutral which leads to a fact that they simply can't make a negative review. For example,  a piece of gear can sound average and nothing special but it is super comfortable or it is super compact or it is very cheap or it looks cool - you can't give a negative review. There is always a feature in a product that won't allow to bash it completely.
 
A negative review would come out of a person who is not interested to be objective. For example, a person can't tolerate bad ergonomics. He will bash every product with bad ergonomics and call it crap even if it sounds great.
 
Let's say that any experienced reviewer is politically correct by default. He is very afraid to say strong opinions because of fear to sound prejudiced or rude.

 
I can't agree with this either I'm afraid.  A good reviewer will give a balanced truthful review.  And sometimes they are negative. I have no issues giving 2.5 stars or lower if the product warrants it. And that can even be if the product is average (sound, build, ergonomics etc) but completely overpriced.

 

[money4me247]

 
I can't agree with this either I'm afraid.  A good reviewer will give a balanced truthful review.  And sometimes they are negative. I have no issues giving 2.5 stars or lower if the product warrants it. And that can even be if the product is average (sound, build, ergonomics etc) but completely overpriced.

I follow this mentality as well. I think the best reviewing comes from a comparison against relevant market competitors with analysis of performance based on price point. Performancerice ratio & overall value is probably the best thing to use for ranking purposes (imo). An inexpensive option with sound quality much better than its current competitors would score a higher ranking from me than an pricey product with objectively better sound, but not really outstanding compared competitors in its price range. I do tend to rank based on my personal judgement of a product's value. I also think that it is important to weigh all the various factors (sound, design, build quality... etc). If there is an especially glaring flaw with any of these one single aspects, I would also feel comfortable being extremely critical and ranking a low score. The problem for me is that there is not enough scale within a 5 star ranking system for me to really illustrate how I think products rank as I use 3-star as "average" so I really only have 3.5 to 4.5 stars to play with for good products. I hardly ever give a 5/5 star ranking as I view that as a perfect ranking and I don't think there are really any item on the market that is truly flawless.
 
At the end of the day for me, I don't think rankings or scoring is really that pertinent. The sign of a good review (in my opinion) is one that covers all important considerations (pros, cons, and other things of note) so the reader can get a sense whether that device is the right fit for them. I think everyone will rank gear differently and using someone else's rankings has never really that helpful for me personally. The key is finding the product that fits an individual's need and preferences, which varies greatly. So it is hard to use a single number or star scaling to illustrate this.
 
I do think critical comments direct at the process of reviewing and on reviewers is actually an extremely good thing for this hobby. There is quite a pervasive amount of reviews that are really nothing more than advertisements and there is a surprising amount of "rewards/compensation" being offered in attempts to curry favory with reviewers (from amateur to professional reviewers). Full disclosure is the best policy in those situations. I actually greatly welcome more critical perspectives on the process of reviewing and some of the things that occur as common practice in this hobby as many people may be completely unaware of some of these concerns that are being brought up. The cool thing in my mind is how much some non-professional reviewers in this hobby care. The other less reputable reviewers who may be writing more for free products/discounts rather than educating consumers, you will not really find engaging in this sort of discussion so passionately.
 
Finally, I think that there is a large flaw with adopting an owner-only guideline for reviews as many enthusiasts in this hobby often strive to demo products prior to purchasing. Hence, there will just a propensity for just positive reviews if following an owner-only approach as we would simply just not waste our money buying products that we would review poorly. I do think review tours have its place and allows passionate reviewers with limited funds to get access to gear they normally would not purchase for some variation in opinions and perspectives that can be extremely helpful.
 
On the whole, I would recommend never placing too much weight into any single review. It is always more helpful for prospective buyers to try to get as many differing perspectives as possible. Also, I think many reviewers have differing approaches to reviews and it is always helpful to view their body of work to get a sense of their background and reviewing style. Different reviewers have different reviewing styles and ranking preferences that are better fit for different readers.

 

[taffy2207]

In the interests of transparency, I've just been offered some gear for review on here and I've politely declined.

 

[taffy2207]

To 'FREE' or not to 'FREE', that is the question.
 
 
Why are reviewers hiding behind the word FREE on here?
 
What exactly is a review sample? I've asked that question with regards to a few reviews here. It turns out it's a FREEBIE if you didn't already know. So why not state the word FREE in the review? Why all the 'smoke and mirrors' wording on here?
 
Wording on 1st review. "Before I start with my review, I would like to thank ORIVETI for providing me with a sample of their PRIMACY in-ears"
 
I asked "Can you clarify that? Is it a loaner or a freebie?"
 
The reply "It is a sample and not a loaner, so I am allowed to keep it (which also means that I will use it for future comparisons and recommendations). "Freebies" would be greedy."
 
So, it's a FREEBIE then.
 
I also had this reply.
 
"Buddy, 'sample' means free by definition. I noticed you've asked that question on a few reviews now, but over here it's really not much of a question."
 
Sample Definition :-  A portion, piece, or segment that is representative of a whole.
 
It does not mean free.
 
Wording on 2nd review "**Disclaimer - I was provided the EX1 from FiiO in return for an honest review**"
 
Again, I asked Can you clarify that? Is it a loaner or a freebie?
 
The Reply "I retain the EX1 after the review is finished."
 
So, it's another FREEBIE then. Cool.
 
The 3rd review I queried  via PM, so I won't disclose that private discussion here. I wasn't happy with the response though.
 
I've noticed that most reviewers won't use the word 'FREE' in their reviews. As a reader I'm perfectly entitled to ask how the gear was obtained.
 
It's a general point that applies to reviews here, I personally think needs addressing right across the board. If it's FREE, reviewers should state that in plain English IMHO. It's a FREE review sample, not a review sample. I'm entitled to know about any bias that 'could potentially'  influence any reviews.
 
 
Will reviewers please commit to the use of the word FREE in their reviews, please?
 
People get defensive when I ask like it's a personal attack or something , it's honestly not, but clarity in the world of reviews would be much appreciated.

 

[Brooko]

I'm quite happy disclosing the contents of the PM because I want to be very up front about it. And I appreciate Taffy's request for transparency - but there is no conspiracy. Here's what I replied:

Yep - sure thing, The review unit is supplied by the manufacturer totally free of charge - and all they expect is for me to review it. There are no expectations on the review being positive or negative - they get no say at all with how I review it. It's my honest opinion.

I work a little differently to most other reviewers though - in that I have contacted all the manufacturers that send me review gear with following understanding:

Unless I buy it, I always consider the product their property
I'm more than happy to return all equipment following a review - as long as they pay the freight in getting the goods back, I used to do this myself for a few guys - but it got too expensive (the freight) as I got more popular.
If they choose to leave the gear with me (for follow up comparisons / reviews), then I am more than happy to keep it here in pristine condition with full packaging etc - and always happy to return it if they ever want it back

So I never regard the products as mine. They aren't freebies - as much as loaners. In reality most manufacturers now leave me with the gear. It is helpful for comparisons at later dates - and because I get so many questions (following the reviews) it's helpful to be able to refer back to the actual product to answer them.

The other reality is that when I'm listening for relaxation - most of the time I'm actually using my own bought and paid for gear. And I try (funds allowing) to always buy anything I consider exceptionally good that I intend using privately. My last purchase was the Jays q-Jays (I could have kept the review unit, but I insisted I pay for it because I liked it so much). My next will likely be the Fiio X7.

My other main listening equipment is mostly bought and paid for (or occasionally prizes won in competitions) - including iDSD, NFB-12, LD MKIV, X1, E11K, X3ii, Alclair Curve, Adel U6, q-Jays, HD600, T1, QC25 and a lot more I can't remember - like the A83, Altone 200 etc.

Basically I like to remove as much potential bias as I can when I'm reviewing. If I regard it as simply borrowed, then there is reduced potential for making reviews overly positive.

Cheers

Paul

To which I essentially got "so it's free then"

I then replied:

There is no smoke or mirrors. Never has been with me. For what it's worth I was one of the first (if not the first) to actually start using disclaimers. And as I stated quite clearly to you - the gear is not "free". Contact any of my manufacturers. They own it - I don't. A lot of them think my stance is a bit "quaint" - maybe even eccentric. But if the gear was mine, and was free - I would state it.

You just can't interpret your idea of what is going on & then put that label on all of us. If you want confirmation of how I operate - ask Alex (Twister6) - he knows and can confirm it.

Paul

Now - I'm not trying to obfuscate or try and pull the wool ver anyone's eyes. This is truly how I operate. I even tried to pay Fiio recently for a replacement E17K recently. They refused to accept payment. I've tried, my conscience iis clear - so the E17K remains a loaner.

 

[paradoxper]

LOL. Could just be upfront about everything from the beginning and avoid dirty laundry. Brooko, you're a strong reviewer in this space set the example.
 
Taffy is simply saying you say it's not FREE but a loaner that you'll never return. That's weaseling.
 
 
And on that note; further reasons simply not to trust these sources and salt their impressions, naturally.

 

[Brooko]

If it was free I could go ahead and sell them.  If it was free I could do whatever I wanted with them.  They don't belong to me - so how can they be free?
 
In my last review I chose to use the word gratis - as in:
 

The L5 Pro was provided to me gratis as a review sample.  I have made it clear to L&P that I still regard any product they send me as their sole property and available for return any time at their request. But I thank them for the ability to continue use of the L5 Pro – both for follow up comparisons and also for my own personal use.

 
Gratis quite literally means "without charge or recompense", or "used to indicate no money was paid for anything".
 
I believe that describes my situation a lot better than saying they were "free".  And I have been completely honest stating that my understanding with the manufacturers I deal with is that the gear really is a loner unless I pay for it.  You can choose what you want to believe but that is what I state.
 
I have stated my position very, very clearly.  I don't see how much more forthright I can be.  Why should I conform to what you want?  Like Taffy - you don't know me, or my morals, or my own personal standing in my real-life community.  Yet you are quite content to tell me that what I am doing is weaseling.
 
I could (and probably should) just ignore you - but this little crusade that a few are on has so incensed me that I am very close to just chucking it in.  The worst part about doing that would that the loser would ultimately be me - because it would stop me doing something I love doing (writing about audio gear).
 
And this part got me:
 

  LOL. Could just be upfront about everything from the beginning and avoid dirty laundry. Brooko, you're a strong reviewer in this space set the example.

 
I believe I already have.  I was one of the first to consistently write disclaimers into my reviews.  How many are doing it now?  I was one of the first to consistently list my own biases.  How many are doing it now? I have also set ground rules as already stated so that (to me anyway) it reduces my bias - knowing the gear is never a "keeper" unless I buy it.

 

[paradoxper]

LOL. What you're doing is absolutely weaseling. 
 
I'd say my issue is not obtaining review samples - per gratis, but rather your BS about not owning any of the gear you review.
 
You are sent review samples (review samples aren't retail units, though they may be final production units) and your review sample is free of charge.
 
The reason you can't sell your units is because that's considered flipping and you damn well know it. 
 
Unless the manufacture explicitly states return the review sample, you keep it. Stop dodging that fact.
 
As for how many other reviewers are as transparent as you, no clue. I don't really pay attention to reviewers nor trust them. If a product sparks my interest I'll buy it and draw my own conclusions.

 

[Brooko]

If the community thinks I'm obfuscating (http://www.head-fi.org/t/797093/call-to-all-head-fi-members-i-need-help-guidance-please), I'll do something about it.