[thievesarmy]

 
 
No one in this thread has been able to demonstrate that reviewers are not similarly subconsciously biased by the relationships they have established with manufacturers. No one has really pointed out why what is true for doctors, is not true for reviewers. No one has really pointed out why what is true for doctors is not true for moderators or administrators.
 

 
well, I could be wrong, but I don't think that you can just take a phenomenon like that, which has been studied in one specialized field and just apply it to another different field and then say "you haven't proven that this isn't true!" YOU haven't actually proven that this IS true, you're assuming it is true, when the circumstances of these 2 things are not the same. Doctors prescribe drugs in a way that is completely different from gadget reviews that are essentially positive or negative recommendations. The relationship and dynamic between you and a doctor is quite different from you and a reviewer. The point is I think a lot of people disagree with the method of which you have made your claim. It's far from scientific and certainly not a given in this respect. 

 

[thievesarmy]

You clearly have a limited understanding about public disclosure. IT IS THAT SIMPLE. A short disclosure at the beginning or end of a review is all analysts do and that is all I am advocating here on Head-Fi.

 
well I'm sure any minute now the SEC will get involved in this matter.

 
 

 Please read my biography. I am a corporate partner in law firm with 15 years experience

 
good for you! Sure, I'll read it. Just send me a "review unit".

 

[fiascogarcia]

 


Are we talking about reviews by Jude & the head-fi staff, or about any random thread with a user review? I will say again: Anyone can post a review. Anyone.

I dislike the Audio-Technica ATH-M50. I find it lifeless and lacking in any kind of sonic character. I think it is vastly overrated, especially for the price.

There - is that a review?

Thanks for the review.  You obviously get free gear from AT's competitors!   

 

[spook76]

well I'm sure any minute now the SEC will get involved in this matter. :rolleyes:  


good for you! Sure, I'll read it. Just send me a "review unit".

I stated I am using the SEC as analogy in the beginning because I think it is a far better comparison than Lachlan's pharmaceutical analogy. The pharmaceutical analogy is more apt for a salesman in a headphone store selling to the public than a reviewer.

 

[a_recording]

  It's far from scientific and certainly not a given in this respect. 

 
And of course I could be wrong too. But this is just an analysis of incentives. Doctors have a incentive to prescribe a given medication from increased exposure to drug reps. It's hard to argue that for audio reviewers these incentives aren't stronger given that reviewers aren't exposed to the same kind of liability that doctors are.
 
The problem is even worse with reviewers because a Doctor has a relatively symmetrical relationship with the representative, or perhaps superior. The Doctor doesn't have a dependency on the rep to prescribe medication. The reviewer as a dependency on the manufacturer to provide review units.
 
Everything I am saying is based on my experience as a reviewer. Allowing manufacturers to allocate review units means allowing them to allocate visibility.
 
Considering that people like Tyll Hertsens has expressed publicly that he doesn't see much point in posting negative reviews, I think it is really hard to argue that the incentives are not tilted towards more positive reviews than negative ones.
 
 

A short disclosure at the beginning or end of a review is all analysts do and that is all I am advocating here on Head-Fi.

 
It's crazy that even in most big audio publications, even on Innerfidelity, there is no clear statement of ethics and frequently no disclosure of the source of the unit. This may well be because basically as Tyll has said before, every headphone he receives is a review sample from Headroom and therefore he doesn't feel it's necessary to state this, but for someone who does not have this information it is absolutely unclear.

 

[Brooko]

Lachlan - I really do see your viewpoint.
 
But can you also see the point in my past few posts.  What I object to is that you are proposing that all reviewers are subject to this bias, and all reviewers colour their reviews accordingly, and therefore none of us can be ultimately trusted.
 
However - as I've been at pains to point out:

1. For almost all of my reviews I haven't approached the manufacturer (unless via threads on Head-Fi where they are actively asking for potential testers).  Most of mine now are manufacturers approaching me, or other reviewers recommending me to other manufacturers as a possibility for future reviews.
2. I do not ask for, nor expect to keep the review units.  All I do expect is that the manufacturer will at least pay the freight (including return) - I would prefer not to be out of pocket
3. I always disclose my relationship with a manufacturer if the review is a sample, and also the status on whether it is a loaner or a gift
4. Where I see real value in a product (for my own uses), I offer to purchase it.  So far doing this - I have actually paid twice for the review unit (albeit at the manufacturer's suggested price idea / ie the price was reduced).  This has always been negotiated after the review was posted.  It hasn't influenced my review.  Most manufacturers I've approached in this way have insisted they do not want payment - and considering freight costs - usually ask me to keep the review units.  I thank them - I note it in the review - and usually the boxes end up in storage, and available if I need to compare with a new review unit at a later date.
5. Even at the risk of doing no more review work for a manufacturer (has happened once) I have always been a scrupulously honest as I can be with what I am reviewing, and I continue to stand by that.  At the same time - one of my natural bias is that I am usually a glass half full instead of glass half empty kind of person.  Anyone following my reviews will know that.  I will point out faults as I see them, offer the manufacturer suggestions, and grade accordingly.
6. Unfortunately there are also a number of extremely critical users in some parts of the forum who are very quick to point fingers and make a pointed remark about bias (I'm not talking about anyone in this thread BTW).  They see some manufacturers faults as being glaring and that I've glossed over them.  But - they don't have my experiences, expectations, or preferences - and I don't have theirs.  This to me has a far greater effect than possible bias because a manufacturer chooses me to test and review their gear.  Take a look at the extremes on the Fiio X5 thread if you want an example. If you come from an Apple ecosystem, chances are you may dislike the UI.  If on the other hand you've had multiple DAPs with both good and bad, you'll see a much more (IMO) balanced viewpoint.
7. There is no real value in getting and retaining the samples - other than getting to try new gear, and occasionally getting something you would definitely continue to use for yourself.  You can't sell them.  You can't trade them.  Both is against the TOS - and anyone who's tried (to my knowledge) has been "bounced" pretty quickly.  I personally offer to buy anything I think is really good.  See my point?

 
I would agree with an amendment to the TOS on Head-Fi (as long as Jude and Administrators saw it as adding value ultimately to the reviewing process) for reviewers to disclose:

• Affiliation with a manufacturer if it exists
• If the sample is loaner or gifted

 
I do it already - so ultimately it changes nothing in the way I do things.
 
But I cannot support the crowd-funding idea.  It supports only a few - and I think there is as much bias in that as there is in the current method we already use. 

 

[a_recording]

  Lachlan - I really do see your viewpoint.
 
But can you also see the point in my past few posts.  What I object to is that you are proposing that all reviewers are subject to this bias, and all reviewers colour their reviews accordingly, and therefore none of us can be ultimately trusted.

 
I do understand your points as well. And to be clear some of what I am saying probably applies more to people who are making any kind of significant money from reviews (Tyll, myself, HiFiGuy, Headfonia, etc). But I feel like ethical standards should be the same for everyone involved.
 
I definitely am not saying that no reviewer can be trusted. I very happily read Tyll's reviews and I think they are useful, though I do notice that he uses a lot of qualifiers and modifiers to his words. That's okay - I do that as well. But lack of disclosure and transparency is what feeds the conspiracies, and there is a basic grain of truth to the idea that review units must in some way create a relationship of dependance between a reviewer and a manufacturer.
 
I'm not saying I don't trust reviewers, I'm saying I don't trust the reviewing system, and I want to seek ways to improve it. Maybe crowdfunding isn't going to work for everyone. Maybe we should more actively encourage or organise lending or tours or something. Whatever we do it has to be a broad community action. 
 
All I am trying to do is to get people to recognise a potential problem and come up with solutions to address it. In order to do that the community as a whole has to talk about it and to explore all possibilities.
 
This is why I was upset that my original comments were moderated. People have said that maybe I could have started the discussion, and then started the crowdfunding, and then maybe everything would have looked better. But like so much in life I didn't plan this. I thought "gee crowdfunding might be a good idea!" I tried it and then thought, "gee, maybe Head Fi should be talking about this because it could work for other reviewers!" 

 

[spook76]

Brooko,

Your suggestions have merit. I like HeadFi and all I am seeking is ways to improve the site without a wholesale change in the reviewing process which I do not think is warranted. We are all intelligent people who, with disclosure, can make an informed decision about products and reviews.

 

[Claritas]

I would add returning or buying review samples afterwards whenever possible.

 

[a_recording]

At the very least, here is something to look at. At the time of posting here are the front page featured reviews on Head Fi, selected by the moderators. You would expect that the moderators would read these reviews and check if they comply with guidelines before posting. Here is the rule:
 

 Importantly, you must state, when expressing an opinion on or writing a review of equipment if it was loaned or given to you for posting comments or review or discounted in any form at purchase. You must also declare any relationships you have or had to audio equipment manufacturers or resellers. We like everything to be open and transparent here and that includes whether or not people are receiving any financial or other benefit from posting.Members of the Trade and Sponsors are clearly marked in their profile and posts so there is no ambiguity about a person's reason for posting. Failing to declare a discount or association may result in your reviews being deleted and your account being banned and the manufacture may risk black-listing from Head-Fi.

 
Now here are the reviews:
 

 "Comparison / Review : A tale of two Sennheisers – HD600 vs HD700" by Brooko. Source disclosed:
 
"So it’s perhaps telling that the two headphones I’ve had the longest – and also purchased twice (after selling once) are Sennheiser’s HD600 and Beyerdynamic’s DT880.
 
But my quandry now lies in the fact that I purchased (on a whim – and despite some of the early criticsms) Sennheiser’s latest mid-to-hi-fi headphone release (the HD700)."

 

"Oppo Digital HA-1: A DAC Amp that's fully packed jammed with features" by AnakChan. Source disclosed:
 
"Now I have to admit, prior to getting the loan from Oppo Digital Japan (thank you to them by the way!!), I was looking forward to the PM-1 but after receiving the set, I think I'm left by being more impressed with the HA-1 DAC/Amp."

 

 "REVIEW: Rock It Sounds R-50M - Well Done!" by Zelda. Source not disclosed or unclear.
 
 Review mentions street price but does not make clear this was a purchase.

 

 "Comparison: Grado PS1000 and PS1000e" by MacedonianHero. Source not disclosed or unclear.
 
Review makes NO mention of unit origin.

 

 "Comparison: V-Sonic VSD3S vs Havi B3 Pro 1" by peter123. Source not disclosed or unclear.
 
Review makes NO mention of unit origin.

 

"(2C62) Astrotec AX-35" by ljokerl. Source not disclosed or unclear.
 
Review mentions street price but does not make clear this was a purchase.

 

 "MHDT Labs Stockholm v2 : A Review" by s1rrah. Source not disclosed or unclear. 
 
Review makes NO mention of unit origin.

 

"Meridian Prime on Head-Fi TV!" by Jude. Source not disclosed or unclear.
 
Review makes NO mention of unit origin.
 

 

"Impressions: Audiofly AF180" by jonyoo. Review unit declared, but does not make clear if it was a review loan or a keeper.
 
"I was offered a pair of AF180 for reviewing here in Korea and I thought it would be nice to share and talk about it here on head-fi as well."

 

Impressions: HiFiMAN HE-400i by ThePianoMan. Source declared.
 
"Just finished up at the Chicago meet, and I have to say the HE-400i was one of my two favorites of the show!"

 

"Reference sound from Nuforce" by Dsnuts. Review unit declared, but does not make clear if it was a review loan or a keeper.
 
"I managed to wrangle a pair from Nuforce to write this review and I am so happy I did..I would like to fist publicly thank Wolfgang0119 formerly of Nuforce for reaching out to me for my views regarding various Nuforce gears and phones."
 

 

"Possibly the best value in Custom-IEM world." by AmberOZL. Unit origin declared.
 
"Purchased on: 06/19/14 Price paid: $225.00"

 

 "Oppo has created a great successor to the old vintage Yamaha and other orthodynamic (planar) headphones with the PM-1." by Currawong. Source not disclosed or unclear.
 
There is nothing in the written review about unit origin. I watched the video but could not find any mention of unit origin in the beginning or the end of the video. If it is somewhere in the middle of the 11 minute video please let me know.
 
EDIT: I informed Currawong and he confirmed this information was missing. Following this post, this has now been appended to the review: "Thanks to Oppo Digital Japan for lending me the PM-1 and HA-1 for review." 

 
Now, I am sure that many of these cases are units where the unit has simply been purchased and the reviewer didn't see the need to declare it. When I was reviewing products in the beginning I didn't think about this stuff because I never got review units. But some of these reviews seem to be review units that have no declaration as such.
 
These reviews have been picked by the moderators to be featured on the front page. Surely you would expect at least these reviews to have been reviewed by the moderators for rule compliance? EDIT: I have been told that it is not the moderators who choose which reviews to feature, but administrators. The point still stands.
 
Mods could always ask a reviewer to edit the review and clear things up before featuring it on the front page. This is clearly not happening.
 
Only 4 or 5 out of 13 of these hand picked front page reviews actually comply with the posting guidelines. Some of the reviews that fail to comply were posted by the moderators and administrators.
 
How can anyone take the issue of bias here seriously if the leadership of this site doesn't even comply with the rules that they created to combat bias?
 
EDIT: Some of these posts I imagine will now be retroactively edited. At the time of posting I did read through the full text of the post that was linked through the front page. Check the post history for edits.

 

[thievesarmy]

   
All I am trying to do is to get people to recognise a potential problem and come up with solutions to address it. In order to do that the community as a whole has to talk about it and to explore all possibilities.
 
This is why I was upset that my original comments were moderated. People have said that maybe I could have started the discussion, and then started the crowdfunding, and then maybe everything would have looked better. But like so much in life I didn't plan this. I thought "gee crowdfunding might be a good idea!" I tried it and then thought, "gee, maybe Head Fi should be talking about this because it could work for other reviewers!" 

 
Maybe it could work for other legitimate reviewers, but consider that it could also be exploited or adopted by people who suddenly claim to be reviewers, and then begin soliciting the board for contributions to review the next hot product so they don't have to buy it. So not only are they spamming the board, they're also posing as legit reviewers, so there is that concern as well. Head Fi is a source of news & reviews, but it's also a forum / community (arguably their greatest asset) so when something pops up that could have some unintended negative implications on the community, even if that is something that could have some potential benefits, the mods need to consider it and react accordingly. I'm guessing that is what happened here, and you may have taken it personally when in fact, it wasn't. Also I'm not sure if this would have played out differently if you weren't actively engaged in a crowd-funding review effort, but that could have also played a role in their decision to "nip it in the bud" so to speak. 
 
One other thing I think you should consider, on the topic that you brought up about a lot of reviewers only doing positive reviews (your claim being that it was they have either bias or incentive due to their relationships w/ manufacturers). There was an episode of the PBS show Frontline that looked at advertising and marketing, where they talked to the head of one of the big NYC advertising firms - I think it was Saatchi & Saatchi (think modern Mad Men). This was close to a decade ago, but he was talking about how we have reached a point where for the most part "everything works nowadays". In the old days of advertising, they frequently used "-er" words to market products, "Fresher, brighter, cleaner, BETTER!" But now, pretty much everything works well enough and is of a minimum useful quality standard. I often think of that when it comes to headphones, and reviews. How many truly BAD headphones are there out there? OK, yes, I know there are some bad ones, but those aren't the ones getting reviewed by Tyll or Headfonia. They're looking at the stuff that is at the higher-end of the market, generally made by longstanding reputable companies who really know audio and know what they're doing. These companies aren't likely to put an outright BAD product into the market. So what is a reviewer to do when products aren't necessarily bad, so much as they are DIFFERENT? (or perhaps expensive compared to other cheaper options, but again that doesn't mean they're bad) 
 
They can comment on these differences or specifics while still being honest that the product is for the most part good. Sometimes you see them point out some thing that is bad, like a design flaw or particular sonic weakness, but it's just one aspect of an overall picture. If they want, they can give a subjective opinion about if THEY liked it or not, but the thing is to keep in mind that all reviews are subjective in this way. Regardless, there is still no true way to know if bias exists (whether there is a relationship between reviewer & manufacturer or not), OR for that matter if you will like a product that is positively reviewed or not. Even your idea of a crowdfunded review is not immune to any forms of bias, and my guess is if you're crowdfunding to purchase higher-end gear, what are the chances those products will really earn negative reviews? 
 
Anyway, I'm glad to see that you have returned and I hope you continue looking for ways to improve reviews & the community.

 

[Hutnicks]

I have a couple of questions here.
 
1) OK so you've crowdfunded your little reviewing venture and all is well, what happens to the gear after the review?
 
2) The physicians/pharma scenario that you keep referring to is a "tip of the iceberg study" have you done any other reading / research on the subject of what influences choice?
 
 
To THE MANAGEMENT of Head Fi.
 
What happens if a crowdsource funded manu wishes to become a sponsor here?

 

[sfwalcer]

   
No, of course I can't compel you to do anything. This isn't me simply just mouthing off about this subject though. I actually have to put my money where my mouth is and do what I am talking about, which is why I decided to stop accepting review units. I don't want to go off and start any witch hunt but at the very least there are forum rules that insist on a certain degree of disclosure. As far as I know, the community has never really publicly discussed reviewer's ethics in one thread, which is again what I wanted to start here. People can always try their best, but people also need to discuss standards and raise them where appropriate and feasible.
 
I appreciate the moderators letting this discussion continue, but this was the discussion I wanted to have in the first place and I am disappointed about what happened to get us to this point.
 
The moderators could compel people to disclose these relationships, but they are already very busy with the rest of their moderation. I don't think anyone actually goes around flagging reviews for not elaborating on following the disclosure rules either.
 
On a side note I would encourage the moderators to think about switching Head-Fi to a semi community moderated system where people vote on comments and comments with enough negative votes will disappear below the visibility threshold. Similar systems are used in other forums and places like Reddit, and it seems to prevent the kind of incivility that moderators are currently busy trying to eliminate. It may or may not lead to similar levels of groupthink but at the very least it makes moderation more transparent since it is in everybody's hands.

^
Oh hells nahs!!! That would no doubt spell the end of my trollish ways, cuz i gots haterz EVERYWHERE. Bye Bey head-fi, it has been a fun run so far...... T_T

Spoiler: Warning: Spoiler!

Oh a serious note, welcome back again Ayyy meng, and is so glad that you are not a man of your words. haha

This conversation has been damn interesting so far, so keep it coming folks. Spirited and intelligent discussions like these will only make head-fi a even better forum and i hope it will bring some interesting posters back as well cuz the forums here are pretty dead of late.

 
 
edit: oh and btw since you mentioned this topic on your "I am done with head-fi!!!" vid, just wanna update ya' on the topic is alls.
http://www.head-fi.org/t/654159/the-great-global-warming-a-k-a-climate-change-thread-and-the-science-behind-it-conspiracy-theory-or-fact
 
Don't believe EVERYTHING they teach ya' in school meng. I know it's the cool and hip thing to "believe in" nowadays buts..... Just facts meng straight facts and common sense. :  )
 
Too bad political discussions aren't allowed on this neck of the woods cuz i will own ya' like how i owned ekey boi. hoho

 

[a_recording]

  I have a couple of questions here.
 
1) OK so you've crowdfunded your little reviewing venture and all is well, what happens to the gear after the review?
 
2) The physicians/pharma scenario that you keep referring to is a "tip of the iceberg study" have you done any other reading / research on the subject of what influences choice?
 
 
To THE MANAGEMENT of Head Fi.
 
What happens if a crowdsource funded manu wishes to become a sponsor here?

 
I can't answer your first question because I have now been warned I will be banned for talking about it.
 
2) I have done a lot of reading about economics and incentives. I wish I could point you towards more specific things, but it's been a lot of background reading. As I said, I'm not a behavioural economist. I have no expertise in the field besides intense interest. I can tell you that all the research on compliance and conformity would suggest that people are influenced by their peers. Foot in the door technique, etc.

 

[LaPierre]

Answer to Hutnicks (1): How about gifting it back to the contributors, somewhat like a raffle?  (y)