[Claritas]

Are we talking about reviews by Jude & the head-fi staff, or about any random thread with a user review? I will say again: Anyone can post a review. Anyone.

I dislike the Audio-Technica ATH-M50. I find it lifeless and lacking in any kind of sonic character. I think it is vastly overrated, especially for the price.

There - is that a review?

If Audio-Technica were paying you for anything or giving you anything, I hope you'd say so from the start. That's the minimum the public can ask for. Same goes for anyone.

 

[Brooko]

But in many cases - the more established reviewers do already disclose this.  I know I do.  So if that is being done, and becomes the standard expectation, then doesn't this go a long way to answering Lachlan's concerns?

 

[Claritas]

But in many cases - the more established reviewers do already disclose this.  I know I do.  So if that is being done, and becomes the standard expectation, then doesn't this go a long way to answering Lachlan's concerns?

It's a first step, though it's far from universally observed. (I won't name names.) And it doesn't go far enough toward minimizing bias. The next step would be for reviewers to either return--or buy--the item after the review.

 

[spook76]

Are we talking about reviews by Jude & the head-fi staff, or about any random thread with a user review? I will say again: Anyone can post a review. Anyone.

I dislike the Audio-Technica ATH-M50. I find it lifeless and lacking in any kind of sonic character. I think it is vastly overrated, especially for the price.

There - is that a review?

Only reviews where the reviewer has a possible conflict of interest.

 

[spook76]

But in many cases - the more established reviewers do already disclose this.  I know I do.  So if that is being done, and becomes the standard expectation, then doesn't this go a long way to answering Lachlan's concerns?

Exactly Brooko. Once we are armed with the knowledge of any bias it is up to the listener or reader to discount the review as they may.

Lachlan's approach assumes we cannot make out own decision once we have the disclosure.

 

[Brooko]

It's a first step, though it's far from universally observed. (I won't name names.) And it doesn't go far enough toward minimizing bias. The next step would be for reviewers to either return--or buy--the item after the review.

 
And here's another part of the problem.  I've already offered to do that.  With the BA-100 (which I really liked), I offered payment to HSA because I wanted to - but they insisted I keep them free of charge.  They instead suggested if I ever need another pair - to pay for the next ones.  I've also offered to return samples after use.  Almost inevitably (especially with my location being relatively expensive for freight - google Invercargill, New Zealand 

 ) manufacturers again tell me to keep the review samples.
 
Another example is the development work some of us are doing with Rockjaw.  I've offered to assist with freight costs, offered to return the review units, and in the case of the Alpha-Genus told Bob I'd like to pay for it - as I really do like this unit. They've said no - insisted actually that they wouldn't take payment - mainly due to them working with the reviewers/testers to help develop the product.  This is a model that really works well - for both manufacturer and tester/reviewer, and ultimately gives the community a better product.  Win-win-win.
 
And again - there is a lot of talk about bias.  Maybe some reviewers can be influenced.  I won't be.  I always give my honest opinion.

 

[Claritas]

Other reviewers should follow your example. Industry-wide standards can be adopted, if the public isn't complacent or cynical.

 

[spook76]

Brooko as long as there is brief disclosure of such a relationship then your behavior/relationship you noted is perfectly reasonable and I agree a win-win for all concerned.

 

[thievesarmy]

Would it be so difficult for a moderator to begin or end a review with "this product was manufactured by a sponsor of Head-Fi". You are right that would be just too onerous. No, a past sponsorship relationship would not have to be disclosed. It is an simple matter of disclosure of conflict of interests at the time of the review.

FYI the SEC standard is applied to all analysts even if they work for "small, private, niche websites"

 
lol, You're talking about stock analysts that sometimes work for Fortune 500 companies (who make upwards of 6 to 7 figures for their JOB of analyzing such things) recommending the buying and selling of stock in large PUBLIC companies. Comparing that to a site of this size is… well, in my humble opinion, downright silly. What reviewers make that kind of money for their analysis? Do ANY of them make money? What organization on par w/ the SEC regulates the "small, niche website" market? There isn't such a thing, which I'm sure you realize. So basically I think you're comparing apples to oranges. 
 
As for your example of "this product was manufactured by a sponsor of Head-Fi" - is it really that simple? Once again, what if the company was a sponsor back in 2002 for a 3-month period, for an entirely different product. You expect someone to look that up and include all those details? If you do, OK, that's your prerogative, but I wouldn't expect that from a site like this. It wouldn't be as simple as you're making it seem.

 

[Claritas]

As for your example of "this product was manufactured by a sponsor of Head-Fi" - is it really that simple? Once again, what if the company was a sponsor back in 2002 for a 3-month period, for an entirely different product. You expect someone to look that up and include all those details? If you do, OK, that's your prerogative, but I wouldn't expect that from a site like this. It wouldn't be as simple as you're making it seem.

The better question is, "Is it really that hard?" I don't understand why the slightest difficulty should be so off-putting.

 

[a_recording]

Okay, I just can't help myself. I've calmed down and it looks like we can actually have a positive discussion here, so I will at least continue posting here because I feel that this discussion can benefit Head Fi, which was my intention throughout all of this. Firstly, I'll say this: when I made that video I was very upset, and I probably could have explained my points better than I did. I've delisted the video because it's run its course and won't add anything more to the discussion. I'm glad to see that this thread has not been locked like I was originally told it would be simply from the mention of crowdfunding. I think that as billybob_jcv pointed out, a discussion about bias in the audio industry is an interesting topic at least worth discussing. So putting aside all the drama, I'm glad that this is happening.
 
I do not think that the decision to moderate my comments about crowdfunding were part of some larger conspiracy on the part of the moderators to protect the sponsors. In all these discussion about bias, as I stated in the original post, I DO NOT think that there is any intentional bias on the part of ANY reviewer or moderator.
 
I was upset by the moderation decisions because the decisions was based on curbing a direct solicitation for crowdfunding. I did not feel I was making that direct solicitation, and I offered repeatedly to edit the post so that it was clearly NOT a direct solicitation. I thought to block a general discussion about crowdfunding and how it could work for reviewers was unreasonable, because many other crowdfunding campaigns DO get talked about on Head Fi, and I was trying to propose it as an alternative model which we don't even know the feasibility of.
 
So I interpreted the decision as having two possible rationales. The first rationale being that I was being a scammer or a spammer myself, which I find a little depressing because I feel like I have contributed enough here that that is demonstrably not true. On that basis I would not be able to continue posting here anyway, since if I write a review I would have to disclose that I obtained it through crowdfunding, and if mentioning crowdfunding in any way is somehow a direct solicitation then obviously I could not continue.
 
The second rationale is simply that the moderators did not see the merit of my proposal, and since they didn't see any merit at all they thought it simply wasn't worth discussing given the potential (but unsubstantiated) prospect of spam and scammers. But while I acknowledge Jude has a right to moderate in any way he likes, to me I feel this is an overreach because that is an arbitrary decision to curtail discussion in the name of some bogeyman.
 
All this aside.
 
On the point of bias, I have said again and again, the bias I am talking about is not the kind of bias where reviewers are consciously doing dodgy deals with manufacturers. 
 
The bias I am talking about is a systemic bias. It's a question of whether the system, over time, produces effects which lead to overall MORE positive reviews, and whether or not this pollutes the shared pool of information available for consumers.
 
The only person who seems to really grasp what I am trying to say here is Norway, so I'll repeat some of his words:
 
Quote:

 
Yes, each reviewer brings his own biases, but what a_recording brings up is a systemic bias. It is structural by nature and probably works in most if not all professional reviews. Individual biases come in different variations, whereas this structural bias is likely to be extremely strong and will probably dominate the review more so than individual biases. As a reader you can also eliminate a person's individual biases by (1) noting them and interpret their content in that light or (2) stop reading their content.

Quote:

 
Also, there is the bias of social bonds. I don't know about you gatekeepers, such as Jude, Tyll, Hifiguy, ... But I have the impression that you guys have personal relationships with some of the manufacturers and retailers (maybe strictly professional, but on an ongoing basis). It's probably unavoidable in a niche of a niche, but then again, it introduces biases. I think most people who aren't psychopaths would find it hard to say something that might work against a good (professional or personal) social relationship.
 
The essence of what a_recording tries to do, is to eliminate this coupling with reviewer, manufacturer and retailer, because it does create biases.

 
The principles I gather are the right to discuss potential structural (unconscious and conscious) biases and how they might color the perception of professional reviewers with the current review model on this platform, as this is the biggest community and where the elimination of such biases might be to the best of the community at large. By not allowing for such discussions the intention can not be to do what is best for the community.
 
I agree with this. Just because we have a lot of personal biases, does not mean we should not talk about or try to avoid the biases introduced under status quo. We should work to eliminate the most dominating ones, and I agree with a_recording that the structural biases are the most dominating of all, because it affects everyone who operates within the current model, which is pretty much every professional audio reviewer out there.

 
The most valid comparison is Pharmaceutical companies. It has been demonstrated in studies that Pharamceutical companies giving out little pens and notepads and trinkets and taking Doctor's out to dinner and having meetings etc. etc., have a subconscious effect on the prescribing habits of Doctors. This is not an intentional bias that you can control for. I'll repeat this quote again:
 

Dana coauthored a 2003 article in JAMA outlining the social science research that indicates that the former drug representatives and other critics are missing the point. The problem is not unethical behavior but rather an unconscious, self-serving bias that distorts the judgments of doctors and anybody else who is offered a gift, he said. Experiments show that most people are unaware that they constantly use this bias and have little control over it. So when physicians say they don’t think gifts influence them, they may well be telling the truth as they see it. 

 

We place doctors in the highest positions of esteem and trust, and we expect the highest standard of ethical behaviour from them, yet even in this industry there is systemic bias introduced by a model where Pharmaceutical companies have direct contact with doctors, and doctors are taking the step to eliminate this systemic bias not by trying really hard to not be biased, or telling patients to 'judge for yourself!' if they are biased or not, but by eliminating contact with manufacturers and therefore eliminating the SOURCE of the systemic bias.
 
No one in this thread has been able to demonstrate that reviewers are not similarly subconsciously biased by the relationships they have established with manufacturers. No one has really pointed out why what is true for doctors, is not true for reviewers. No one has really pointed out why what is true for doctors is not true for moderators or administrators.
 
Now we can argue whether or not this even matters, because Doctors can kill people with bad treatment and headphone reviewers (hopefully) can't kill people with bad recommendations. But I don't think that the argument that 'this doesn't really matter' is really a satisfying reason why we shouldn't work towards a better system - not necessarily crowdfunding, but ANY of the other alternatives discussed in this thread.
 
At the very least, Head Fi as a community should be insisting that in any case of receiving a review unit the review unit MUST be disclosed and the basis of how that unit was received MUST be disclosed. This is in the rules, but it hardly seems to be demanded by the community, and consequently is never flagged for the moderators to look into.
 
I don't think the standards I am talking about are anything special. They are standards for all kinds of different journalism, from normal reporting to gadget blogs.

 

[thievesarmy]

Maybe it's not, honestly I don't know and neither do you - but the ones suggesting this aren't thinking the idea through to completion or getting into the specifics, which is always the case with keyboard quarterbacks. So your skepticism of their reviews would suddenly end if you knew there was a sponsorship in place? Or would that just reinforce your suspicion that bias exists? How can you ever know? If you're so skeptical you're best of just discounting all the staff reviews, as Jude has suggested.
 
Regardless, it comes back to the fact that they don't HAVE to do anything ! There is no SEC regulating what sites like this do, so the fact that they're transparent about anything says something about their ethics & values. Actually the fact that they're even letting this discussion happen says a lot too. Once again, you can't please everyone. You can only try your best and know that you'll always be criticized for not doing more. It's always so simple for the guys telling you what you should be doing.

 

[Claritas]

I'm sure some of us have grasped what you were saying. I chose to ignore it and focus on small, absolutely worthwhile, steps to minimize bias and increase transparency such as returning or buying review samples and disclosing sponsorship.

 

[a_recording]

 
Regardless, it comes back to the fact that they don't HAVE to do anything ! There is no SEC regulating what sites like this do, so the fact that they're transparent about anything says something about their ethics & values. Actually the fact that they're even letting this discussion happen says a lot too. Once again, you can't please everyone. You can only try your best and know that you'll always be criticized for not doing more. It's always so simple for the guys telling you what you should be doing.

 
No, of course I can't compel you to do anything. This isn't me simply just mouthing off about this subject though. I actually have to put my money where my mouth is and do what I am talking about, which is why I decided to stop accepting review units. I don't want to go off and start any witch hunt but at the very least there are forum rules that insist on a certain degree of disclosure. As far as I know, the community has never really publicly discussed reviewer's ethics in one thread, which is again what I wanted to start here. People can always try their best, but people also need to discuss standards and raise them where appropriate and feasible.
 
I appreciate the moderators letting this discussion continue, but this was the discussion I wanted to have in the first place and I am disappointed about what happened to get us to this point.
 
The moderators could compel people to disclose these relationships, but they are already very busy with the rest of their moderation. I don't think anyone actually goes around flagging reviews for not elaborating on following the disclosure rules either.
 
On a side note I would encourage the moderators to think about switching Head-Fi to a semi community moderated system where people vote on comments and comments with enough negative votes will disappear below the visibility threshold. Similar systems are used in other forums and places like Reddit, and it seems to prevent the kind of incivility that moderators are currently busy trying to eliminate. It may or may not lead to similar levels of groupthink but at the very least it makes moderation more transparent since it is in everybody's hands.

 

[spook76]

lol, You're talking about stock analysts that sometimes work for Fortune 500 companies (who make upwards of 6 to 7 figures for their JOB of analyzing such things) recommending the buying and selling of stock in large PUBLIC companies. Comparing that to a site of this size is… well, in my humble opinion, downright silly. What reviewers make that kind of money for their analysis? Do ANY of them make money? What organization on par w/ the SEC regulates the "small, niche website" market? There isn't such a thing, which I'm sure you realize. So basically I think you're comparing apples to oranges. 

As for your example of "this product was manufactured by a sponsor of Head-Fi" - is it really that simple? Once again, what if the company was a sponsor back in 2002 for a 3-month period, for an entirely different product. You expect someone to look that up and include all those details? If you do, OK, that's your prerogative, but I wouldn't expect that from a site like this. It wouldn't be as simple as you're making it seem.

A short disclosure at the beginning or end of a review is all analysts do and that is all I am advocating here on Head-Fi.

Edit: I have edited my post as I hope this can be a constructive conversation to improve Head-Fi.