or Connect
Head-Fi.org › Forums › Misc.-Category Forums › Members' Lounge (General Discussion) › OPINION: Review Units Hurt the Audio Community
New Posts  All Forums:Forum Nav:

OPINION: Review Units Hurt the Audio Community - Page 5  

post #61 of 149
My work depends on having good working relationships with other people in my company. One of the axioms I try to instill in my employees is: "Assume good intentions." This means that there needs to be an element of trust in order to have a relationship that allows two or more people to work together. It doesn't matter whether the relationship is provider-customer, management-employee, peer co-workers, etc - without some level of trust, the relationship will fail. That doesn't mean that you simply believe everyone and everything - but it does mean we should believe that each person is doing what they believe is right. If one or both parties is assuming evil intentions, they will never come together and will not be as productive as they could be.

I think Jude's rules are intended to proactively reduce problems on this site. I don't think they are an evil capitalist plot to allow sponsors free reign to fleece the unwashed masses.

I think a_rec genuinely believes he can improve the audio reviewing industry. I don't think he is purposely attempting to attack the sponsors of this site or bring head-fi down.

Both of those statements assume good intentions on the part of Jude & a_rec. Neither statement prevents me from agreeing or disagreeing with either Jude or a_rec. What it does do is allow me to focus on the facts and the real meat of their arguments, rather than on any possibly nefarious underlying motives. It makes the debate MUCH more logical and productive. I happen to think Jude is right, but that doesn't keep me from believing in a_rec's good intentions - even if I also think they are wrapped around a fundamentally flawed idea and being delivered without consideration for constraints imposed by the real world.

I tell my employees that if they cannot bring themselves to assume the good intentions of someone, then perhaps this is not the right job for them. In the same way, I would say that if you cannot assume the good intentions of this site, then perhaps this site is not where you should be. It's also the difference between believing that sponsors come to head-fi because they want to participate in the head-fi community and get access to a population of audio enthusiasts, versus believing the sponsors come here because they believe Jude will write good reviews for them.
post #62 of 149

Sticking my big head in just to say that even though Shure, Stax and other brands mentioned in official editorial content is not always directly sponsoring the site, retailers who sell those brands are. So in that way, skewing the information in favor of those products would still be beneficial to both Head-Fi and (some of) the sponsors.

 

Also, I think that there can be an incentive to be opportunistic about editorial content. For instance where one gives a glowing review of say Meridian which is not a sponsor in hopes that the company will see an effect from or notice that in some way, and then there is something to base a potential new relationship and sponsorship on.

 

Also, there is the bias of social bonds. I don't know about you gatekeepers, such as Jude, Tyll, Hifiguy, ... But I have the impression that you guys have personal relationships with some of the manufacturers and retailers (maybe strictly professional, but on an ongoing basis). It's probably unavoidable in a niche of a niche, but then again, it introduces biases. I think most people who aren't psychopaths would find it hard to say something that might work against a good (professional or personal) social relationship.

 

The essence of what a_recording tries to do, is to eliminate this coupling with reviewer, manufacturer and retailer, because it does create biases.


Edited by Norway - 7/1/14 at 8:40am
post #63 of 149
Some of you guys sound like you have never been in a business relationship. We seem to have it in our heads that reviews would be wonderful if only the reviewers could have free access to any gear they want without the manufacturers being involved. I just don't buy it. Each reviewer still brings their own biases to the review. In the end, it doesn't really matter a hill of beans where the review unit came from. If the reviewer wants to write a good review - for whatever reason they can. I suspect that there is an underlying belief that all will become clear once this pesky "bias" issue is solved. I call BS to that. There will *still* be just as many lists of the top 10 headphones as there are reviewers, and there will *still* be absolutely no agreement or correlation between them. And that's just the top 10! Below that it gets even worse.
post #64 of 149
Quote:
Originally Posted by billybob_jcv View Post

Some of you guys sound like you have never been in a business relationship. We seem to have it in our heads that reviews would be wonderful if only the reviewers could have free access to any gear they want without the manufacturers being involved. I just don't buy it. Each reviewer still brings their own biases to the review. In the end, it doesn't really matter a hill of beans where the review unit came from. If the reviewer wants to write a good review - for whatever reason they can. I suspect that there is an underlying belief that all will become clear once this pesky "bias" issue is solved. I call BS to that. There will *still* be just as many lists of the top 10 headphones as there are reviewers, and there will *still* be absolutely no agreement or correlation between them. And that's just the top 10! Below that it gets even worse.

 

Yes, each reviewer brings his own biases, but what a_recording brings up is a systemic bias. It is structural by nature and probably works in most if not all professional reviews. Individual biases come in different variations, whereas this structural bias is likely to be extremely strong and will probably dominate the review more so than individual biases. As a reader you can also eliminate a person's individual biases by (1) noting them and interpret their content in that light or (2) stop reading their content.

post #65 of 149
Quote:
Originally Posted by billybob_jcv View Post
...
I think Jude's rules are intended to proactively reduce problems on this site. I don't think they are an evil capitalist plot to allow sponsors free reign to fleece the unwashed masses.

 

True capitalism is wonderful, but it requires transparency. Read any textbook on economics and you will see that symmetrical information is essential to a well oiled free market economy. Crony capitalism is based on a select few dictating all encompassing policies of both fiscal and monetary nature, and this often goes on behind the scenes by the gatekeepers. This is information asymmetry and it is not efficient. It does not benefit the market, but select groups which the gatekeepers choose.

 

a_recording is trying to achieve transparency.


Edited by Norway - 7/1/14 at 9:09am
post #66 of 149

One facet that is overlooked is the unintentional bias in the comments and personal reviews of members who have invested in equipment and therefore give positive comments as a means of subconsciously justifying their own investment.  If you want to take this rational to that extreme, then you might as well not believe anything anyone writes.  I side with those that believe everyone on this site come in with the best intentions to help others make a good choice.

post #67 of 149
Quote:
Originally Posted by Norway View Post

Sticking my big head in just to say that even though Shure, Stax and other brands mentioned in official editorial content is not always directly sponsoring the site, retailers who sell those brands are. So in that way, skewing the information in favor of those products would still be beneficial to both Head-Fi and (some of) the sponsors.

Also, I think that there can be an incentive to be opportunistic about editorial content. For instance where one gives a glowing review of say Meridian which is not a sponsor in hopes that the company will see an effect from or notice that in some way, and then there is something to base a potential new relationship and sponsorship on.

Also, there is the bias of social bonds. I don't know about you gatekeepers, such as Jude, Tyll, Hifiguy, ... But I have the impression that you guys have personal relationships with some of the manufacturers and retailers (maybe strictly professional, but on an ongoing basis). It's probably unavoidable in a niche of a niche, but then again, it introduces biases. I think most people who aren't psychopaths would find it hard to say something that might work against a good (professional or personal) social relationship.

The essence of what a_recording tries to do, is to eliminate this coupling with reviewer, manufacturer and retailer, because it does create biases.

The idea that a_rec would somehow be completely unbiased is, to put it lightly, contrary to what I've seen of his postings around here for the last few years, and that's without sponsor incentives. redface.gif
post #68 of 149
Quote:
Originally Posted by billybob_jcv View Post

Some of you guys sound like you have never been in a business relationship. We seem to have it in our heads that reviews would be wonderful if only the reviewers could have free access to any gear they want without the manufacturers being involved. I just don't buy it. Each reviewer still brings their own biases to the review. In the end, it doesn't really matter a hill of beans where the review unit came from. If the reviewer wants to write a good review - for whatever reason they can. I suspect that there is an underlying belief that all will become clear once this pesky "bias" issue is solved. I call BS to that. There will *still* be just as many lists of the top 10 headphones as there are reviewers, and there will *still* be absolutely no agreement or correlation between them. And that's just the top 10! Below that it gets even worse.

Agreed. I'll admit it, I'm biased. When I see an ortho or stat I automatically assume it's going to sound more balanced and tonally even, with superior DR, to a traditional dynamic headphone. I'm aware that the bias exists and try to counter it, but it does color my initial reaction. Removing corp sponsors doesn't mean you remove bias, it simply means that you're open to other forms. ex. You can lean so far over into antiestablishmentarianism* that it becomes a huge bias, and then you're only giving 5-star awards to obscure brands that no one else in the hobby has even heard of, while penalizing big manufacturers just because they're big.








(* my $10 word for the day)
post #69 of 149
Quote:
Originally Posted by mutabor View Post
 

I've just watched a_recording's last video. He is blaming Head-Fi in promoting greed in buying more headphones. It's like blaming Amazon that they make Americans puppets of corporate industry and capitalist consumerism. It's interesting to hear that from people who are utter examples of consumerism themselves.

 

Agreed. No one forces you to go out and buy the next best thing all the time. We can only blame ourselves, and to some extent, the media, for planting these ideas in our minds.

post #70 of 149
Quote:
Originally Posted by Magick Man View Post

You can lean so far over into antiestablishmentarianism* that it becomes a huge bias, and then you're only giving 5-star awards to obscure brands that no one else in the hobby has even heard of, while penalizing big manufacturers just because they're big.

*cough* 6 Moons *cough*
post #71 of 149
Quote:
Originally Posted by billybob_jcv View Post

We seem to have it in our heads that reviews would be wonderful if only the reviewers could have free access to any gear they want without the manufacturers being involved. I just don't buy it. Each reviewer still brings their own biases to the review. In the end, it doesn't really matter a hill of beans where the review unit came from. If the reviewer wants to write a good review - for whatever reason they can. I suspect that there is an underlying belief that all will become clear once this pesky "bias" issue is solved. I call BS to that.

The regrettable persistence of personal bias isn't a compelling argument for not making an effort to overcome potential sources of bias. After writing a review, a reviewer should decide either to buy or to return the review sample.
post #72 of 149
Quote:
Originally Posted by Claritas View Post

The regrettable persistence of personal bias isn't a compelling argument for not making an effort to overcome potential sources of bias. After writing a review, a reviewer should decide either to buy or to return the review sample.

My point was that some of the discussions seemed to be saying this would "fix" the nature of reviews. I disagree and claim that it would not have any appreciable effect. Either you believe the reviewer to be genuinely providing their true opinion, or you do not. If you believe a professional reviewer would allow himself to become significantly biased based on the source of his gear, then nothing that reviewer would do to assuage your doubts would help. You can certainly try to build as bias-free an environment as you can, and you can make it as transparent as you wish - and ultimately it still comes down to whether the reader trusts the reviewer. Do you trust the headphone reviews at Consumer Reports? No? Why not? There is *no* bias at Consumer Reports. There's also very little audiophile knowledge at Consumer Reports, so we all still dismiss their reviews. Again, my point is simply that *this* proposal may or may not help - but it won't magically transform audio reviews into golden eggs of knowledge.
post #73 of 149
Quote:
Originally Posted by billybob_jcv View Post


My point was that some of the discussions seemed to be saying this would "fix" the nature of reviews. I disagree and claim that it would not have any appreciable effect. Either you believe the reviewer to be genuinely providing their true opinion, or you do not. If you believe a professional reviewer would allow himself to become significantly biased based on the source of his gear, then nothing that reviewer would do to assuage your doubts would help. You can certainly try to build as bias-free an environment as you can, and you can make it as transparent as you wish - and ultimately it still comes down to whether the reader trusts the reviewer. Do you trust the headphone reviews at Consumer Reports? No? Why not? There is *no* bias at Consumer Reports. There's also very little audiophile knowledge at Consumer Reports, so we all still dismiss their reviews. Again, my point is simply that *this* proposal may or may not help - but it won't magically transform audio reviews into golden eggs of knowledge.

 

As far as I see it there are two main groups of biases, those one is conscious of and decide to act upon or not, like incentives to get more review samples from company x by only giving a positive review of product y, even though doing so means not being completely truthful.

 

Then there is the large part of biases that are unconsciously implanted and not acted upon consciously. However, the end result is that one's conclusions are distorted by them without one even knowing.

 

So then implicitly you discard the possibility of subconscious biases in the current review system?

post #74 of 149

mutabor's takeaway from the video is very different from mine. I interpret a_recording's video as a message of principles.

 

The principles I gather are the right to discuss potential structural (unconscious and conscious) biases and how they might color the perception of professional reviewers with the current review model on this platform, as this is the biggest community and where the elimination of such biases might be to the best of the community at large. By not allowing for such discussions the intention can not be to do what is best for the community.

 

I agree with this. Just because we have a lot of personal biases, does not mean we should not talk about or try to avoid the biases introduced under status quo. We should work to eliminate the most dominating ones, and I agree with a_recording that the structural biases are the most dominating of all, because it affects everyone who operates within the current model, which is pretty much every professional audio reviewer out there.

 

It would be wonderful to hear what professional reviewers such as Tyll has to say in this matter, because I am absolutely positive that every rational human being trying to review gear are faced with the incentive structure and, in turn, the ethical dilemmas it presents.

 

Edit: Added link to post


Edited by Norway - 7/1/14 at 2:35pm
post #75 of 149
I think you are making the leap that only "unbiased" reviewers (ie, reviewers subscribing to the proposed method) would provide reviews. But that will not be the case. Other reviewers would still write reviews and possibly continue to engage in the "biased" behaviors. How exactly would other reviews be prevented? Would the average head-fi visitor only read the reviews of the "trusted" reviewers? What if I simply start a site and start posting reviews - and the mfrs start sending me gear? Or, what if I do that and then I *claim* that although I'm not part of the "unbiased commune", I also don't take compensation, so my reviews are good too. Debate all the semantics of biases that you wish - this entire thing is simply not a workable solution in the real world. It benefits some small subset of reviewers, for no tangible gain to the community at large.
New Posts  All Forums:Forum Nav:
  Return Home
This thread is locked  
Head-Fi.org › Forums › Misc.-Category Forums › Members' Lounge (General Discussion) › OPINION: Review Units Hurt the Audio Community