[billybob_jcv]

I would say he is only tainted if the history of his past reviews is found to be inaccurate by purchasers of products he has reviewed.  You really have to establish a type of trust based on experience with any given reviewer.  You can't just accept a guarantee of the reviewer's integrity from someone else, unless that someone is a person you already trust.  That being said, it circles back around to  your personal experience with the guarantor.  It boils down to this:  Ideally, we should all start out with an offering of the benefit of the doubt.  Our trust level is then adjusted based on our personal experience with any particular individual.  It would be nice to have guarantees, but there are very few of those in life.

I agree - which is why I disagree with the basic premise of this proposal, which is built upon the assumption that a reviewer receiving gear is biased and cannot be trusted.

 

[fiascogarcia]

I agree - which is why I disagree with the basic premise of this proposal, which is built upon the assumption that a reviewer receiving gear is biased and cannot be trusted.

As do I!  

 

[Hutnicks]

Let's say I'm a manufacturer, and I choose to ignore this new model and send out review units anyway. How many of the noble reviewers will simply return the box unopened? What if the box is accompanied by a letter that explains the box contains a pre-release sample of a heretofore only rumored product? Would the reviewer NOT review the advance product? How does that benefit the community, or the manufacturer that genuinely desires feedback on the new product? If the reviewer accepts even *one* instance of this sort of item, is he not then forever tainted by the specter of bias?

Aye, there's the rub. The preproduction test units make the "buy to review" model un workable.

 

[Claritas]

Let's say I'm a manufacturer, and I choose to ignore this new model and send out review units anyway. How many of the noble reviewers will simply return the box unopened? What if the box is accompanied by a letter that explains the box contains a pre-release sample of a heretofore only rumored product? Would the reviewer NOT review the advance product? How does that benefit the community, or the manufacturer that genuinely desires feedback on the new product? If the reviewer accepts even *one* instance of this sort of item, is he not then forever tainted by the specter of bias?

Audio can regulate and establish tolerably high standards as many other industries have.

The alternative is pessimism and maintenance of the status quo using every conceivable exceptional case to argue that improvement is impossible and to ensure that it never occurs.

 

[billybob_jcv]

Audio can regulate and establish tolerably high standards as many other industries have.

The alternative is pessimism and maintenance of the status quo using every conceivable exceptional case to argue that improvement is impossible and to ensure that it never occurs.

In what other industry are there enforced quality standards for subjective metrics?

Show me the standards for how food tastes, or for the color schemes used for clothing, or for how the inside of a new car smells.

There are standards for professions like being a sommelier, but they are simply certifications that show prowess, there is nothing that prevents anyone from working as a wine steward at a restaurant that will hire them, or from opening a wine store.

 

[Magick Man]

There is a model that already works and does not involve soliciting actual funding.  It involves being part of a community or group of friends with access to a variety of goodies.  Send things on loaner tours to each other, or have a few central users who have fairly trusty ears who can take measurements and are generally even handed.  Round out with some subjective impressions.  

Wasn't there an issue where certain people were selling the loaner tour samples they received, the gear just "disappeared"? Some folks are just crummy people and they wait until the most opportune moment to reveal it. We (my company) recently loaned out a very expensive tester/flasher to a smaller business and after a week or so, when they were supposed to send it back, they stopped communicating. Then, after bugging them for another week, because we needed the thing back, they finally decided to talk to us and informed us that the device was "missing". We'd let them borrow several things in the past without incident, but nothing quite that valuable. Ultimately their final statement was, "Well, you have insurance, right?" (They claim they don't and also told us they can't afford to replace it.) We do, but it only covers things that are in our possession at the time. That's a bit of a blow to that part of our business, because R&D runs on a fairly tight budget and we had to buy another, but lesson learned. Now they can't loan out anything.

 

[thievesarmy]

Jude,

A solution might be disclosure. I have enjoyed yours and other administrator's reviews but I have often wondered which manufacturers are sponsors so that a casual reader like me can make an informed decision regarding your review and any possible bias.

As an analogy, the US Security and Exchange Commission requires analysts to disclose if such analyst has a position in the stock or company he or she is discussing to avoid just this problem about perceived bias.

On Head-Fi it would be quite easy for you or any administrator to state somewhere at the beginning or end of a video or written review whether the product being discussed is manufactured by a sponsor of Head-Fi. That way the reader is supplied with the information needed to make an informed decision.

Steve aka Spook76

 
i assume someone has pointed you here alreaedy: http://head-fi.org/sponsors

 

[Claritas]

In what other industry are there enforced quality standards for subjective metrics?

I thought we were talking about not keeping review samples after the review and how to enforce that. If you will it, it is no dream.

i assume someone has pointed you here alreaedy: http://head-fi.org/sponsors

I think you've hit on a problem. My sense is that the list is incomplete.

 

[spook76]

i assume someone has pointed you here alreaedy: http://head-fi.org/sponsors

Yes, but that is not the disclosure I am talking about. If I may continue to use the securities analogy, the US Securities and Exchange Commission requires more than a link to a spokesman's website. While discussing a company or security that person must state AT THAT TIME (on television, radio or other media) whether his or her company has a position in the company or security being discussed.

https://www.sec.gov/answers/analyst.htm

Watch CNBC or Fox Business when an analyst is giving an opinion they are always asked whether their company has a position or possible conflict of interest in the company or security being discussed.

All I am advocating is full disclosure that way the viewer or reader knows of any possible conflict of interest. A_recording's position assumes we cannot make our own decision if we are aware of a conflict.

 

[thievesarmy]

That's the SEC standard. This is a small, private, niche website. I'm not going to hold them to those standards, and they really don't have to disclose ANYTHING to anyone. The fact that they have that sponsor page - even if it's out of date - is a credit to them and is at least a degree of transparency.
 
There is no perfect solution. As Bill Cosby said, "I don't know the key to success, but the key to failure is trying to please everyone." Where would you draw the line? Past sponsor's even if they only did a 3-month campaign in 2001? What about if a company isn't currently a sponsor at the time of a review, but then becomes one? No one will ever be 100% satisfied w/ their system. As Jude stated earlier, if any of this is an issue for you or anyone else, simply discount all the mod & admin reviews. Pretty easy solution IMO.
 
Maybe I'm alone here, but individual biases in reviews, whether from site staff or otherwise, don't bother me at all, because I know there is no such thing as a perfect review - or, perhaps better stated, there's no such thing as a review that will tell me if I'll like something or not. One review alone will NOT convince me of anything. Is eliminating all bias even possible? I don't think so. Maybe Lachlan's idea has some merit, and he's free to explore that elsewhere, but IMHO the drama that has resulted from this thread is completely RIDICULOUS. I have to give Jude credit for even letting this silliness continue (although I also understand that closing it would only create more evil conspiracy theories). Lachlan is taking their moderation of a specific rule & topic to an extreme and asserting they won't allow ANY talk that could negatively affect their sponsors, AND being overly dramatic about it. Anyone that watches his video can see he's clearly clouded (and IMO, misguided) with emotion right now - something I'm sure he'll regret after he has some time to cool off. To be honest I enjoy his reviews and I respect his opinions, but I completely agree w/ Jude on this. 

 

[Claritas]

It would be fairly easy to post a complete and updated list.

 

[billybob_jcv]

Wasn't there an issue where certain people were selling the loaner tour samples they received, the gear just "disappeared"? Some folks are just crummy people and they wait until the most opportune moment to reveal it. We (my company) recently loaned out a very expensive tester/flasher to a smaller business and after a week or so, when they were supposed to send it back, they stopped communicating. Then, after bugging them for another week, because we needed the thing back, they finally decided to talk to us and informed us that the device was "missing". We'd let them borrow several things in the past without incident, but nothing quite that valuable. Ultimately their final statement was, "Well, you have insurance, right?" (They claim they don't and also told us they can't afford to replace it.) We do, but it only covers things that are in our possession at the time. That's a bit of a blow to that part of our business, because R&D runs on a fairly tight budget and we had to buy another, but lesson learned. Now they can't loan out anything.

Yeah, your only recourse in that case would be to sue the guy for negligence because he breached his "duty of care" for your property.

 

[Makiah S]

  Great post but then again no one ever claimed audio to be sane.
 
To paraphrase, have faith in God but trust only your ears.

pretty much, I do like trading cans amoungs't members though as a way for us to get real reviews
 
for example the LCD 2 Fazor kinda sucks... technically it's great. Sonically it's... kinda meh 
 
good post though 

 

[spook76]

That's the SEC standard. This is a small, private, niche website. I'm not going to hold them to those standards, and they really don't have to disclose ANYTHING to anyone. The fact that they have that sponsor page - even if it's out of date - is a credit to them and is at least a degree of transparency.

There is no perfect solution. As Bill Cosby said, "I don't know the key to success, but the key to failure is trying to please everyone." Where would you draw the line? Past sponsor's even if they only did a 3-month campaign in 2001? What about if a company isn't currently a sponsor at the time of a review, but then becomes one? No one will ever be 100% satisfied w/ their system. As Jude stated earlier, if any of this is an issue for you or anyone else, simply discount all the mod & admin reviews. Pretty easy solution IMO.

Maybe I'm alone here, but individual biases in reviews, whether from site staff or otherwise, don't bother me at all, because I know there is no such thing as a perfect review - or, perhaps better stated, there's no such thing as a review that will tell me if I'll like something or not. One review alone will NOT convince me of anything. Is eliminating all bias even possible? I don't think so. Maybe Lachlan's idea has some merit, and he's free to explore that elsewhere, but IMHO the drama that has resulted from this thread is completely RIDICULOUS. I have to give Jude credit for even letting this silliness continue (although I also understand that closing it would only create more evil conspiracy theories). Lachlan is taking their moderation of a specific rule & topic to an extreme and asserting they won't allow ANY talk that could negatively affect their sponsors, AND being overly dramatic about it. Anyone that watches his video can see he's clearly clouded (and IMO, misguided) with emotion right now - something I'm sure he'll regret after he has some time to cool off. To be honest I enjoy his reviews and I respect his opinions, but I completely agree w/ Jude on this. 

Would it be so difficult for a moderator to begin or end a review with "this product was manufactured by a sponsor of Head-Fi". You are right that would be just too onerous. No, a past sponsorship relationship would not have to be disclosed. It is an simple matter of disclosure of conflict of interests at the time of the review.

FYI the SEC standard is applied to all analysts even if they work for "small, private, niche websites"

 

[billybob_jcv]

Would it be so difficult for a moderator to begin or end a review with "this product was manufactured by a sponsor of Head-Fi". You are right that would be just too onerous. No, a past sponsorship relationship would not have to be disclosed. It is an simple matter of disclosure of conflict of interests at the time of the review.

FYI the SEC standard is applied to all analysts even if they work for "small, private, niche websites"

Are we talking about reviews by Jude & the head-fi staff, or about any random thread with a user review? I will say again: Anyone can post a review. Anyone.

I dislike the Audio-Technica ATH-M50. I find it lifeless and lacking in any kind of sonic character. I think it is vastly overrated, especially for the price.

There - is that a review?