Head-fi Philosophy; or Adding Honesty and Objectivity to the Forum

[seacard]

I'm spending less and less time on head-fi these days, and I was trying to figure out why that is (since I don't love music any less). Obviously part of the reason is that after finding the sound that you like, this place is a bit less exciting. But there are two things that really bug me about this place that have pretty much led me to abandon it, and I am curious as to why these issues exist here.

(1) The "can't say anything bad about a manufacturer" rule [edit: I am in no way suggesting that this is an official head-fi rule or is somehow imposed by the moderators]. There is nothing more frustrating to me than people who buy a product, post incredibly positive reviews of it, beautiful pictures, call it the greatest thing ever, and then, a month later sell it. Sure, initial reactions may be wrong, so let's accept that. But then, months and months later, they post that the product was defective, that the left and right channels were switched, that something wasn't working, that it was scratched, etc. Why is it so improper to criticize a manufacturer just because they are a small company? I don't want to pick on any posters or manufacturers, so I'm going to leave it at that. In any event, I've come to the point where I don't take anything away from a review, so there's really no point in reading them.

(2) But much more important, the "no blind testing" rule. This may be the most ridiculous thing I've ever heard of. Maybe I'm more demanding than others, but the idea of spending even a dollar when it cannot be shown that the more expensive item is better is unacceptable. But we are talking hundreds, or thousands of dollars on amps and cables. This should not only be allowed, but I believe blind testing should constitute most of the discussion here. Blind testing should be the point of mini-meets: 10 people get together and take turns trying to identify which amp they are listening to, which cable, whether they are listening to a balanced set-up. I want to see the results of these meets (e.g. 8/10 people were able to identify and MPX3 in comparison to HeadRoom Desktop). This is meaningful information. When I buy a new product, I always do blind tests to see if I like it better. I have often times liked a $300 amp over a $5000 amp. I have not once been able to tell apart a cheaper cable versus a very expensive one. But when people tell me that "this $3000 amp just blows away my old $500", what does that mean to me without a blind test (doesn't have to be double-blind, just blind). Obviously you can't do this well with headphones, but with amps, sources, and cables it's so easy. Until that day, head-fi is largely useless, especially for anyone who has studied basic psychology and understands the placebo effect, or anybody that has an advanced engineering degree.

Is it just me, or do others feel this way? And if not everyone is satisfied with "Nordost Valhalla cables make my Grover cables sound like trash," or "I don't need a blind test to tell the difference because the difference is obvious to anyone that is not deaf," maybe we should push for a separate sub-forum where sound is analyzed objectively?

 

[Superpredator]

I think your criticisms of this place are fair. However, I don't necessarily think that the objectivity that typically comes with double-blind testing can fairly be applied to audio. Because something may win out in a DBT does not make something truly "better," but I think it could absolutely add another level of information. I think the psychological factor is definitely an issue. I've simply tried to learn who I can trust here and who I can't. Maybe a separate forum here with ONLY double-blind testing, while keeping the others DBT-free?

 

[marvin]

Quote:

Originally Posted by seacard The "can't say anything bad about a manufacturer" rule. There is nothing more frustrating to me than people who buy a product, post incredibly positive reviews of it, beautiful pictures, call it the greatest thing ever, and then, a month later sell it. Sure, initial reactions may be wrong, so let's accept that. But then, months and months later, they post that the product was defective, that the left and right channels were switched, that something wasn't working, that it was scratched, etc. Why is it so improper to criticize a manufacturer just because they are a small company?

Not too sure that this is a Head-Fi only issue.

If I were looking to sell a known defective item without listing the defects, I sure as heck wouldn't tell anyone that it was defective in other threads before selling it.

/Note: I don't sell defective items without noting the operational condition.

 

[raymondlin]

The people who reviews a product mostly have ALREADY paid for it, thereforee for the sake of self padding on one's own back or some kind of justification to one self will automatically praise it, at most the most critical comment would be "average" as a description.

The blind test is that when that happens, say in a meet. You are practically guaranteed that someone will say some cheaper headphones is better than something that cost 10 times more. But you don't want to insult someone by doing so, and the awe of using something expensive takes over, and your brain will tell you it is better when a blind test might suggest otherwise.

 

[Superpredator]

Quote:

Originally Posted by marvin /img/forum/go_quote.gif Not too sure that this is a Head-Fi only issue.

Actually I'd say this is less an issue at Head-Fi compared to other marketplaces. I have far more confidence buying an item here than Craigslist or eBay.

Quote:

Originally Posted by marvin /img/forum/go_quote.gif If I were looking to sell a defective item, I sure as heck wouldn't tell anyone that it was defective before selling it.

Noted for the next time you put something up for sale.

Quote:

Originally Posted by marvin /img/forum/go_quote.gif /Note: I don't sell defective items without noting the operational condition.

 

[seacard]

Quote:

Originally Posted by Superpredator /img/forum/go_quote.gif I think your criticisms of this place are fair. However, I don't necessarily think that the objectivity that typically comes with double-blind testing can fairly be applied to audio. Because something may win out in a DBT does not make something truly "better," but I think it could absolutely add another level of information. I think the psychological factor is definitely an issue. I've simply tried to learn who I can trust here and who I can't. Maybe a separate forum here with ONLY double-blind testing, while keeping the others DBT-free?

I agree that we can't objectively say something sounds better. But blind testing (DBT is just not worth the trouble, and would take a lot more controls) could tell us whether two items cannot be reliably told apart. If a number of people do a cable or amp or source comparison and cannot (statistically speaking) accurately say which one is which, I think that's important to know. Sure, there will be others out there who will say "My ears are better than everybody else's," and so this information will be irrelevant to them, but there are many (like me) who would find this information very useful, especially because it's impossible to try all the amps and cables that are out there. For example, Markl's review of the RP010 would have so much meaning to me if he could blindly identify the RP010 in a blind test against RPX-33 ten times out of ten, each time being able to pinpoint what the RP010 does better. Whereas I now simply ignore threads like that.

Your point about finding a person you agree with is a good one, but can only get you so far. I usually tend to agree with Wmcmanus, but I've also noticed some discrepancies. There are people that love the R10 as much as I do, but I hate the L3000 whereas they consider it as one of their favorites.

 

[senns&nonsense]

...about the DBT sentiments. Especially as regards cables. There are cretainly people (Skylab comes to mind) who you come to trust because of the thoroughness and quality of their experience/postings/aesthetic sense/literate manner, but DBT would certainly be very useful in a forum like this as well.

 

[Veniogenesis]

I think most of the older Head-Fiers have been aware of these problems. As a matter of fact, these problems might appear as just the tip of the iceberg to a number of the older generation. It's an iceberg of which much darker facets exist.

But I would keep optimistic. The following might sound like a paradox, but it's true when it comes to online forums. Basically I think Head-Fi has matured to a point where many views and people exist, such that passion (and other specific motives) has taken over and masked reality. When a forum is small, camps of ideals and opinions are specific in their mindset but are aware of those of others at the same time. These members had found out for themselves what they preferred and stuck with it. Those were the days when people were wary. Things were new. People opened up their wares, dissected them, questioned things, scoffed at certain things, and settled down. When a forum turns large, members no longer discover on their own nor raise doubts. They believe in the structure of a established camp and use it to boost their confidence and promote their lack of curiosity. If others say it's good, it must be. It's psychology as you've mentioned. And as forums turn large, many of those that are satisfied or have found what they believe to be the truth laugh deep inside and stay quiet and inactive in their expressions of opinions. And those who are vocal, become the dominant camps or more specifically, conspicuous camps. Some that would not have appeared to be excessively passionate fanboys now appear as so in this environment where new members that are more wary no longer know where to look but can only question if all that is said is true. Few will stand up to answer for those that have already found the answers themselves express apathy in certain debates and do not wish to further exacerbate the hidden conflicts throughout the community.

Sorry for the rant. I had nothing to do. And I probably make no sense whatsoever.

 

[seacard]

Quote:

Originally Posted by marvin /img/forum/go_quote.gif Not too sure that this is a Head-Fi only issue. If I were looking to sell a known defective item without listing the defects, I sure as heck wouldn't tell anyone that it was defective in other threads before selling it. /Note: I don't sell defective items without noting the operational condition.

But often times the product can be returned, or exchanged, yet people still would never criticize some of the favorite manufacturers here. If a EMM Labs CD Player came with scratches, or the buttons didn't match up, I am quite certain that this information would come out immediately.

Quote:

Originally Posted by raymondlin The blind test is that when that happens, say in a meet. You are practically guaranteed that someone will say some cheaper headphones is better than something that cost 10 times more. But you don't want to insult someone by doing so, and the awe of using something expensive takes over, and your brain will tell you it is better when a blind test might suggest otherwise.

I've never seen a blind test at a meet. Also, just because somebody picks a $300 heaphone over a $3000 one does not disprove the placebo effect, as it can work in reverse ("I don't believe a cable can make a difference, so this $3000 one can't be better than this $300 one"). It has to be done blindly. And it's so easy to do.

 

[Febs]

Superpredator, I'm not sure I follow your comment that "something may win out in a DBT ..." The purpose of a double-blind test is not to determine whether A is better than B, but merely to determine whether A is different from B. In the audio context, a DBT cannot be used to determine whether something is "better" than something else. However, DBT testing can be used to answer the threshold question of whether the person who is making the subjective comparison can actually hear a difference.

I do tend to agree with seacard that DBT discussion could serve a useful purpose. However, I've seen enough of those discussions degenate into to flamefests to understand why the forum has implemented the "no DBT" discussion rule with respect to cables.

 

[seacard]

Quote:

Originally Posted by Febs /img/forum/go_quote.gif I do tend to agree with seacard that DBT discussion could serve a useful purpose. However, I've seen enough of those discussions degenate into to flamefests to understand why the forum has implemented the "no DBT" discussion rule with respect to cables.

That's why I suggested a separate forum, a forum designed for people who believe in science, believe in measurements, believe in statistics. I, for example, have no idea whether cables can make a difference. I have never heard it yet, but maybe I've been trying the wrong ones. I would love to see a well-designed experiment that tells me that Cable X is statistically different from cable Y, that 9/10 people could tell them apart, and 9/10 preferred the more expensive one (again, this doesn't prove that one is better than the other; just that a stastically significant portion of listeners preferred it). If that is done, then I would be happy to spend any amount of money on that cable.

 

[cantsleep]

when the review is only full of positive comments, i dont believe.

that is like.. lets all spend and die together

 

[jjhatfield]

Quote:

Originally Posted by raymondlin /img/forum/go_quote.gif The people who reviews a product mostly have ALREADY paid for it, thereforee for the sake of self padding on one's own back or some kind of justification to one self will automatically praise it, at most the most critical comment would be "average" as a description.

I think this is the purest truth we can get out of sensational early reviews of a product. Just check out the "I hate waiting for parcels" thread. In addition to the money spent, there is all the time spent researching and thinking about the product beforehand, then the time spent waiting for it to be built, then the shipping, and how insane one can get when UPS/Fedex/USPS/DHL messes up your shipment, and then you get the product... and its the greatest thing to ever happen to audio in history ever!!!!!!

Of course, there are a lot of newer members who stumble in here, bewildered, just looking for a <$100 set of headphones to replace their iBuds (read: me less then a year ago), and they get caught up in the excitement, the culture, the "sorry about your wallet", and they naturally want to be a part of the group. They get a few hundred posts under their belt, and suddenly think they know all about audio. Then these incredibly unqualified individuals write reviews. This is fine for me and others who recognize this, but not fine for the next newbie who stumbles upon this review.

The thing is, when a new can or product comes out, even if it gets glowing reviews and has a dedicated thread with hundreds of enthusiastic new owners bumping it everyday, I still have to ask someone backchannel who knows what they hell they are talking about (read: not me) if the product really does have potential. Folks who actually know about how audio gear works can usually just check out the innards of a product and tell you if it is something cool or innovative, or if it is a derivative design with poor workmanship.

And not being able to criticize vendors is silly. A free market depends upon free communication. If a vendor chooses to treat a customer poorly, they do so at their own peril.

 

[jjhatfield]

Quote:

Originally Posted by cantsleep /img/forum/go_quote.gif when the review is only full of positive comments, i dont believe. that is like.. lets all spend and die together

And cantsleep basically summarize my entire post in two sentences.

Perfectly said.

 

[marvin]

Quote:

Originally Posted by Superpredator /img/forum/go_quote.gif Actually I'd say this is less an issue at Head-Fi compared to other marketplaces. I have far more confidence buying an item here than Craigslist or eBay. Noted for the next time you put something up for sale.

If I did do that, I certainly wouldn't be outing myself.

EDIT: Misunderstanding. Sorry 'bout that Superpredator.