[Uncle Erik]

Most skepticism is reactive - we respond against miraculous claims and debunk.

Let's turn this around.

What tools and methods do you use to determine whether a claim is true? What is is your skeptic's toolbox to evaluate a claim? What is your starting point for debunking nonsense?

One of my favorite starting points is doing a cost to retail price analysis. If $20 of parts is being sold for $300, red flags pop up.

What raises red flags for you?

Let's keep attacks against particular manufacturers out of this. Some are gladhanding frauds and deserve to be flayed, but let's talk about what you use to determine snakeoil and bogus claims.

Like Carl Sagan's Skeptic's Toolbox, let's talk generally about what you use to ferret out fake claims and phony products. Not just the snakeoil, but also horribly overpriced products.

I started to compose a post on evaluating the actual cost of products, but need to spend some time fleshing it out. I'll post it when done.

Instead of railing against certain products, I think we need to equip people to evaluate claims on their own. Skepticism and a demand for valid results should be instilled in people; it should not be an argument over each product that rolls out.

So let's have a discussion about the tools and tools you use to determine if something is real or bogus. Keep manufacturers and specific products out. I want to know how you begin your evaluation then how you determine whether or not a claim is legitimate.

 

[nikp]

What raises the red flag? The company's past history. 

 

[Uncle Erik]

Not necessarily. Some companies have great customer service yet produce phony products at inflated prices.

I want to know the methods to determine which products are bogus and methods for evaluating phony products.

If someone dismisses a product as snakeoil, how did you reach that conclusion?

How can you equip others to reach the same conclusion?

Most of this stuff is easily debunked. Help others learn to do the same.

 

[nikp]

Quote:

Not necessarily. Some companies have great customer service yet produce phony products at inflated prices.
I want to know the methods to determine which products are bogus and methods for evaluating phony products.
If someone dismisses a product as snakeoil, how did you reach that conclusion?
How can you equip others to reach the same conclusion?
Most of this stuff is easily debunked. Help others learn to do the same.

It's a bad thing to generalise but if you're going to be a bit more specific, I might be able to help. Like Ebay? Black Market Goods? Well I did plentiful of research and go to the point of origin. Sources and connections offer great help too.
 

 

[Currawong]

I don't think that a product's part costs is really a good guide. If that were the case, you'd never buy clothes again, from anywhere! You'd sew your own..
 
Anyway, what raises my eyebrows is a lack of any THD or S/N ratio measurements in the specs of an amp or DAC. While I know well enough that comparing figures is meaningless (as the noise floor is technically only as low as the highest distortion peak, even if for most of the audible spectrum it might be considerably lower) but nothing at all is rather suspicious.

 

[Parall3l]

Graphs, graphs and more graphs.  

 

[Steve Eddy]

Quote:

I don't think that a product's part costs is really a good guide. If that were the case, you'd never buy clothes again, from anywhere! You'd sew your own..

HA!
 
No Schiit!
 
se

 

[Head Injury]

Lack of measurements
Misleading measurements (THD with no THD+N, frequency range but no mention of variation, measurements taken with unrealistic loads or at low voltages)
Audiophool claims like "musical" output impedance and roll-off
Lots of copy about how the product fixes an issue which isn't an issue ("digititis" and jitter for example)
Impossible or overexaggerated claims ("Restores the lost information of your MP3s")
Any measurements published that don't look anything like the measurements a third party finds
Any mention of cables

 

[Willakan]

What puts me off is when the PR people shove loads of reasons as to why their product isn't as good as the competition in their marketing literature. (DACs that refuse to use opamps, that sort of thing.)
 
 

 

[tvrboy]

Well, as a marketing major and former colleauge of some great salesmen, I would say the only defense against the dark arts is a thorough knowledge about the product.
 
You can make almost anything sound plausible. People necessarily have a limited amount of knowledge of most things, and it's easy to come up with marketing copy that sounds believable. It's easy to write your own testimonials. It's easy to do 10 DBTs and choose the two in which your product was recognizeable, and only use that in your advertising. Supplement companies do that all the time. Do enough small scale studies of a diet supplement, and you will get one where you have an average weight loss. Diets all have a "scientific study" in their favor. Put any group of obese people on a strictly supervised diet with free coaching and you will get some average weight loss in a short period ... never mind whether they gain it back after the study is over. Send your product to 10 magazines to get reviewed and you will get enough positive ones to use in your advertising. The bottom line is you are gonna get taken some of the time. We can't know everything about every product category. Better figure out where it's most important to you not to get taken and learn all you can about it. Reading books on sales techniques helps too, as you learn how to recognize the worst kind of salesmen and avoid them.

 

[firev1]

The best defence is some or good knowledge of electrical theory, works on everything and anything in defence against the dark arts. Published measurements using an audio scope from a manufacturer would be good too, if ASUS can do it(ie. Essence One but they forgot the reference voltage) why not others, the report maybe in the form of a scanned copy for download especially when the amps are costing say more than 1k usd. Having the balls to do so would provide surefire sales.
 
That being said it would be great if more fellow head fiers learn how to really analyse measurement data, that too would help in filtering out snake oil and other overpriced products. 

 

[InnerSpace]

I approach it from the other end - not just, "Why would a manufacturer lie?" but, "Why do we want to be lied to?"
 
For some people music is a very emotional thing, leading to joy and pleasure and even a kind of ecstasy.  Some of those people love the high so much they chase a higher high.  That motivation is very powerful - they want the higher high so much they do 90% of a manufacturer's marketing work for him.
 
In other words - be very aware of the very human, very natural, almost inescapable, incredibly powerful placebo effect.  Put it front and center, and demand of yourself that you think rationally.

 

[dBel84]

I do like the topic of this discussion. 
 
Price point is hard for me , as a DIYer I know what it costs to build someone else's designs. But I also understand that bringing a product to market costs a lot of money. If I am not mistaken Industry standard predicts about 5x for retail in order to pay for R&D, manufacture, stock on hand, dealers cuts, shipping, support, advertisement (including demo units ) etc etc and this is only to make the product self sufficient with marginal profit to re-route to building the business. This is why Ti Kan ( AMB ) said that the B22 would be a problem for the commercial market - I would guess at $8K or higher   ( ave build cost of about $1500 with a reasonable chassis ) . Price is more relevant when you hit the extreme end , I recall the 80k gbp ongaku which was a hand built amp ( transformers hand wound etc ) by the zen master himself - I still could not justify this to myself. 
( as an aside, I suspect that many of our headphone vendors operate below this market standard - the Pinnacle comes to mind, I am damn sure it costs more than $2K to manufacture ) 
 
snake oil is probably my red flag more than measurements. Extravagant claims regarding "new" design with flamboyant monikers and pseudoscience justification. 
 
The defense is knowledge as stated above, know your subject well enough not to be hood winked by technical jargon which may or may not be applied correctly. 
 
..dB

 

[Radio_head]

The hallmarks of an actual snake oil salesman back in the days were: 1) the salesman's charisma and 2) the product's packaging.  3) plants in the audience who experienced ___ miraculous health benefits
 
1) If the product's makers (or salesmen to a lesser degree) spend more time using buzzwords, platitudes, and cliches than discussing components and hard facts, color me skeptical.
2) If I see an amp priced at 3K that I estimate the casework to be 1.5K on (from my own manufacturing knowledge), I'm a skeptic. (of its sound quality, not its aesthetics.)
3) If I see initial reports/reviews/impressions from websites/reviewers/forum members who I know to have an interest/are friends with/are a little too enthusiastic in general about the companies products, followed up by similar reviews by newbies who I consider to just be jumping on the bandwagon, while I do not think the product is bad, there is no indication to me that it is good.  So I would be skeptical that X new DAC with Y proprietary technology does indeed clear up your psoriasis.
4) I trust my own ears.  If it doesn't sound good in my setup then I can say confidently it doesn't sound good in my setup and be mildly skeptical of it sounding good in others rigs. 
 
As I am not an electrical engineer and don't really have as much time as I'd like to learn about topography, layout and implementation of chips, opamps, tubes, etc, I have to rely on others to for that to some degree.  When a number of these users chime in on a product and lambaste it from an electrical engineering standpoint, it would also raise concerns for me, although I would still try the product myself before pronouncing judgement.

 

[Rdr. Seraphim]

Uh, common sense?