Are headphone reviews essentially useless? - Page 3  

I need to find it. I'm a bit wary of it because it was done by yet another anon blogger and it involves the O2.

Quote:

Originally Posted by dancingpotato 

They provide 4 objective measured parameters:

 

1. Frequency response

2. Distortion

3. Tracking

4. Isolation

 

and a few subjective parameters such as comfort ...

 

What other measurable data might be important ?

1. + 2. The frequency response and distortion measurements don't go below 100 Hz. So you have no information on how the low bass behaves. May headphones have problems in the low bass, which show themselves in high distortion figures and severe bass roll off.

3. Tracking can be taken from the FR.

4. OK, isolation is important for portable use.

Take a look at the links I posted. Some of them have square wave responses (how to read them), CSD plots, step and impulse responses and measurements that go below 100 Hz.

Not entirely useless. Things like comfort, durability, photos or even extreme comments such as "weird-sounding"/"no bass" give indications of what to expect from a headphone. Objective measurements can be fun to look at too - some may work, some don't.

A reviewer's bias (thanks to donated product samples) and consumer defensive attitudes are the things to watch out for.

Quote:

Originally Posted by Currawong 

 

Such perceptions could be related to the presence of small amounts harmonic or inter modulation distortion going by impressions I've read compared with measurements of the same item.

Maybe for amps and transducers but for competent CD players the levels of IMD and THD are normally vanishingly low. The research on distortion thresholds is patchy but always places the threshold well above what a normally competent CD player can manage. The blind tests I have seen for CD players (to date) never show an audible difference unless the measured difference is comparatively large. Of course you can engineer in these differences such as absurd roll-offs or uneven FR (if you add some badly designed tube stages) 

Quote:

Originally Posted by whitedragon551 

Everything is subjective in a sense.

 

Everything is not 'subjective in a sense'.

This is like saying 'left is right, in a sense'.

There is subjective, and there is objective, and never the twain shall meet.

Objective ultimately means 'the collective opinion of everybody in the world'.

Objective is what you get if you parade everybody in the world past a thermometer, ask them to take a reading, and take an average of the readings. The result you get becomes 'the objective truth'. That's how hot the thing you're measuring is. It's the temperature, beyond any question of argument.

That is the meaning of the word 'objective'. It's what you get when you've done everything possible to take the 'subjective' out of a judgement call.

Most people would agree that you don't need to poll everybody in the world, it's probably enough to get 100 people to look at the thermometer. Or less.

Until you've got this idea into your skull, there's not much point in even trying to have a discussion on the subject.

OBJECTIVE is SUBJECTIVE with all the subjective taken out.

OBJECTIVE is a word that is defined to mean NOT SUBJECTIVE.

What this means is that if you say, 'Everything is subjective in a sense' you could not be wronger. You just took millennia of language development and attempted to deep-six it.

This is why we have these words. To prevent argument about what is, and what is not the case. To divide views of the world, as far as possible, into opinion and fact.

Opinions are not facts. They may coincidentally be. 'In my opinion it's seventy degrees Farenheit.' It could be true. It could be false. When enough people have looked at the thermometer, then we'll know.

When there is no possibility of using a thermometer, then we have to collect opinions.

There's no such thing as a musical 'thermometer', no instrument for measuring musicality. So if we want to assess whether one musical passage (burst of sound) is more musical than another, then we have to ask everybody in the world what they think. The one that more people think is more musical than the other is the more musical, because more people think so.

I shouldn't have to explain this stuff, but I do.

When somebody says 'Everything is subjective in a sense', then you know it's time to get down to the nitti-gritti.

People's opinions can be biassed. Some people are Russians, some are Americans.

Some people are incapable of dissociating their opinions of a piece of music from the nationality of its composer.

So you might find that Americans voted for an American composed bit of music as being more musical than a Russian composed piece, and vice versa.

So when we know that there are factors which bias a judgement, we have to take special measures to exclude these factors.

Sometimes, it has been shown, (in medical testing), that it is sufficient for the people conducting the tests to know that (for example) the composer was American, for it to influence the outcome of the tests.

Hence a system of testing has been devised using rigid protocols to ensure that neither testers nor testees have prior knowledge of any factors likely to cause bias. This system is called double-blind testing.

The results of properly conducted double-blind testing are objective truth. Not subjective. Objective truth.

That's all she wrote.

w

Quote:

Originally Posted by preproman 

I don't see how you folks think that measurements are objective..   They are just as flawed as anything else.  How can three different objective measurements of the same headphone show different results?  Each site we go to for these measurements use different amps different tools, everything is different thus giving different results.  

 

Sorry, I don't buy it.  try to sell me something els.

Inconsistent methodologies.  That's why scientific publications always contain detailed methods sections, which is often the only part that people care about.  You can't throw out measurement and you can't not buy it.  If you live past the age of 35 you've already bought it.

Why such a big fuss over this? I think we can all agree that objective measurements (balanced FR, low distortion, etc) generally correlate to how well someone might subjectively evaluate a certain headphone. And since the vast majority of high-end headphones perform incredibly well on paper, if you were someone who wanted to buy one, you might want a flesh-and-bones human to sit down with all of the models and explain which ones he thought were best. This is the purpose of headphone reviews. It won't be as good as sitting down with them yourself, but most of us don't have that sort of cash to blow.

Quote:

Originally Posted by OJNeg 

Why such a big fuss over this? I think we can all agree that objective measurements (balanced FR, low distortion, etc) generally correlate to how well someone might subjectively evaluate a certain headphone. And since the vast majority of high-end headphones perform incredibly well on paper, if you were someone who wanted to buy one, you might want a flesh-and-bones human to sit down with all of the models and explain which ones he thought were best. This is the purpose of headphone reviews. It won't be as good as sitting down with them yourself, but most of us don't have that sort of cash to blow.

I would say you also need a certain amount of understanding to go through and understand technical details, and reviews generally make it easier by dumbing down these technicalities. 

Its perfectly alright to do that, but here's what I think a good review has:

a) A set of measurements, and

b) Their analysis, both technical/non technical (i.e. graphs and sound).

c) The ability to distinguish which is which (objective vs subjective).

Bad reviews just make it easier to confuse readers.

In my experience, relying on measurements gives you a much better shot at getting what you want, because its a common ground, a translator of sorts. You convert your preferences into a broad set of measurable characteristics, and it eases your search by a big amount by removing all the noise. With a set of clearly defined requirements, you'll mostly end up with a list of headphones you can count on your fingers. Then you can take a look at a few detailed measurements, subjective reviews, find out whats the general view on a particular headphone. Its not as complicated as its made out to be.

@ Wakibaki: Great post.

Quote:

Originally Posted by eucariote 

 

Inconsistent methodologies.  That's why scientific publications always contain detailed methods sections, which is often the only part that people care about.  You can't throw out measurement and you can't not buy it.  If you live past the age of 35 you've already bought it.

Says who?  Yes I'm past 35, well past it.  

If it's inconsistent then it's not objective. Point Blank.  Like I said.  I don't buy it.  I would rather rely on my own subjective ears.  Thank you very much.

I (and probably the average consumers out there) don't know how to read graphs. We're talking audio here, something that cannot be seen. I don't see anything wrong with subjective reviews and audiophile lingo is just a 'language' to get people to understand the same thing.

Then you never heard about bias.

Quote:

Originally Posted by gorboman 

I (and probably the average consumers out there) don't know how to read graphs.

But those who care about high fidelity should be able to read them.

Well you can look at measurements. Can you see the air temperature? Or do you use a thermometer to take measurements?

What idyllic thinking. Some reviewers do not even understand the terms, yet use them. Most of these terms are very fuzzy, almost meaningless. In many reviews they seem to be used as fillers.

Quote:

Originally Posted by Puranti 

Then you never heard about bias.

I know bias when I see one. Not always, but often I see them. :)

Quote:

Originally Posted by xnor 

But those who care about high fidelity should be able to read them.

 

Well you can look at measurements. Can you see the air temperature? Or do you use a thermometer to take measurements?

 

What idyllic thinking. Some reviewers do not even understand the terms, yet use them. Most of these terms are very fuzzy, almost meaningless. In many reviews they seem to be used as fillers.

If you think reading graphs works for you, then good for you. I don't read graphs because I don't know how to interpret them nor use them to determine whether a gear is good for me or not.

I'll pass commenting on the thermometer. I think I've made my point above.

I see what you mean. I would see a reviewer's credibility by going through his/her previous reviews, how long has he/she been doing reviews, does his/her reviews has active comments, etc. Also we can always sort who can give reviews that's in line with our preferences. So every time he/she use a term, we've got an idea on how he/she really means. But if I see things as fuzzy or meaningless to me, then I'd ask the reviewer in the comment section.

Quote:

Originally Posted by gorboman 
If you think reading graphs works for you, then good for you. I don't read graphs because I don't know how to interpret them nor use them to determine whether a gear is good for me or not.

 

I'll pass commenting on the thermometer. I think I've made my point above.

But it's basically the same with the thermometer, you have to know for example that 39°C is not normal body temperature but fever. In case of frequency response curves (preferably raw) you also have to know that ~100 Hz is bass and +10 dB relative to the midrange is a big bass boost.

So to actually make some sense of the reviews you have to have heard at least some of the headphones the person reviewed before and drawn connections between his/her and your impressions.

Also, the "flexible" use of these terms makes them pretty much fillers. What's a term good for if you have to guess what it means. Maybe some terms didn't appear fuzzy or meaningless to you, but the reviewer meant something else? It can all be very vague, which makes purely subjective audio reviews not very useful.

Quote:

Originally Posted by xnor 

But it's basically the same with the thermometer, you have to know for example that 39°C is not normal body temperature but fever. In case of frequency response curves (preferably raw) you also have to know that ~100 Hz is bass and +10 dB relative to the midrange is a big bass boost.

 

So to actually make some sense of the reviews you have to have heard at least some of the headphones the person reviewed before and drawn connections between his/her and your impressions.

Also, the "flexible" use of these terms makes them pretty much fillers. What's a term good for if you have to guess what it means. Maybe some terms didn't appear fuzzy or meaningless to you, but the reviewer meant something else? It can all be very vague, which makes purely subjective audio reviews not very useful.

Like I said before, I use a tool if I need it. Your previous example was the air temperature, right? I don't have to use a thermometer to know whether the day is hot or if it's cold outside. If you prefer to use one to know how hot the air is, then that's fine. So, do I have to know about reading the graphs? I don't see the necessity of it yet.

Again like I said before, if there are things that we see unclear in a review, we can always ask the reviewer in the comment section. Regarding the use of terms, don't Head-Fi have a glossary of what a term means? I'm sure there are other sources too on the internet to learn what a term means.