or Connect
Head-Fi.org › Forums › Equipment Forums › Sound Science › Evidence that subjective, sighted reviews fail
New Posts  All Forums:Forum Nav:

Evidence that subjective, sighted reviews fail - Page 2

post #16 of 71
IEMs: go to rinchoi.blogspot.com and look up the ASG-2, Earsonics SM64 and anything made by Heir. As for reviews, look them up on head-fi, you won't find a single reviewer badmouthing them except for a handful of objective lunatics getting rotten eggs thrown at them everywhere they go wink.gif
Edited by Joe Bloggs - 10/8/13 at 6:28am
post #17 of 71
Quote:
Originally Posted by xnor View Post

Well, I kinda agree! In my book that's a fail.

Again, a fail at what exactly?

If the purpose of a sighted/subjective review is for someone to simply relate their sighted/subjective experience with a given piece of gear, then what can it possibly be failing at?

se
post #18 of 71
Quote:
Originally Posted by Steve Eddy View Post


Again, a fail at what exactly?

If the purpose of a sighted/subjective review is for someone to simply relate their sighted/subjective experience with a given piece of gear, then what can it possibly be failing at?

se

 

One issue with magazine reviews is that the reviewers are representing themselves (or being represented by their editors) as expert listeners. The implication is that they are better and more capable of determining the "quality" of the item being tested. Yes it is just their opinion but we are supposed to think that they have a certain expertise, funnily enough when Harman tested listeners they found that HiFi salespersons were actually better than professional reviewers, were more capable at detecting the more objectively capable speakers. 

post #19 of 71
Thread Starter 
Quote:
Originally Posted by Steve Eddy View Post

If the purpose of a sighted/subjective review is for someone to simply relate their sighted/subjective experience with a given piece of gear, then what can it possibly be failing at?

Conforming to reality due to biases?

 

But reviewers somehow have to convey that their reviews are not heavily biased else they'd be more or less useless and I'd argue that most reader will read reviews in that light. I'd also argue that they read those unsubstantiated claims ("best DAC for years to come" etc.) as more than just claims.

post #20 of 71
Thread Starter 
Quote:
Originally Posted by Joe Bloggs View Post

IEMs: go to rinchoi.blogspot.com and look up the ASG-2, Earsonics SM64 and anything made by Heir. As for reviews, look them up on head-fi, you won't find a single reviewer badmouthing them except for a handful of objective lunatics getting rotten eggs thrown at them everywhere they go wink.gif

Thanks. Couldn't the aurisonics (?) asg-2 though.

 

Will add those I could find to the first post.

post #21 of 71
Quote:
Originally Posted by xnor View Post

Conforming to reality due to biases.

But we're talking about subjective reviews. What the hell does conforming to reality have to do with it? Conforming to reality is decidedly objective.

I'm sorry, but your premise makes no sense. You're saying that subjectivity fails because it's not objective. But by that token, one could just as well say that objectivity fails because it's not subjective.

It's nonsense.

se
post #22 of 71
Thread Starter 
Quote:
Originally Posted by Steve Eddy View Post

But we're talking about subjective reviews. What the hell does conforming to reality have to do with it? Conforming to reality is decidedly objective.

So subjective reviews can be completely fictitious, purely invented nonsense? Are subjective reviews of real things not supposed to be a description of the experience of these real things? I always thought that when someone did a review on a component on a forum he wanted to convey his experience of the real thing to others, not some made up stuff.

 

Let's take one of the IEMs from above for example. 20 dB dip at ~3 kHz but reviewers say it is a "giant slayer" with treble having "nice details" and being "well balanced". That's nonsense.

 

Quote:

I'm sorry, but your premise makes no sense. You're saying that subjectivity fails because it's not objective. But by that token, one could just as well say that objectivity fails because it's not subjective.

No, there's nothing wrong with subjective tests per se (already mentioned in #3). I'm saying those reviews fail because they're nonsense. Nonsense because they're interminably biased with no basis in reality, there are no attempts of controlling variables, no attempts of a proper test setup.

Just sitting down and putting one's oar in it seems.

 

 

PS: We're not talking about the meaning of someone's dreams here.


Edited by xnor - 10/8/13 at 1:53pm
post #23 of 71
I give up.

se
post #24 of 71
Thread Starter 
Quote:
Originally Posted by Steve Eddy View Post

I give up.

That was quick. :D 

 

So are you saying that experiences can be off but because they're subjective it's all dandy even if they're completely off (and by the linked reviews that seems to be more the rule than the exception)?

 

A: "The moon is made of cheese!"

B: "You belong in an asylum."

A: "No, that's just my subjective review of the moon."

B: "Oh, in that case it's fine."


Edited by xnor - 10/8/13 at 3:56pm
post #25 of 71

I don't think sighted tests are entirely subjective at all. A direct A/B matched comparison can give you a very good idea of the general lay of the land. The only time a blind test is really needed is if two sounds are so close it's hard to tell them apart. That rarely matters in home audio.

post #26 of 71
Quote:
Originally Posted by xnor View Post
 

Thanks. Couldn't the aurisonics (?) asg-2 though.

 

Will add those I could find to the first post.

 

I was going to point you to rinchoi's review but did a double take when I looked at your updated OP.  I think you need more different sources, and rinchoi gets enough eggs thrown at his face as it is :D Then again, I suspect the objective performance of these phones is good enough reason why more major, more commercial measurement-based review sites like headphone.com haven't got a hand on them :D


Edited by Joe Bloggs - 10/8/13 at 5:43pm
post #27 of 71
I consider myself an objective person so I'll tell my story. A long time ago a guy like me bought an etymotic mc5, which in theory has the most flat, highly developed, controlled decay, undistorted sound ever created... Then I got the w4r with one of the most awful graphs I ever seen in my life. Needless to say the result. Both were not my style however the w4r has something difficult to explain that makes the sound pleasing. Do you think any test can show me what I heard?

In theory the frequency response differs greatly from what I consider optimal...
post #28 of 71

I think subjectivity is fine, but where most of us falter is clarifying the fact that the assertion is subjective.

 

For example:

 

-- The moon is made of cheese

 

as compared to:

 

-- In my opinion, the moon is made of cheese.

 

 

This makes a huge difference, the first one seems like a universal truth, while the other one seems like an individual opinion, meaning you may be open to changing it based on better evidence.

 

 

As you might have seen in the media, most good publications follow either of the two styles:

 

-- Just report the facts, and let the readers make up their mind. BBC is a good example.

-- Admit upfront that the statement is the newspaper's stand on the issue. The Economist does that often.

 

I don't think either of them are wrong, the idea is to know what a claim is as opposed to a fact.


Edited by proton007 - 10/8/13 at 6:58pm
post #29 of 71
Quote:
Originally Posted by xnor View Post
 

That was quick. :D 

 

Just don't have the time to bang my head against a wall that I used to.

 

Quote:
 So are you saying that experiences can be off but because they're subjective it's all dandy even if they're completely off (and by the linked reviews that seems to be more the rule than the exception)?

 

"Off" from what? In the context of someone relating their subjective experience, how can it be "off" unless you're going to accuse that someone of lying about what their subjective experience was? One's subjective experience is what it is. And in my opinion the only thing that counts for anything at the end of the day. I don't understand why you insist on imposing objectivism into a context where it is utterly irrelevant.

 

Quote:
 A: "The moon is made of cheese!"

 

That's a decidedly objective claim. Do you even know the difference between subjective and objective?

 

se

post #30 of 71
Quote:
Originally Posted by Steve Eddy View Post

Just don't have the time to bang my head against a wall that I used to.


"Off" from what? In the context of someone relating their subjective experience, how can it be "off" unless you're going to accuse that someone of lying about what their subjective experience was? One's subjective experience is what it is. And in my opinion the only thing that counts for anything at the end of the day. I don't understand why you insist on imposing objectivism into a context where it is utterly irrelevant.


That's a decidedly objective claim. Do you even know the difference between subjective and objective?

se

Apparently there were claims made about the objective performance of the Wadia in the subjective reviews (I say "apparently" because I haven't bothered to look closely at them at all)

The other contention was that looking at an item's shiny exterior and shiny price tag may bias one's impressions of the item. While one might say that such biased impressions are still valid subjective impressions one might also say that those are silly reasons for liking an item that should be filtered out.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › Evidence that subjective, sighted reviews fail