When trust disappears, Reviewers are simply “noise”
Nov 8, 2017 at 5:08 PM Post #181 of 198
I've massively improved the objectivity of my reviews in 2017 by not posting any. the improvement is spectacular, I haven't misguided or lied to anybody. perfect truth telling score. yet, somehow I feel like there might be a tiny flaw in my strategy. I can't really put my finger on it.

(I may have a dry sense of humor today)

I think you're putting the accent to much on not posting rather than posting.

You need that Yes in your life.

1hqj5d.jpg
 
Nov 9, 2017 at 2:52 AM Post #182 of 198
Hopefully, discussion between reviewers going on in background. Looking forward to conclusion & agreed actions.

Sadly - while the post sparked discussion, there does not seem to be enough drive toward meeting a standard. To really gain traction, Head-Fi Admins would need to recognise the problem and get behind it too. I personally don't see this happening anytime soon (I would love to be wrong on this).

One good thing thats come out of it is the need for me to develop an objective scoring matrix for myself - so I guess at least I've gained something new :)
 
Nov 9, 2017 at 2:58 AM Post #183 of 198
One good thing thats come out of it is the need for me to develop an objective scoring matrix for myself - so I guess at least I've gained something new :)

There's no reason why a few of us can't collaborate on this together and start using it on our own. Baby steps
 
Nov 9, 2017 at 4:29 AM Post #184 of 198
Well if we can get a few to collaborate - I'm there. But if people are going to rate "fun factor" count me out :wink:
 
Nov 9, 2017 at 6:52 AM Post #187 of 198
So do your own thing George - no-one says we all have to be the same. I just wanted a bit more accuracy in reviewing - and I think i made my point there. No use going over old ground again.

But I am NOT going to be calling "fun factor" a topic in a review, and definitely never grading based on it. I'd lose all credibility ......
 
Nov 9, 2017 at 6:53 AM Post #188 of 198
So do your own thing George - no-one says we all have to be the same. I just wanted a bit more accuracy in reviewing - and I think i made my point there. No use going over old ground again.

But I am NOT going to be calling "fun factor" a topic in a review, and definitely never grading based on it. I'd lose all credibility ......

I won't use that as a factor for reviewing either tho :triportsad:
 
Nov 9, 2017 at 7:43 AM Post #189 of 198
The thing is, "fun" is subjective. The whole point of this discussion is that we're trying to remove the "subjective opinion" part as much as possible. I mean, heck, we're talking about a feeling here....objectivity has no place for emotion. Yes, music is emotional, but I'd argue that it's impossible to accurately type out the exact emotion that a reviewer gets when using a particular device, and quite probably impossible to then have the reader feel that exact same emotion.
 
Nov 9, 2017 at 9:31 AM Post #190 of 198
The thing is, "fun" is subjective. The whole point of this discussion is that we're trying to remove the "subjective opinion" part as much as possible. I mean, heck, we're talking about a feeling here....objectivity has no place for emotion. Yes, music is emotional, but I'd argue that it's impossible to accurately type out the exact emotion that a reviewer gets when using a particular device, and quite probably impossible to then have the reader feel that exact same emotion.

I think I am very much objective and impartial to things.

I gave high grades to signatures that I didn't like because the technical performance was good.

At any rate, I understand the issue with my approach and won't insist on something silly I said, that was my mistake :smile_phones:
 
Nov 9, 2017 at 1:18 PM Post #191 of 198
This is excellent news. Absolutely nothing wrong with baby steps, Great to see a few good people collaborating & leading the way for the rest to follow. Can't wait to read the first review that says "peer reviewed" and uses an agreed scoring matrix, regardless of who writes it & whatever the item being reviewed. Somebody please PM me to look out for it. I reckon you could be up and running this side of Christmas or very early 2018. The very best of luck to each of you!
 
Nov 9, 2017 at 6:49 PM Post #192 of 198
Just my 2c (and purely my own opinion of course, feels like I have to disclaim this a lot these days) I find point scoring systems a little too arbitrary and susceptible to hindsight. Like, scoring IEM "a" a 7/10 in a certain aspect, forgetting about it for a while, and then scoring another IEM "b" the same even though it's probably better or worse than "a", only recognisable in hindsight.

I mean it's probably just me rambling. It seems that a "standardised" point system would only work for one reviewer but cannot be extrapolated to another, since we all obviously have different perceptions of what constitutes anything from a 1 to a 10. And it seems that it would lull readers into a false sense of comparability in which they think they can compare the points of one reviewer to that of the other, when in fact it's extremely individualised.

Not knocking on the format, people can use it however they like. Just skeptical of its objectives.
 
Nov 9, 2017 at 7:20 PM Post #193 of 198
Just my 2c (and purely my own opinion of course, feels like I have to disclaim this a lot these days) I find point scoring systems a little too arbitrary and susceptible to hindsight. Like, scoring IEM "a" a 7/10 in a certain aspect, forgetting about it for a while, and then scoring another IEM "b" the same even though it's probably better or worse than "a", only recognisable in hindsight.

I mean it's probably just me rambling. It seems that a "standardised" point system would only work for one reviewer but cannot be extrapolated to another, since we all obviously have different perceptions of what constitutes anything from a 1 to a 10. And it seems that it would lull readers into a false sense of comparability in which they think they can compare the points of one reviewer to that of the other, when in fact it's extremely individualised.

Not knocking on the format, people can use it however they like. Just skeptical of its objectives.

Now I'm feeling kinda fuzzy. A bit drunk as well. Ignore this rant!

I agree with @crinacle

At the end of the day, my general mood and approach is to be objective, but I just am a bad person. I take up to two months to finish a review to apply some degree of objectivity. I write something then read it again, then judge it after I weight how I felt.

Most reviewers aren't expected to and probably shouldn't do this. I'm too weary from my anti-bias method (This is how I named this process)
 
Nov 9, 2017 at 9:09 PM Post #194 of 198
Just to get this back on track - this was never about different reviewers having different opinions. That was not my intent. People should score as they honestly experience the product.

My rant was about the standard of some reviews, and how some of it was so subjective (and IMO poor), that it could allow products with obvious faults be ranked as almost perfect or perfect - when they are clearly not.

I was suggesting this is down to inexperience, failure to apply objectivity, and a case of wanting to please manufacturers more than the real audience (ordinary readers and potential buyers).

Crinn I know you agree with this because we have both heard and measured the Kinera H3.

There may be no solution to this - as it is clear that HF as an entity prefers to post favourable reviews and most manufacturers will reward positive reviews which have maximum exposure with more samples. The reverse of course is that those who are very transparent with reviews of product with issues are potentially penalised. As long as this behaviour is continually encouraged, the worse the standards will likely get.

My concern is that the site I chose to call home, and be exclusive for my own reviews has been increasingly seen as having more noise than signal. If that behaviour does not change, then eventually I would either need to adapt or move. My personal ethics will not allow me to change the way I review - so in that case I will have no choice but to move where the signal is stronger. This may end up being a reality. The sad thing is that I’ve made a lot of friends here over a number of years. I don’t want to move on but am fast realising I may have no choice.
 
Nov 9, 2017 at 10:01 PM Post #195 of 198
Why not stay to give everyone your input as well, keep the ratio up? :smile_phones:

I think people should listen to something they are reviewing and that they should be fair to something, that much is clear. Regardless of the result or methods, at least they should be doing their best. From there, if they are willing to learn, the system can always improve.
 

Users who are viewing this thread

Back
Top