[Brooko]

I get what you mean, but my intention was just so that we can get a better standards between reviewers. That alone will already make it easier for readers/viewers as there will at least be a bit more objective consistency between reviewers..

But yeah, I think we need to come up with a review system. We can come up with a catchy name maybe a logo... And the when reviewers score and compare using our system, they can go ahead and attach that logo to their review. I know this thread started off about the trust of reviewers, but I feel that consistency and having a standard would perhaps make it all the more easier to build that trust

It was something Warren and I discussed some time ago when I was a Moderator. He actually had the bare bones of a reviewing format fleshed out. Add that to a scoring format - including giving people a template - and yes it could work quite well. Whether other people were actually interested in using it - would be another matter.

 

[Brooko]

To expand on this - and this is headphone / earphone only - you'd need consensus on how to set up other audio gear.

Simple Excel spreadsheet - has base score you fill in. Weightings need to be agreed. Spits out a total score, and a number of stars for the review. Saves the guess work and means everyone would be consistent in their scoring.

Its just a baseline. Might be worth discussing further.

Headphones	Kinera H3
	My Score	Out of	Weighting	Weighted Score
Accessories	7	10	5.00%	0.35
Build	10	10	10.00%	1
Design	7	10	5.00%	0.35
Fit/Comfort	10	10	15.00%	1.5
Sound Quality
Bass quality	5	8	7.50%	0.375
Mid-range quality	2	8	7.50%	0.15
Treble quality	2	8	7.50%	0.15
Overall tonality	3	10	7.50%	0.225
Clarity	2	8	7.50%	0.15
Stage/Imaging	3	8	7.50%	0.225
Value	5	10	20.00%	1
Total	56	100	100.00%	5.475
Stars 2.7375

 

[Brooko]

Excuse the formatting - not forum software friendly, but could actually write a macro to spit out the formatted code too.

 

[Brooko]

Nobody cares about your graphs bruh.

Evidently some do - different people will have different things they relate to. Ultimately they are simply a tool to illustrate what a person is hearing. So you may not care about the graphs - and I get that. I do - because to me they are more important than someone stating what they hear with a particular piece of music. That is truly subjective. I live a lot in the objective realm too. There is no issue with stating both.

 

[TheoS53]

To expand on this - and this is headphone / earphone only - you'd need consensus on how to set up other audio gear.

Simple Excel spreadsheet - has base score you fill in. Weightings need to be agreed. Spits out a total score, and a number of stars for the review. Saves the guess work and means everyone would be consistent in their scoring.

Its just a baseline. Might be worth discussing further.

Headphones	Kinera H3
	My Score	Out of	Weighting	Weighted Score
Accessories	7	10	5.00%	0.35
Build	10	10	10.00%	1
Design	7	10	5.00%	0.35
Fit/Comfort	10	10	15.00%	1.5
Sound Quality
Bass quality	5	8	7.50%	0.375
Mid-range quality	2	8	7.50%	0.15
Treble quality	2	8	7.50%	0.15
Overall tonality	3	10	7.50%	0.225
Clarity	2	8	7.50%	0.15
Stage/Imaging	3	8	7.50%	0.225
Value	5	10	20.00%	1
Total		100	100.00%	5.475
Stars 2.7375

Yup, I've done something similar with the way I score, but I break it I to 2 parts. O e score focuses on the device as an audio product, and then an overall score that grants bonus points to what are essentially unnecessary bells and whistles too.

 

[TheoS53]

TheoS53

But then again, my situation is different to most people. The gear I get are sent to me through a local audio shop. I work closely with them (and in some regard for them), and I get paid for doing my reviews. So the gear I have are either units I have purchased through the shop at a 15% discount, or are units that were sent for the purpose of the review (but not items that the shop sells). But even if the review items are "gifts" front he manufacturers, I still consider them to be loaner units from the shop... If that makes sense.

________________________________________________________________________________________


There has got to be a rule preventing this or accommodating this and I think it is called M.O.T

Well, I don't think there's anything inherently wrong with getting paid to do reviews. Heck, is it really all that much different to a columnist? Sure, that can generate a certain skepticism regarding a person's authenticity. I guess that's partly why I'm rather quick to point out a product's flaws, in an attempt to make it clear that if I don't like something, I'll pick at it....and I have...rather relentlessly in the case of Fiio's products.
At the end if the day, what I get paid to do is to write my opinion. If the product is good enough that it meets my own expectations and as a result I have overly positive things to say, then good job to the manufacturer.

I have, on more than one occasion, expressed my concern to the shop (as they're the ones supplying the products to me) about my negative views possibly hurting their sales. Here I really have to give them kudos as they've explicitly said "let us worry about that, you just go ahead and be honest". I really respect that.

 

[Hawaiibadboy]

Well, I don't think there's anything inherently wrong with getting paid to do reviews. Heck, is it really all that much different to a columnist? Sure, that can generate a certain skepticism regarding a person's authenticity. I guess that's partly why I'm rather quick to point out a product's flaws, in an attempt to make it clear that if I don't like something, I'll pick at it....and I have...rather relentlessly in the case of Fiio's products.
At the end if the day, what I get paid to do is to write my opinion. If the product is good enough that it meets my own expectations and as a result I have overly positive things to say, then good job to the manufacturer.

I have, on more than one occasion, expressed my concern to the shop (as they're the ones supplying the products to me) about my negative views possibly hurting their sales. Here I really have to give them kudos as they've explicitly said "let us worry about that, you just go ahead and be honest". I really respect that.

There is a bad habit IMO of leaning on the folks that run this site to do or say something when we as adults can do some of that ourselves.
I think it is expected that we do as much as possible and lean on the structure and Team Head-Fi as a last resort.

We are not kids and they are not parents.

You gave me a great compliment of respect and I did not see what was typed beyond that.

It was shared with me.
in a perfect situation it would have been asked or clarified right away by those who saw it. I woke up the next day and nobody had yet said "hey, this actually alright? I think it may not be?"
So I said it.

12 hours after getting kind words from you and maybe being on the best footing yet with you I chose to jeopardize that by not stabbing you in the back and instead give you the courtesy of replying in public as you had brought it out yourself..in public.

I did it to be fair.

Most folks won't get that. Hope you do. If not, cannot be helped.





My graph comment was a mock
Shoulda been seen.


The graph tells if a bloat is gonna be there and maybe result in dipped mids which may result in vocals being distant, or a mid hump which will make vocals (fundamental) too overbearing and on and on.
This has become a focus on unity with not enough focus on the consumer.
If your gonna make charts make one explaining the Freq graph and how it can be read to give a glimpse into what it all means for a regular non audio head so they can better navigate this world.
That picture (graph) tells a story.
Help explain how to read it.

 

[TheoS53]

There is a bad habit IMO of leaning on the folks that run this site to do or say something when we as adults can do some of that ourselves.
I think it is expected that we do as much as possible and lean on the structure and Team Head-Fi as a last resort.

We are not kids and they are not parents.

You gave me a great compliment of respect and I did not see what was typed beyond that.

It was shared with me.
in a perfect situation it would have been asked or clarified right away by those who saw it. I woke up the next day and nobody had yet said "hey, this actually alright? I think it may not be?"
So I said it.

12 hours after getting kind words from you and maybe being on the best footing yet with you I chose to jeopardize that by not stabbing you in the back and instead give you the courtesy of replying in public as you had brought it out yourself..in public.

I did it to be fair.

Most folks won't get that. Hope you do. If not, cannot be helped.

That's exactly why I brought it out in public. I've got nothing to hide. I get paid a fixed amount ($45) per review for my written reviews, but the video reviews I do for myself and under my own "brand", so there's no incentive to sway my opinion to write something more positively or negatively than what I feel it needs to be. If, somehow, for whatever reason, later on people realised that I got paid to write without me disclosing it, then I think it's fair to assume that they would (rightfully so) question my integrity for having hidden that fact. Why would someone hide that unless they're knowingly doing something shady? lol

But as I said, there's nothing inherently wrong with getting paid for it. Providing reviews is providing a service. There's a need/want for it, and it takes time to do. I think what helps is that the payment comes from the shop (which is an online tech magazine first and foremost), rather than from the manufacturers. As long as a person does not allow the money to corrupt their opinion, then there's no issue.

But the fact that you have an issue with this goes to show what little trust there is in a person's integrity when money is involved. C'est la vie.

I think it's a lack of transparency that often creates distrust, as people will always end up wondering if the person is being truly honest. And that's perfectly fine, I can't blame people for having those doubts. Hopefully my reviews are what will show an attempt at being as balanced and objective as possible.

 

[Dobrescu George]

That is precisely the reason you have to reset. You're reviewing a new IEM (RE-800 is a classic example). You use it and only it for 3-4 days. Your ears get used to the treble peak. When you come to write your review, you describe it as neutral with smooth treble, or maybe a mild bump at 7kHz, because that is what you now hear), and make no mention of the size of the peak. Someone buys it based on your recommendation - because they are looking for a flat signature, and they may be sensitive to lower treble spikes.

Again I'll use the Curve for comparison - what you describe in orange. The reality in yellow.

Interesting. I get your point!

To expand on this - and this is headphone / earphone only - you'd need consensus on how to set up other audio gear.

Simple Excel spreadsheet - has base score you fill in. Weightings need to be agreed. Spits out a total score, and a number of stars for the review. Saves the guess work and means everyone would be consistent in their scoring.

Its just a baseline. Might be worth discussing further.

Headphones	Kinera H3
	My Score	Out of	Weighting	Weighted Score
Accessories	7	10	5.00%	0.35
Build	10	10	10.00%	1
Design	7	10	5.00%	0.35
Fit/Comfort	10	10	15.00%	1.5
Sound Quality
Bass quality	5	8	7.50%	0.375
Mid-range quality	2	8	7.50%	0.15
Treble quality	2	8	7.50%	0.15
Overall tonality	3	10	7.50%	0.225
Clarity	2	8	7.50%	0.15
Stage/Imaging	3	8	7.50%	0.225
Value	5	10	20.00%	1
Total		100	100.00%	5.475
Stars 2.7375

You know, that is excellent. We could use something like this to keep the quality of our assessments up. I might want to argue about the weighting, and someone would have to make this an excelt that doesn't require us to do pure calculus each time we try to do this, but man, this looks exactly like what I was trying to say a few pages ago with my "fun factor". My idea was a bit simplified. We can't quantum the signature, but we can quantum the quality of each component. I don't know if the accessiroies and design should weight as much, even lower, comfort a bit less, builkd a bit more, all sonic characteristics a bit more, and I think we should take out overall tonality, this is not quantifiable since there are tastes out there.

Well, I don't think there's anything inherently wrong with getting paid to do reviews. Heck, is it really all that much different to a columnist? Sure, that can generate a certain skepticism regarding a person's authenticity. I guess that's partly why I'm rather quick to point out a product's flaws, in an attempt to make it clear that if I don't like something, I'll pick at it....and I have...rather relentlessly in the case of Fiio's products.
At the end if the day, what I get paid to do is to write my opinion. If the product is good enough that it meets my own expectations and as a result I have overly positive things to say, then good job to the manufacturer.

I have, on more than one occasion, expressed my concern to the shop (as they're the ones supplying the products to me) about my negative views possibly hurting their sales. Here I really have to give them kudos as they've explicitly said "let us worry about that, you just go ahead and be honest". I really respect that.

I think someone who gets paid to do this will have a higher quality of their work actually... Because you're on a critic point where if you lie, you don't get paid anymore, so you need to be honest. I think everyone should get paid for what is basically a job.

That's exactly why I brought it out in public. I've got nothing to hide. I get paid a fixed amount ($45) per review for my written reviews, but the video reviews I do for myself and under my own "brand", so there's no incentive to sway my opinion to write something more positively or negatively than what I feel it needs to be. If, somehow, for whatever reason, later on people realised that I got paid to write without me disclosing it, then I think it's fair to assume that they would (rightfully so) question my integrity for having hidden that fact. Why would someone hide that unless they're knowingly doing something shady? lol

But as I said, there's nothing inherently wrong with getting paid for it. Providing reviews is providing a service. There's a need/want for it, and it takes time to do. I think what helps is that the payment comes from the shop (which is an online tech magazine first and foremost), rather than from the manufacturers. As long as a person does not allow the money to corrupt their opinion, then there's no issue.

But the fact that you have an issue with this goes to show what little trust there is in a person's integrity when money is involved. C'est la vie.

I think it's a lack of transparency that often creates distrust, as people will always end up wondering if the person is being truly honest. And that's perfectly fine, I can't blame people for having those doubts. Hopefully my reviews are what will show an attempt at being as balanced and objective as possible.

45$ is very low for a payment though, you could even think it as nothing. What can you do with that? Buy pizza and beer two times...

 

[Dobrescu George]

I marvel at the display going on in here.

I noted that the site folks were watching the thread as they could be seen in the viewing this thread bar

Did they see?

_______________________________________________________________________________________

TheoS53

But then again, my situation is different to most people. The gear I get are sent to me through a local audio shop. I work closely with them (and in some regard for them), and I get paid for doing my reviews. So the gear I have are either units I have purchased through the shop at a 15% discount, or are units that were sent for the purpose of the review (but not items that the shop sells). But even if the review items are "gifts" front he manufacturers, I still consider them to be loaner units from the shop... If that makes sense.

________________________________________________________________________________________


There has got to be a rule preventing this or accommodating this and I think it is called M.O.T








________________________________________________________________________________________

ryanjsoo

I was also talking to some other reviewers that loved the H3 a LOT, I guess those comments got to me, at that time I was less confident in my own analysis.

And that's the only product that I've covered that I feel was misrepresented so it's like a big stain on my reputation


_________________________________________________________________________________________

That is peer pressure, real or imagined.
It is not a stain that should be of concern it should be money spent based on your words, and Head-Fi gives you banner space

I started on you after saying the Q1MKII had loose bass. Like George and his "U" shaped DAP.....that is very very very likely a driver issue not an amp/dac. What drivers and what song presented loose bass on an amp/dac?

You admitted you are new and maybe Head-Fi should consider waiting and watching longer before giving prime space to a "reviewer".
This comes from a guy that could not possibly giva less shiii about front page stuff. Who cares?




_________________________________________________________________________________________

George


no copy paste because that would take a long time

Odd statements then personal attacks then oops sorry and then rinse/repeat. This is an observation shared by many but not being said public
There should be no reviews done by this person until issues are sorted out as again...THIS IS A PUBLIC SITE AND INVOLVES REAL MONEY SPENT BUY STRANGERS YOU WILL NEVER MEET AND WHOOOPS DOES NOT CUT IT



_________________________________________________________________________________________


On a site with audio gear used to play music nobody is considering an agreed set of tracks that the reader could easily get in anyway they choose and test the explanation of recessed mids and forward mids using a part of the track to demonstrate.
Same could be done with bloated bass, rolled off treble, spikey treble. Nobody cares about your graphs bruh. They care about what they mean to them, the music lovers they likely are.


i got my badge by speaking to the public in a real guy way, not geek speak. Stop overthinking
This is all about the music.


I am going out to eat good Japanese food and get freaky with my lady. Not interested in replies. Life is too short, folks not saying in public what they say in private and the music is what this is all about and nobody saying that.

Getting a long is great. Being honest with myself and readers is much more important.

Good luck....i think? Trying harder would be better than luck

Why do you want to fight tho?

I think there should be a rule against what you yourself said you're doing. Promoting something without weighting things. This is exactly what @Brooko said was wrong with this site lately, many are promoting and not reviewing...

Also, I don't do personal attacks in general.

I might make mistakes once or twice, but I always correct them.

 

[Brooko]

You know, that is excellent. We could use something like this to keep the quality of our assessments up. I might want to argue about the weighting, and someone would have to make this an excelt that doesn't require us to do pure calculus each time we try to do this, but man, this looks exactly like what I was trying to say a few pages ago with my "fun factor". My idea was a bit simplified. We can't quantum the signature, but we can quantum the quality of each component. I don't know if the accessiroies and design should weight as much, even lower, comfort a bit less, builkd a bit more, all sonic characteristics a bit more, and I think we should take out overall tonality, this is not quantifiable since there are tastes out there.

Shows how we think differently and there would need t be discussion and consensus if it was to be adopted. For instance "fun" factor is not something I would include - it is pointless, overly subjective, and conveys nothing about an IEM or headphone. You can't define it. Comfort to me is huge - I love the sound of the DK3001 but can't wear them because of the comfort. The Dipper I can wear literally for hours. To me that is far more important. Sonic characteristics as a whole make up 45% of the entire score - personally I wouldn't up them any more than 50%. Overall tonality to me is a must. You can have an IEM which has good bass, so-so mids (isolated), and poor treble - BUT - have dissonance or problems with cohesion between the 3 sections. Take the Raisel. Some would call the bass good, the mids recessed and the trbel smooth. the reality is that when taken as a whole the bass bleeds (masks) the lower mids and most of the lower treble. As a whole its tonality is awful.

I would imagine debating a standard would take some time - and this might sound a bit pompous, but I'd similarly weight the inputs into the process of deciding - ie give a higher consideration to those who have been reviewing a lot longer. I'm not meaning to be snarky here - its just that in the reviewing game, experience very much trumps enthusiasm (IMO of course).

I think someone who gets paid to do this will have a higher quality of their work actually... Because you're on a critic point where if you lie, you don't get paid anymore, so you need to be honest. I think everyone should get paid for what is basically a job.

I couldn't disagree with this any more vehemently. the quality of the review depends on the experience, the knowledge, and the reviewing and writing skills of the reviewer. Whether they get paid or not has nil to do with quality.

 

[Brooko]

HBB / George (and anyone else). My opening post:

This is an opinion piece – purely my opinion. Feel free to discuss, to agree, to disagree. But please keep it polite, and keep the emotion to a reasonable level. I am calling out the system here – not necessarily the people.

If anyone wants to have further personal discussions (accusations) about who is right or wrong - please take it to PM. We're now talking about potential solutions - which is what I wanted to do in the first place. Talk about the issues, raise awareness, and talk about how we could fix things. I would prefer not to get into slanging matches. I can and will delete posts from this point which start crossing that line.

 

[Dobrescu George]

Shows how we think differently and there would need t be discussion and consensus if it was to be adopted. For instance "fun" factor is not something I would include - it is pointless, overly subjective, and conveys nothing about an IEM or headphone. You can't define it. Comfort to me is huge - I love the sound of the DK3001 but can't wear them because of the comfort. The Dipper I can wear literally for hours. To me that is far more important. Sonic characteristics as a whole make up 45% of the entire score - personally I wouldn't up them any more than 50%. Overall tonality to me is a must. You can have an IEM which has good bass, so-so mids (isolated), and poor treble - BUT - have dissonance or problems with cohesion between the 3 sections. Take the Raisel. Some would call the bass good, the mids recessed and the trbel smooth. the reality is that when taken as a whole the bass bleeds (masks) the lower mids and most of the lower treble. As a whole its tonality is awful.

I would imagine debating a standard would take some time - and this might sound a bit pompous, but I'd similarly weight the inputs into the process of deciding - ie give a higher consideration to those who have been reviewing a lot longer. I'm not meaning to be snarky here - its just that in the reviewing game, experience very much trumps enthusiasm (IMO of course).

I couldn't disagree with this any more vehemently. the quality of the review depends on the experience, the knowledge, and the reviewing and writing skills of the reviewer. Whether they get paid or not has nil to do with quality.

About the standard, that can be discussed.

Yes, that is correct too, the more experience, the more knowledge, the higher the quality. I guess my poing was that someone who got paid would be more neutral or... how to put it. Honest. But that's just my thinking, thinking how to remove more bias.

I wanted to ask a question.

An increase of 6dB is perceived as double the intensity - This means that CL1 having 10 db of treble more of a difference than ie800 means they are about 4 times as more U or V shaped? I'm trying to understand how quantifiable things are.

 

[Brooko]

Thats talking about total SPL. The easiest way to do it is use EQ (not from a DAP - use a PC based EQ), and using my graph (and the RE800) simply adjust the EQ curves. If you want I can send you comparative graphs. Let me know via PM.

 

[sjb57]

To save me doing the maths, @Brooko what is the Maximum Score? In your example, the unit is well built but sounds terrible? Yet gets nearly 3 stars (is that out of 5?) for something not worth buying or is it 3/10? BTW I like graphs to help explain why I might (not) hear something the way you describe otherwise pure prose may be written to convey a command of vocabulary or technical jargon beyond my understanding. Not sure if you have a graph to illustrate "smooth" sound of DAP like X5iii?

As for $45 per review, it seems little more than a Thank You rather than a viable rate per hour even if you could knock out reviews with a boilerplate template, and I'm sure some of the non-returnable samples are worth much more to many reviewers (5-10 reviews to buy a bit of kit).

If a reviewer over eggs a poor product will it get that many more sales or will word get out that it is actually a poor choice or will it at least get a contradictory review by someone else and so buyer beware? Or will the poor review just get buried? If the latter, then the forum is corrupt.

To my mind, a detailed review is very helpful but if 5/10 real customers said it was a poor unit then I would not buy it regardless of the review. But, one bad review would need lots of real customer praise to get me to risk a purchase. Products that do not deserve +4/5 do not deserve my money, regardless of price.

Personally, I'm unlikely to spend +$1000 on headphones or DAP and those that do might not think too much about 'value' because it is all about preferences & diminishing returns. And, even at the mid-lower level, we can compare prices but must then rely on either personal testing or reviews / comments about features & quality of execution. For example, my reading (but never heard either) would suggest the Fiio x7ii is a far better product than the x7 yet not much more money & so as a potential buyer I would deem it better value and probably better value than DX200 on a feature-by-feature basis but if I have £750 is the DX200 better value than the X5iii? How will your reviews / scores help a buyer choose if they can't audition?

Just a layman's thoughts.