[Stillhart]

  Yeah, metacritic focus of publishers and idiot gamers wanting/expecting a "9/10 or will buy for 5$" (I understand if money's a concern or plenty of games that want to be played before) often leads to undeserved bad sales. 
 
I think you should get rid of scores and just do comparisons after the main review (like Project86 does in his amp reviews), based on the price range, like Q701 vs. X1, HE-500 vs. TH-600 etc.and even with headphones above so people might even go higher.
 
Will also make people actually read your review and not stare on scores which is a bummer since your reviews are well written.

FWIW, I read the reviews and use the numbers more to help me understand the reviews in summary form.  But I still think it would be a great idea to do some comparisons in the price ranges.
 
I don't know if you guys realize how intimidating a 1400-page thread is.  I normally like to read everything on long informative threads, but it'd take all my free time for a month to get through everything.  I've scanned back a few tens of pages and I've done some searches, but man it seems like there's not much way to get relevant recent information other than asking things you may have answered before. 
 
I hope you guys cut newbies like me some slack going forward.  Nobody likes feeling like a ******* tool for asking the same question again, but there isn't a ton of choice.  You're a victim of your own success.

 

[Change is Good]

This is really annoying. Shipment was delayed for my K545, so they did not arrive until today.
 
And guess what,,, the left driver is defective... AGAIN! No driver crinkling issues, this time, however... just significantly uneven volume compared to the right driver.
 
This is extremely odd because Brian said he tested these out before shipping. It seems that these headphones may not respond well to cold weather environments during shipment. This is the only theory I can think of....

 

[Fegefeuer]

 
FWIW, I read the reviews and use the numbers more to help me understand the reviews in summary form.  But I still think it would be a great idea to do some comparisons in the price ranges.
 
I don't know if you guys realize how intimidating a 1400-page thread is.  I normally like to read everything on long informative threads, but it'd take all my free time for a month to get through everything.  I've scanned back a few tens of pages and I've done some searches, but man it seems like there's not much way to get relevant recent information other than asking things you may have answered before. 
 
I hope you guys cut newbies like me some slack going forward.  Nobody likes feeling like a ******* tool for asking the same question again, but there isn't a ton of choice.  You're a victim of your own success.

 
I didn't address you in my posting, it was just a general thought about ratings and scores, not meant towards you personally. 

 

[roguegeek]

I've watched this thread since Dec 2012. There are things that I agree with and there are things I have a different opinion on. No doubt, do I appreciate the content here as it has helped me down my path to a very large extent. It's obvious that this kind of comparison is very much needed since it is one of the most popular and active threads here on Head-fi. Please understand that I have nothing but respect and admiration for everyone who contributes here.
 
That being said, I've seen more than a handful of people confused on the scoring system and I honestly don't blame them for that misinterpretation. The subjectiveness is fine, of course, but it makes sense people are going to be looking at this as less of a guide and more of a comparison simply because scores are provided over different factors. The confusion is emphasized when you discover the numbers are not relative. Beyond this, not having a comprehensive description of the methodology used when "scoring" doesn't help the situation. It is also my opinion that retorts such as, "why don't people read?" to their questions about the scoring system don't help matters. I write a fair amount of technical documentation in my role and if I have a lot of people asking the same question about something specific, I look at that as a failure on my part to properly communicate what I am trying to document.
 
In contrast, you have something like Battle Of The Flagships where a scoring system is employed, it's comparative, and there is a decent amount of time spent on describing the methodology. My constructive feedback would possibly then be to work this guide towards something more extensive and complete like the flagship comparison or possibly keep it in the spirit of a guide where things don't have hard scores and impressions are emphasized instead.
 
So, MLE, I hope you or anyone else who has contributed doesn't takes offense to this. That was not my intent. I, in fact, will always be thankful of what you've created. I just see a good piece of documentation that, in my opinion, has room to grow and can possibly become great. This is my way of communicating constructive feedback and, as always, I'm open to hearing thoughts and feedback.

 

[Evshrug]

I mean, I've been reading this thread long enough that I know I can read Mad's in-thread posted reviews, and get a good sense of how excited he is for a particular product. And, I pretty much ignore the scores, mostly just descriptions of imaging, soundstage, and realism catch my interest — because I've figured out my main preferences.

Not everybody else has had those preference discoveries though, and to them I would just say "look at your budget, and any of the headphones in this guide are some of the best headphones out there. Pick one that fits your budget and the review catches your interest."

I suppose I would also recommend removing the numbered scores, just letting the enthusiasm for the product (for it's price bracket) speak for itself of how good "good" is.

I can understand Mad's frustration... This isn't my thread, but still it bugs me when somebody asks "Does the X1 have good bass?" or "I found a good deal on a top-shelf AKG [they're ALL great deals ], if I want one should I upgrade from my turtle beaches/TV speakers?" or "Can I hear footsteps and get good surround imaging with Koss clip-ons [Yes!]" or other things that were clearly answered on the first post, or are the kind of thing where the person has all the info but wants someone to choose for him. To an extent, new posters are what give this thread it's life, it's just that some people expect silver platter service from other people who are simply hobbyists just like them.

 

[Stillhart]

To be fair though, I think a lot of the vets in this thread don't seem to recognize the difference between what's on the front page and what's "common knowledge" but never explicitly stated.  This can cause frustration for everyone.
 
Back on topic, I just went to Fry's to grab something and by complete chance stumbled across a SB Recon 3D on super clearance for $5!  I bought both of them so I can have one for the PC and one for the console if I decide to go that route.  I figure if I don't like it, it's only $5 down the drain. 
 
But from the comparison video in the first post, I thought the THX sounded much better than the Dolby Headphones.  The Dolby had some weird reverb going on that was annoying me; the THX sounded much more natural while also giving the best imaging for me.  Seemed like a no-brainer to grab these.
 
On a related note, I also just discovered that my old AVR (an HK AVR 340) does Dolby Headphone 5.1!  Question for you vets:  will this negate my need for an amp on something like the DT990 or the Q701?  I understand that the mixamp doesn't really amp those quite enough so an external amp is required.  I am wondering if my AVR counts as more amplification than the mixamp.

 

[AxelCloris]

  To be fair though, I think a lot of the vets in this thread don't seem to recognize the difference between what's on the front page and what's "common knowledge" but never explicitly stated.  This can cause frustration for everyone.
 
Back on topic, I just went to Fry's to grab something and by complete chance stumbled across a SB Recon 3D on super clearance for $5!  I bought both of them so I can have one for the PC and one for the console if I decide to go that route.  I figure if I don't like it, it's only $5 down the drain. 
 
But from the comparison video in the first post, I thought the THX sounded much better than the Dolby Headphones.  The Dolby had some weird reverb going on that was annoying me; the THX sounded much more natural while also giving the best imaging for me.  Seemed like a no-brainer to grab these.
 
On a related note, I also just discovered that my old AVR (an HK AVR 340) does Dolby Headphone 5.1!  Question for you vets:  will this negate my need for an amp on something like the DT990 or the Q701?  I understand that the mixamp doesn't really amp those quite enough so an external amp is required.  I am wondering if my AVR counts as more amplification than the mixamp.

 
MLE did explicitly state that the scores should not be compared to one another. It's the third sentence on his initial post. "

I NEED TO STRESS THIS. DO NOT COMPARE SCORES FROM ONE REVIEW TO ANOTHER. EACH SCORE IS REFLECTING A HEADPHONE'S OWN MERITS, AND NOT COMPARED TO ANOTHER UNLESS I SPECIFICALLY STATE SO.​

" Common knowledge may state that scores are a direct comparison but common reading comprehension should prevent misunderstandings.
 
$5 for a Recon3D USB is a crazy deal. Too bad there are none nearby, I'd stop in and pick up a spare for travel purposes. THX vs DH is a topic where I bounce back and forth. I prefer THX for some games and DH for others. And your AVR340 could potentially drive headphones very well. MLE ran his headphones off a Marantz receiver for a while before it crapped out on him. Best way to tell if yours is suited for the task is to try it out and see.

 

[Stillhart]

   
MLE did explicitly state that the scores should not be compared to one another. It's the third sentence on his initial post. "

I NEED TO STRESS THIS. DO NOT COMPARE SCORES FROM ONE REVIEW TO ANOTHER. EACH SCORE IS REFLECTING A HEADPHONE'S OWN MERITS, AND NOT COMPARED TO ANOTHER UNLESS I SPECIFICALLY STATE SO.​

" Common knowledge may state that scores are a direct comparison but common reading comprehension should prevent misunderstandings.
 
$5 for a Recon3D USB is a crazy deal. Too bad there are none nearby, I'd stop in and pick up a spare for travel purposes. THX vs DH is a topic where I bounce back and forth. I prefer THX for some games and DH for others. And your AVR340 could potentially drive headphones very well. MLE ran his headphones off a Marantz receiver for a while before it crapped out on him. Best way to tell if yours is suited for the task is to try it out and see.

Of course, that was made very clear.  I wasn't really referring to comparing scores, as I thought I made it pretty clear this morning that MLE misunderstood the intent of my question (because I worded it poorly).  I was talking more about other things.  Here's an example, and this may sound really dumb to you guys, but bear with me.  I actually am STILL not 100% sure if the Mixamp 5.8 is just a wireless transmitter or if it actually does the DH conversion as well.  It's never explicitly stated anywhere and since the product is discontinued, I couldn't even find the info on the Astro site.
 
Thanks for the info on the AVR.  I had to grab a 1/4" to 1/8" adaptor so I could test my (super crappy) headset with the AVR.  I'll know more tonight on how good it sounds on ****ty cans.

 

[AxelCloris]

  Of course, that was made very clear.  I wasn't really referring to comparing scores, as I thought I made it pretty clear this morning that MLE misunderstood the intent of my question (because I worded it poorly).  I was talking more about other things.  Here's an example, and this may sound really dumb to you guys, but bear with me.  I actually am STILL not 100% sure if the Mixamp 5.8 is just a wireless transmitter or if it actually does the DH conversion as well.  It's never explicitly stated anywhere and since the product is discontinued, I couldn't even find the info on the Astro site.
 
Thanks for the info on the AVR.  I had to grab a 1/4" to 1/8" adaptor so I could test my (super crappy) headset with the AVR.  I'll know more tonight on how good it sounds on ****ty cans.

 
Yes, the Mixamp 5.8 handles both the transmission/reception of the wireless signal and the base unit itself does the processing for Dolby Headphone. Since your receiver can handle it, you wouldn't need a Mixamp. You could use another wireless transmitter to go wireless, but you don't need a device to process the sound.

 

[roguegeek]

   
MLE did explicitly state that the scores should not be compared to one another. It's the third sentence on his initial post. "

I NEED TO STRESS THIS. DO NOT COMPARE SCORES FROM ONE REVIEW TO ANOTHER. EACH SCORE IS REFLECTING A HEADPHONE'S OWN MERITS, AND NOT COMPARED TO ANOTHER UNLESS I SPECIFICALLY STATE SO.​

" Common knowledge may state that scores are a direct comparison but common reading comprehension should prevent misunderstandings.

I agree, this should be specific enough if someone is reading everything. Yet, the confusion exists. I would imaging this has to do with there just being a lot of content in the guide and the people who are going to check it out are going to ingest only what they feel like is value-added for them. So maybe, as a help to readers since this is suppose to be a digestible guide, there is something that can be done to make that information louder. I had a couple suggestions above, but I don't know what's going to be right because I'm not the one telling the story. It should just be looked at as something that could use some improvement. I mean, do we not agree it could be better or is there really a stance of, "this is as good as it's going to get?" If that's the case, I'll gladly refrain from anymore feedback on this subject.

 

[Sam21]

Question regarding THX TruStudio Pro on Creative Soundblaster X-Fi Go! Pro :

 
 
There is an option in the driver software that lets you turn on the THX surround, But there is no option to select  5.1 or 7.1 input channels.
 
Also, when I go to playback devices/choose Go!Pro/configure, I can only choose 2 channels there... Can someone explain to me what is happening here ?
 
We can only feed the sound card with two channels ?

 

[chicolom]

I understand why a non-comparative scoring system is used.  It's really no different than how a headphone would get reviewed on a site like Amazon.  Take the KSC75 for example.  It's an amazing headphone and absolutely deserves 4-5 stars.  No consensus of people would agree to rate the KSC75 as being only a 2-3 star headphone, but with a relative scoring system it would need to be down there to make room for the better headphones.  The KSC75 is good, but it isn't on par with the best mid-fi phones.  Even the best mid-fi headphones would need to make room for things like flagships, so they would be closer to 3-4 stars only.  4-5 stars would be reserved for the absolute best (flagships).
 
 
At the same time, I can understand the confusion of using the non-comparative scoring system, especially since people tend to skip over the bolded text emphasizing it.    
 

 
In contrast, you have something like Battle Of The Flagships where a scoring system is employed, it's comparative, and there is a decent amount of time spent on describing the methodology. My constructive feedback would possibly then be to work this guide towards something more extensive and complete like the flagship comparison or possibly keep it in the spirit of a guide where things don't have hard scores and impressions are emphasized instead.

 
Are you just suggesting that MLE should sort the headphones by "absolute performance" instead of the current "by price" sorting?
 
The only actual scores given in David Mahler's thread are the "For the Price" letter grades he assigns headphones (ie, value), where something like the SR009 scores lower than an AKG K501.  I feel like sorting by absolute performance wouldn't necessarily be a solution, as people would then keep asking things like "I see you put the Q701 higher than the HD558 on the list, but exactly how much better is the Q701 than the HD558???" 
 
No matter which route you go with, there's no way to quantify exactly how much better one headphone is than the other - which is a common question that new members repeatedly ask about headphones.  Ranking by absolute performance (like David Mahler's Thread) won't answer that either.  The only way to quantify them would be to assign an absolute performance score (not just an ordered ranking) to each headphone, but I don't think two similar headphones should try and be quantified in that way as personal preferences play such a huge role.  People would just look at the number and say "Oh, the HE400 has higher overall score than the MA900.  I guess I should go with the HE400 then."  Even though they might find the recessed mids on the HE400 to be a major problem (whereas the mids are a strength on the MA900) - something they could have gleamed if they had read the review instead of just comparing scores and going with the best scoring one. 
 
Ultimately headphone reviews are not and should not be about numbers, as even if two headphones scored exactly the same they will still sound very different.  There's no quick way for someone to "cheat" and quickly assess a headphone's sound by just reading a numbered score.  You have to read the review in order to try understand what the headphone's own personal signature is and its soundstage performance in order to get an idea of how it sounds.  This is what makes researching and shopping for headphones tough (and of course the fact that the majority of them aren't available to demo at stores).  After trying different headphones for a while and developing some experience and your own personal tastes it then becomes easier to understand how a headphone will sound by reading a review - and more importantly whether or not YOU will like it. 

 

[Stillhart]

...People would just look at the number and say "Oh, the HE400 has higher overall score than the MA900.  I guess I should go with the HE400 then."  Even though they might find the recessed mids on the HE400 to be a major problem (whereas the mids are a strength on the MA900) - something they could have gleamed if they had read the review instead of just comparing scores and going with the best scoring one.   
Ultimately headphone reviews are not and should not be about numbers, as even if two headphones scored exactly the same they will still sound very different.  There's no quick way for someone to "cheat" and quickly assess a headphone's sound by just reading a numbered score.  You have to read the review in order to try understand what the headphone's own personal signature is and its soundstage performance in order to get an idea of how it sounds.  This is what makes researching and shopping for headphones tough (and of course the fact that the majority of them aren't available to demo at stores).  After trying different headphones for a while and developing some experience and your own personal tastes it then becomes easier to understand how a headphone will sound by reading a review - and more importantly whether or not YOU will like it. 

See you call out a major problem here, which is that (for the most part) there's not really any way for us to try these for ourselves without buying and returning.  And if we're relatively new to the head-fi thing, something like "recessed mids" is not really helpful.  "7/10" is a lot easier to understand for noobies, which is suspect was the intention originally.
 
Honestly, and I really don't mean this as a dig at MLE, the only good reason to use relative ratings is because it's a LOT of work to go back and update the ratings when things change.  Not to mention MLE may not have the items anymore to re-test against new competition.  That's perfectly reasonable.
 
But in return, I'd expect MLE and others to understand that the result of that is that people are going to be a little confused and a little adrift and they're going to cling to the only thing that makes sense... "7/10".
 
Maybe it'd work to have absolute ratings within a given bracket?  Like a 7/10 in the $300+ range could be better than a 7/10 in the $150-300 range.  This would be a little more concrete and allow slightly more apples-to-apples comparisons for noobies, while still allowing the flexibility of the relative ratings system.

 

[Sam21]

I personally think that MLE's guide is misleading, and sometimes contradictory, He says for instance that the Q701 is 98% as good as the K701 in the competitive category, yet he gives it a 9.5 score. Also, There is no way in hell you can measure these things like soundstage and recession of mids. it is barely noticable, it isn't like a mathematical computation where things are exact and you can assign definite values to certain variables. Heck, even the Headphone makers themselves can't do that. Moreover, he doesn't say what the source/DAC/AMp he used to test each headphone.
 
 
Just my Two cents

 

[Eric_C]

  Moreover, he doesn't say what the source/DAC/AMp he used to test each headphone.

 
Isn't it always the MixAmp, and an additional amp if necessary?