[Foolish Mortal]

Hey Everybody
 
I've been trying to pair down my headphone collection (and maybe make some upgrades) and it's getting difficult. My problem is that I'm having trouble discerning some of the differences in sound quality. Right when I think I've nailed down some favorites, different songs and genres start muddling up my conclusions regarding their sound. I guess what I'm asking is:
 
What methodology do you use to determine the quality of your 'phones?
 
Do you use the same sources, amps and tracks? Do you switch it up? I tried taking notes with just marginal success. When the quality of the headphones being tested is close to begin with, I feel like I completely lose any objectivity in the time it takes to plug in a different pair and adjust the levels. I can easily discern the different sound signatures of various 'phones, but beyond that, I'm just spinning circles.
 
I tend to favor a more analytical approach, but that might not even be very plausable when just listening "by ear." I'm very interested what methods some of you more experienced Head-Fiers use to compare and contrast the quality and value of different headphones.
 
I have purposely avoided mentioning the headphones I'm currently testing because I didn't know if that's relevant to this question or whether it would color the responses I get. If it is important info, please let me know and I'll tell you which ones I'm struggling with. Thanks!
 
Best Regards

 

[elwappo99]

Interesting question! I'm interested to see the responses here. I'm a little curious as to the gear you're attaching the headphones to. I know my amplifiers/sources very well, so when I attach a pair of headphones, I know if they will have enough power, and if some portion of the sound is from the amplifier. 
 
I have a list of higher quality songs that I usually go through when I get a pair of headphones. I compare them to cans I have on hand and see how they fit in. That method usually works for me. But, I've heard a lot of headphones, and can quickly pick out specific qualities of them. 

 

[zazex]

I put 'em on, with one of many favorite CD's, and listen.
If I like the way they sound, they're keepers.
If I don't, back they go.
 
been using this technique for a long time
so far it's worked well for me

 

[Foolish Mortal]

Quote:

I'm a little curious as to the gear you're attaching the headphones to. 

 
Thanks for sharing your perspective on this. My headphones range up to 600ohm Beyers so my source/amp setup can make a big difference. I'm currently testing with various combinations of my low-mid range gear:
 
FiiO e10 / FiiO e17
Asus Essence ST
Zero
O2
Little Dot III
Fournier HTA-1
Marantz 2245
 
At first I tried to test and judge using the same combinations for different headphones, but that didn't workout. Certain source/DAC/amp combos just seem to favor some headphones. This being the case, I was thinking that each should be tested using its most favorable combination so you get the "best" sample you can produce with your gear.
 
That said, in the time it takes me to swap out components and rearrange my setup for another pair, I feel like I lose the immediate sound impression from the previous pair. When trying to discern very small differences in detail that fleeting impression seems important.
 
I did try immediately going back to back using a Y splitter (not ideal, I know) but then I was tied to using the same source/DAC/amp combos again. 
 
Best Regards

 

[elwappo99]

Quote:

 
At first I tried to test and judge using the same combinations for different headphones, but that didn't workout. Certain source/DAC/amp combos just seem to favor some headphones. 

 
Ah! Indeed. The period of the exploring audiophile  It's both frustrating and fun at the same time. You can spend so much time swapping and changing the sound with the different units. As you go higher up, you'll settle on less components, I would suspect, which makes this all easier. 
 
Remember as you're swapping out and changing amps to listen to what your ears like better, not just comparing the headphones. You'll find (hopefully) that some have a flavor you like a little more. 

 

[machoboy]

1000 hours of laura branigan on each headphone until a winner is chosen
 
edit: Now that I think about it, in addition to the standard running them through various output sources (pc soundcard, ipod, audio interface, expensive amp), my favorite test used in the recent "shootout" I did between 4 headphones was plugging them directly into my synthesizer.
 
It's easy to mistake, for example, harsh mid-range for shrill treble on certain songs just because of the instrumentation or way it was mixed. When you're at a keyboard you can really hear the frequency sensitivity in real-time. Some of the headphones really had "ugly" spots on certain keys. The AKGs were no-shows in the lowest octaves and the ultrasones started to distort in the upper mids.
 
Not surprisingly the same headphone that I liked the most across various musical genres was the headphone that had no "ugly spots" when running through 88 keys of raw synth lead (the shures).

 

[obobskivich]

I'll do an initial listen and fitting, and if it's comfortable (as in there's no huge glaring problems anywhere), I will more comprehensively "evaluate" the pair of cans by listening to them with music, videogames, etc (what I want them to be used for) - basically a road test. If I don't have any annoyance from normal use, or compared to other headphones, I keep them.

The "faster" litmus test is I'll listen to my top-dog cans for a few hours (doing whatever I want - whatever music, game, movie, whatever) and then throw the "device under test" on. You will usually pick out huge flaws/disagreements pretty fast doing it that way (for example if they're a lot muddier, brighter, etc than the cans you like most) because those things will stick out like a sore thumb. Subtle differences between sets I like are a lot harder to commit to words - it takes a lot longer to weed those out.

Basically the short answer is: time.

 

[Foolish Mortal]

Quote:

It's both frustrating and fun at the same time.

This is very true. Despite the frustration I have really been enjoying the process, which I guess is the point as much as anything. =]  My wife thinks I'm ridiculous. Every time I ask for her option she pops on a pair and says "these sound really good" with a wry grin.
 
As for finding a preferred flavor of sound, my cognitive dissonance is messing with me. I think I intellectually want to prefer a particular "flavor" but my ears want something different. I don't know if that's a common experience, but I'm trying to let go of the distinction and blend the two together.
 
Best Regards

 

[SoundFreaq]

Quote:

If I like the way they sound, they're keepers.
If I don't, back they go.
 

 
My problem with that is you don't realize their strengths and weaknesses fully until you compare with another headphone, its strengths and weaknesses, that you may be more familiar with. I don't get a rounded idea of how I feel about a headphone until I A/B them with others in my collection, and the long-standing favorites that I am very familiar with. 

 

[Foolish Mortal]

Quote:

my favorite test used in the recent "shootout" I did between 4 headphones was plugging them directly into my synthesizer.
 
It's easy to mistake, for example, harsh mid-range for shrill treble on certain songs. When you're at a keyboard you can really hear the frequency sensitivity in real-time. Some of the headphones really had "ugly" spots on certain keys.
 
Not surprisingly the same headphone that I liked the most across various musical genres was the headphone that had no "ugly spots" when running through 88 keys of raw synth lead (the shures).

This is a cool idea I hadn't even considered! My buddy has a really nice keyboard I might be able to get my hands on for a while. Thanks.

 

[Mad Dude]

For testing & comparing headphones by ear, there are two quintessential requirements:
a) the possibiltiy to switch quickly between different headphones, and
b) level-matching.
 
a) Is important because the human short-time memory "forgets" sound impressions very quickly. I think it's somewhere around five seconds, although I don't know if that is correct. In any case, taking a long time to switch between phones usually dilutes impressions. It can take quite a while to disconnect phone 1, connect phone 2 and set the right volume level. Doing it properly requires, ideally, an amplifier with two outputs, or otherwise two reasonably neutral amps connected to the same source. The difference between the amps is negligible if two different headphones are being compared.
 
b) Is even more essential. Subjective listening impressions are hugely influenced by the volume level, so for any even remotely objective comparisons, level matching is imperative. Level matching by ear is usually impossible or very inaccurate, so a SPL meter should be used - cheap ones can be had for $20 or so.
 
With these basic requirements fullfilled, comparisons should quite easily reveal differences and/or similarities between two headphones. Some people like to use particularly "audiophile" recordings for testing purposes; but I strongly advise against it - really good recordings will sound great on just about anything and will not really reveal a headphone's capabilites or faults. Just use some music that you are familiar with, and listen.

 

[devhen]

Interesting stuff. I'm relatively new to this compared to a lot of you guys but I would agree on time being the key factor. I can notice the more glaring stuff when I first put on a pair of headphones and listen to a few of my reference tracks but it takes weeks, sometimes months, of listening to really feel confident in my knowledge of how a particular set works in all circumstances. That amount of time becomes exponentially longer when you have many multiple combos of sources and amps to factor in... I don't have that problem currently.
 
Having said all that... really poor headphones or headphones I don't like usually take very little time to assess haha.

 

[Foolish Mortal]

Quote:

I'll do an initial listen and fitting, and if it's comfortable (as in there's no huge glaring problems anywhere), I will more comprehensively "evaluate" the pair of cans by listening to them with music, videogames, etc (what I want them to be used for) - basically a road test. If I don't have any annoyance from normal use, or compared to other headphones, I keep them.
 

How much discomfort would you endure for a great sounding headphone? I haven't really been judging them on comfort too much but its definitely worth consideration. I've been tolerating an irritating DT880 premium headband for a while because they sound so good. Maybe it's a more important factor than I originally thought.
 
Best Regards  

 

[Foolish Mortal]

Quote:

b) Is even more essential. Subjective listening impressions are hugely influenced by the volume level, so for any even remotely objective comparisons, level matching is imperative. Level matching by ear is usually impossible or very inaccurate, so a SPL meter should be used - cheap ones can be had for $20 or so.
 

Thanks Mad Dude. Your post mirrors my impressions closely. Regarding level matching, this is really difficult for me. I've found it impossible to do by ear, especially with a wide array of impedances from 32-600ohm 'phones. I hadn't considered using a SPL meter as part of the testing setup. Nice =]  

 

[vid]

I'll first listen to a song with snappy transients, a song with good female vocals, and a song with pleasing bass - all these being songs I'm already familiar with. I might also listen to songs that contain some specific acoustic effects that have caught my interest before and which might betray the level of resolution.
 
Then I listen to a sine sweep and proceed to measure the frequency response, looking for flatness up to 2-3 kHz.
 
After that, I go through my playlist (and amps) and try to decide if I like the sound or not; if there's a specific thing the phones do well and whether I could do without that thing (whether another pair does it better, etc.). This can take some days or weeks.
 
I'll also pay attention to how much sound they leak out when not on my head, i.e. how loud I could potentially play them at 3 am without alerting neighbors.
 
A problem with getting rid of headphones you're uncertain about is that quite likely you'll come to wish you hadn't. So I try not to do it unless I really need to (= been going over the budget).