How do you audition / compare headphones?

I've had the good fortune to accumulate many headphones of interest (NAD VISO hp50, Sennheiser HD 598, Beats Studio 2013, Parrot Zik, Sennheiser PXC 310 BT. and earbuds Klipsch s4i, Sennheiser CX300II, Apple In-Ear Headphones, Apple Earphones, Apple Earbuds).
 
Recognizing that sometimes the best headphone evaluation is a comparative evaluation, I'd be happy to post the results of comparative listening tests here, if you will guide me as to what tests you use when you evaluate headphones.  This might look a bit like the evaluation posted by MacedonianHero, "Comparisons:  13 of the Top Closed/Portable Headphones Around), seen here: http://www.head-fi.org/t/672743/comparisons-13-of-the-top-closed-portable-headphones-around#post_9622611 .
 
This leads to my question:
 
How do YOU audition or compare headphones?
 
I suspect there is a subjective part, in which you assess the overall sound, comfort, and maybe even looks.
 
But when you LISTEN, what do you do?  What source material do you use?  What acoustic "events," if any, do you listen for?  What system do you use to get the source material to drive your headphones?
 
I spent some time yesterday in purchasing my new NAD VISO hp50s in comparing them to my own Sennheiser HD 598s and the store's HiFiMan HE 400s, Grado PS500 and Grado rs2i. 
 
I used three sources of material, as played on my Apple iPhone 5.  No additional preamp was used (and probably not needed, as all except the Sennheiser had input impedance of 32 ohms or less... Sennheiser was 50 ohms).  Test 1 and 3 were encoded in AAC (256 kbs VBR); Test 2 was encoded losslessly (Apple Lossless).  Separate tests on Test 2 source material convinced me I could not tell any difference between Apple Lossless and AAC at 256 kbps VBR, but I had it that way.
 
1)  "You're Gonna Miss Me When I'm Gone," Band of Heathens: listened for a) difference between voice of vocalist on first verse vs. second; b) clarity of bass glissando in third and subsequent verses;
 
2)  "Spanish Harlem"  sung by Rebecca Pidgeon on the Chesky "The Ultimate Demonstration Disk" (track 3):  listened for a) moistness of plosive consonants (e.g., p in "Spanish"); b) quality of maracas (shaker) that begins in Verse 3... is each shake a bit different, as suggested by the narrative introducing the test?; c) position and uncertainty of position (size of "blob") of those maracas in the sound stage;
 
3) "Saint Saens Sympony No. # - Organ Symphony," Lorin Maazel & Pittsburgh Sympohny Orchestra, Sony Records, Movement IV:  listened for the calliope-like reinforcement of the organ chord by the orchestra 3-1/2 beats later, on the 3rd "verse" where the organ rejoins the orchestra and leads the theme (about 1:16 min in).   Barely audible on this recording, though much clearer on another recording by Charles Munch.
 
I'm new at this, and so I'm really interested in what more experienced folks use to compare the sound quality of headphones. 
 
If I can extract a set of common practical tests from your input, I'll be glad to compare the above-listed 10 headphones according to a prescription that I will prepare and post in advance of the test.  I'll even acquire a modest amount of new source material if you use something that I don't have...OK?

I think it's important to use music *you* know very well. This lets you more easily pick out the differences between the units under test. I also think it's important to explain as best you can in both typical audio terms as well as real words. It drives me crazy when I read reviews that just say things like the "the PRAT was excellent, but not as good as X". I have no idea what that really means - the author may know in his own head what "PRAT" means to him, and I may sort have an idea what it means to me - but it's a fuzzy enough term in audiophilia that it doesn't really add very much useful information. On the other side of the coin, using your own adjectives in descriptive ways *without* defining the terms in relation to the standard audio terms is also annoying. Saying a headphone sounds "limp" or "slack" or "artificial" also does very little to add useful information to the review.

Go for it!!

Try to check other copies of Rebecca Pigeon's "Spanish Harlem." I remember back in my car audio days there was one disc that had really bad clipping when she goes up a few octaves and exaggerated consonants, although in our case then, it was the "t" and "s" a lot more than the "p" (it has a lot to do with the tweeters' sound bouncing off the windshield).

You take your portable rig like this...

 
You go to 3 of Japan's largest chain electronic super stores. Bring a notepad.
 
Listen to the same track on all headphones. Quickly eliminate the ones that distort or clearly sound like crap
 
For the next 2 weeks return and re evaluate the ones on your top 12...whittle it down to 3
 
On the last week spend a lot of time with those 3 and try other tracks on them.
 
Make a decision.
 
If you don't live in Japan just do more driving??
 
If you cannot hear the headphones do not trust this site with your money. Trust your ears. This site loves MTH 50 and the Sony 500,700,1000's and V moda. I found out they were inferior to Ultrasones, Onkyo and JVC after long and direct comparisons. In my opinion. I woulda bought the MTH 50 and V Moda  if I had just gone on this sites recommendations.  Instead I bought the Ultrasones and JVC.
 
Trust your ears and re-evaluate as much as you can. Let your ears recover and go back a week later. Be meticulous.

If I plan to use the model at home, I take an armfull of CDs. Otherwise, I bring my iPod. But I have bolder line between home and portable than many.
 
Either way, I use the same tracks that I know very, very well.

hawaiibadboy said:

You take your portable rig like this...



You go to 3 of Japan's largest chain electronic super stores. Bring a notepad.

Listen to the same track on all headphones. Quickly eliminate the ones that distort or clearly sound like crap

For the next 2 weeks return and re evaluate the ones on your top 12...whittle it down to 3

On the last week spend a lot of time with those 3 and try other tracks on them.

Make a decision.

If you don't live in Japan just do more driving??

If you cannot hear the headphones do not trust this site with your money. Trust your ears. This site loves MTH 50 and the Sony 500,700,1000's and V moda. I found out they were inferior to Ultrasones, Onkyo and JVC after long and direct comparisons. In my opinion. I woulda bought the MTH 50 and V Moda  if I had just gone on this sites recommendations.  Instead I bought the Ultrasones and JVC.

Trust your ears and re-evaluate as much as you can. Let your ears recover and go back a week later. Be meticulous.

Click to expand...

How much does that fancy cable improve your sound?

At 1st it clearly degraded it compared with the FiiO cable. I was livid.
After a week or so it surpassed the FiiO interconnect by miles. I got it for much less than it's advertised now. I also got the silver on by mistake instead of the silver/copper one which is still 130 bucks in Japan. This one is 170-190 bucks.
 
So it sounded crappy at first and now sounds quite spectacular. I wouldn't pay 150+ for it though...a lot of Japanese do however.
 

ruthieandjohn said:

 
I used three sources of material, as played on my Apple iPhone 5.  No additional preamp was used (and probably not needed, as all except the Sennheiser had input impedance of 32 ohms or less... Sennheiser was 50 ohms).  Test 1 and 3 were encoded in AAC (256 kbs VBR); Test 2 was encoded losslessly (Apple Lossless).  Separate tests on Test 2 source material convinced me I could not tell any difference between Apple Lossless and AAC at 256 kbps VBR, but I had it that way.
 
 

Click to expand...

Just because the impedence was low, doesn't mean that you don't need an amp
An amp is important and although it doesn't always improve the sound that you're listening to, it can also make a massive difference even in low impedence headphones. I doubt that you thought a massive pair of cans such as the HE-400 could be powered by your iphone 

hawaiibadboy said:

At 1st it clearly degraded it compared with the FiiO cable. I was livid.
After a week or so it surpassed the FiiO interconnect by miles. I got it for much less than it's advertised now. I also got the silver on by mistake instead of the silver/copper one which is still 130 bucks in Japan. This one is 170-190 bucks.

So it sounded crappy at first and now sounds quite spectacular. I wouldn't pay 150+ for it though...a lot of Japanese do however.

Click to expand...

A perfect example of the placebo effect.

Shhhhh...the folks that pay 6,7,8,$900 bucks for headphones might hear you...then wake up the folks who pay even more for shiny amps with caps and chips found in items 1/8 the cost of what they paid.  
The placebo effect is 80% of the ingredients of this site buba....Shhhhhh

I just happened upon "An Audiophile's Workout: Philips Golden Ear Training," posted at Tyll Hertsen's blog two days ago on innerfidelity.com. It trains, then tests, your listening abilities at four levels, Basic through Gold, examining timbre, detail, spatial impression, bass, and loudness.

It is located here: http://www.innerfidelity.com/content/audiophile-workout-philips-golden-ears-training

I eagerly plan to use it, in hopes of performing and posting comparisons of the headphones I have. Hey, if I'm really diligent, I might even learn to overcome my admiration of Beats headphones, so I can fit in better here!

Simple: run them through the paces you'll have them going through when you use them. I have a handful of "test" tracks that I use that represent the extremes of what I listen to and use them off my iPhone or laptop, go through them a few times each, and that's that. I don't think it's the best idea to use clinical trials or things like that, because that's applying artificiality to your listening. Pull up your music, listen to it, decide what fits it best. 
 
It's music, it's there for enjoyment, it's not a class. Trying to turn it into a clinic with "tests" and metrics, to me, is losing one's way.

As my original post indicated, I am trying to learn to audition headphones.  Several folks provided some advice as to how they compare them, and I discovered the "Philips Golden Ears," also mentioned above, for training in exquisite listening.
 
I am still a LONG way from being comfortable in comparing headphones and reporting results, but I have to relate a funny story that just happened to me along the way.
 
To try to train myself, I thought it would be useful to compare the very best headphone to the very worst, purchasing both, applying my listening tests, and trying to objectify just how the good one sounds better than the bad one.  From that, I would hope to have a series of tests to apply to more fully characterize many of 20 or so headphones that I have.
 
So I go to the "Over Ear Headphone Review" section here, where 90 pages of headphones can be listed in order of decreasing performance.
 
Top of the list? Sennheiser HD 650s.  OK... good choice.  I don't have them, they are $500, and they fill a hole in my headphone inventory of open over ear top quality that does not require a lot of fussing with preamps, etc (I know they need a preamp, but I am hopeful that my tiny Fiio E06 is adequate).  Besides, what is $500 against the prospect of moving forward the frontiers of Headphone Science?
 
Bottom of the list?  Certainly if I am willing to buy a Sennheiser HD 650, I would be willing to buy the worst headphone, which would certainly be cheaper... maybe as low as $3.99!
 
The bottom 30 or so of those 90 pages contain headphones that were never reviewed!  I couldn't indict a headphone as the Worst from that, so I clambered, a page at a time, up the list until I found the lowest ranked REVIEWED headphone.
 
But wait... it was only reviewed by One Person!  Not a very robust condemnation... what if the person was having a bad day?  Maybe I better go with two... no, make that THREE bad reviews (two might need a tie-breaker).
 
So, up the list I climb a bit more, until I find the first headphone with Three (3) negative reviews.   
 
OMG... it doesn't cost $3.99... it costs 100 times that... $399!
 
Do I really want to spend nearly as much on the Worst headphone as on the Best?  Even in the name of Headphone Science??
 
But wait... what kind is it?   It is the Beats Pro!
 
Hallelujah!  I ALREADY have bought it... a recent special offered discounts for buying multiple Beats headphones at once..., plus my pleasure with another Beats headphone, the (new version) Beats Studio 2013!  And I have not yet listened to it, but stocked it away as a birthday present to myself that was coming up.
 
(At this point, gentle reader, if you made it this far, I am sure you are wondering at my sanity!   In my defense, I had found several direct comparisons of the (new) Beats Studio to the Beats Pro, all of which rated the Beats Pro higher, due to better stereo separation and tighter bass. One of these, a YouTube video review, was even given by a person with an authoritative British accent, using such words as "alu-MINI-um" (5 syllables not 4) and "lorry", so I KNEW that review was accurate!)
 
So although the "worst" headphone was 100 times more expensive than I had expected, I had already bought it, understanding it to be even better than its younger brother that I enjoy greatly.
 
But I will continue on my quest for a set of headphone tests that in some manner make objective the subjective measurement of sound quality that we enjoy.  These will be listening tests, not tests of looking at oscilloscope traces of transient response and frequency response.   As a stretch goal, perhaps these tests would be so clear that ANYONE could sit with the headphones, the driver arrangement, and the source material that I use and reach the same conclusion.... reproducibility of results, though awesome, is probably far fetched, but we can try.

OK... out of desperation, I developed a 10-point test paradigm to compare three headphones (two that I was considering, which were the Grado PS500 and the HiFiMan HE 500, and one which was my own that I was trying to improve upon, the Sennheiser HD598.)
 
While I appreciate advice on a subjective method of listening to the material I enjoy with each and deciding which I like best, the engineer in me cried out for something more objective, with tables and rankings and numbers (Oh, My!)  (I'm just that way -- I also have to look at a thermometer to see if I am cold!).
 
I used the same source material mentioned above (Band of Heathens "You're Going to Miss Me," Rebecca Pidgeon "Spanish Harlem" from the Chesky Ultimate Demonstration Disk, and Lorin Maazel/Pittsburgh Symphony Saint-Saens Organ Symphony (#3) Movement IV Maestoso).
 
Roughly half the tests were of a "macro" level, i.e., listening to large segments of each source material for headphone characteristics that applied across the wide variety of acoustic styles.  The other tests were of particular acoustic "events," generally of snippets of only a few seconds in length.
 
My tests used the iPod Touch 5th Generation, which has been measured to have a low output impedance of 0.73 ohms, lower than other iDevices and more able to adequately drive headphones without an amplifier.  Tests did not use an amplifier.
 
Scoring of each headphone was done by simple rank order.  For example, if Headphone A was the best at an uncovered, transparent sound, it got 3 points;  second best headphone of the three got two, and worst headphone got 1 point.  I then added up the scores for each headphone across the 10 categories of tests. A headphone scoring as the best of the three in each of the 10 categories would end up with 30 points; one scoring the worst in all categories would end up with a 10.
 
The categories of tests seem to reflect an emphasis on high-frequency performance, which is perhaps revealing what I was looking for as an "improvement" to my Sennheiser HD 598 -- I wanted to again create the sudden increase in openness, immersion, and transparency that I discovered when I first heard the HD 598s as my first "good" headphones, but even more so than my HD 598s.
 
Here are the "Macro" tests that I used, applying them to significant portions of all three test songs:
 
1. Transparency - less transparency "covers" the music like a scratched, yellow plexiglass window covers and obscures the scene behind it;  more transparency replaces the plexiglass with optical-quality glass, and ultimately, removes it... you are there with the musicians
 
2.  Width - over what solid angle does the music seem to arrive?  Best scores not only surrounded me in music horizontally, but also had more vertical extent;
 
3. Position -- For a duet or similar small number of instruments, can I point to where each instrumentalist is?  How confident of their position am I?
 
4.  Volume  -- how loud was the maximum signal played with iPod turned all the way up? (Edit:  Around July, 2014, I replaced this test with "how *visceral* is the bass, i.e., who well does it give me sensation beyond hearing, such as chattering my teeth?)
 
Then I devised two tests consisting of "acoustic events" for each of the three songs.  These took place over a very short time and in best cases, were at the start of the track, so I could rapidly flicker between one headphone and the other without having to re-cue the acoustic event.
 
5.  Drum quality at start of Song 1 ("You're Going to Miss Me") - can you actually hear a tone, not just a thud, from the bass drum that indicates it is a stretched membrane?
 
6.  Bass quality at third verse of Song 1, where the bass guitar enters with a downward glissando.  Hear the frets as they pass?  For the bass runs, can you actually hear the pitch of the bass sufficiently accurately to musically transcribe it?
 
7.  Bass finger pluck at the start of Song 2 ("Spanish Harlem")  -  is it clear that the bass is actually plucked?  Do you think you can hear the finger hit the string prior to the sound?
 
8.  Shaker quality in third verse of Song 2 -  is each shake distinguishably different? This requires high resolution to preserve.
 
9.  What I call the "ripping reed" vibration effect of the medium low organ tones (and especially horns) on the first chord (organ) and first entrance (horn) in Song 3 ("Organ Symphony"). You can tell that something tangible, either reed or lips, is vibrating besides a column of air.
 
10. The entrance of a calliope-like chord into the din of full organ and orchestra about 1:08 into this track, four beats after the long chord sustained by organ and orchestra, and very nearly masked in this particular recording (I found it much more prominent in the Charles Munch Boston Symphony 1959 recording, then found myself listening for it in this test recording).
 
I have posted the point-by-point comparison results in a thread specifically mentioning the three headphones involved, since that will be more easily found by those seeking headphone comparisons than burying it here.  That post is here: http://www.head-fi.org/t/708912/comparing-grado-ps-500-hifiman-he-500-and-sennheiser-hd-598-quasi-objective-tests#post_10341007 
 
In short, the total scores were:
 
1.  Grado PS-500 - 24 points (out of possible 30) - shone on high-frequency-related aspects such as transparency, organ "rip," and drum "twang"
 
2.  HiFiMan HE 500 - 19 points - shone on huge sound stage with vertical extent and ability to pinpoint instrument positions;
 
3.  Sennheiser HD 598 - 17 points - produced loudest volume from my iPod and was nearly as good as the HiFiMan on the spatial aspects of its sound.
 
As a rough check, I had already auditioned the HiFiMan HE 500 against my Sennheiser HD 598s before devising this test, and though money I had accumulated to buy them was burning a hole in my pocket, I concluded that they were not significantly better, for me, than my HD 598s.  Test scores also seem to say that.  I did end up buying the PS-500s after this test and am delighted.
 
I caution all that the particular tests I chose are biased to measure the particular qualities that I was seeking in my next headphone upgrade. Other folks are likely to get other results if they are seeking other thing, e.g., strong bass.

hawaiibadboy said:

  You take your portable rig like this...

 
You go to 3 of Japan's largest chain electronic super stores. Bring a notepad.
 
Listen to the same track on all headphones. Quickly eliminate the ones that distort or clearly sound like crap
 
For the next 2 weeks return and re evaluate the ones on your top 12...whittle it down to 3
 
On the last week spend a lot of time with those 3 and try other tracks on them.
 
Make a decision.
 
If you don't live in Japan just do more driving??
 
If you cannot hear the headphones do not trust this site with your money. Trust your ears. This site loves MTH 50 and the Sony 500,700,1000's and V moda. I found out they were inferior to Ultrasones, Onkyo and JVC after long and direct comparisons. In my opinion. I woulda bought the MTH 50 and V Moda  if I had just gone on this sites recommendations.  Instead I bought the Ultrasones and JVC.
 
Trust your ears and re-evaluate as much as you can. Let your ears recover and go back a week later. Be meticulous.

Click to expand...

 
You call that portable?