[Ramble] On Music and Testing Headphones...

[aerius]

As I've gained more experience with testing headphones & other audio gear as well as being exposed to more music, I've noticed a few things.

My first "that's odd..." moment came about a couple years ago when a friend lent me the Chesky Ultimate Test CD, which is according to the description, a bunch of well recorded music for testing audio systems. And it sounds pretty good...on all but the most craptastic system. So I filed that useless tidbit away and went on with my life.

Not too long after that BestBuy finally put some Pink Floyd albums on sale and I bought a few of them. Reading through audio reviews on various sites, they often use 'Floyd for testing out gear and comment on depth and layering and a bunch of other stuff like that. So I played it on my headphones (Senn 580, Grado 225) and was confused. Had another "that's not right..." moment. Ok, it sounded nice, yeah I hear that layering stuff, but I don't get it, it sounds great, yay...?

Few months later I ended up with some Norah Jones and went through the same thing again. It sounds good but what am I learning from this? That my headphones can play her songs well? Clearly I needed a better way to evaluate my gear.

While this is going on I'd been going to head-fi meets and testing out headphones. And it was through this that I eventually stumbled across a better way to do things. Instead of playing the standard songs to see what the 'phones are good at, go through my music to find out what they SUCK at. Find songs that'll trip them up and have'em make glaring errors, and this turned out to be surprisingly easy and it told me a heck of a lot.

Backing up a bit, let's say I play some the Cowboy Junkies' Trinity Session CD or Pink Floyd's DSOTM. On pretty much every headphone I've heard it'll sound either decent, good, or great, it never sucks. So that means I have to pay lots of attention while listening and attempt to make value judgements on how good it sounds. That's a lot of work and after a couple headphones I start getting inconsistent. Then I also have to think "is this colour? is this brightness? is this detail? is this good and right?" while doing all this. Too much freakin' work for me.

But if I played songs which screwed headphones up, my job gets a lot easier. Instead of trying to grade the various criteria on a scale of 1 to 10, I can reduce pretty much everything to a pass/fail test, see where the fails are, and then decide if it's worth my time to continue testing.

For instance if I played "Before You" by Chantal Kreviazuk, the AT-A900 and stock AKG K340 will instantly fail the acoustic guitar and female vocal test. Play Tori Amos' "Little Amsterdam" and every Senn headphone will fail the piano tone test. Play some SRV and every Senn & Stax will fail the PRaT test. And given my priorities I can go "ok, that's enough of that, can't live with it, move on to the next 'phone". So instead of going through 5 hours worth of music thinking "that's a 5/10 for this, 7/10 for that, really nice on 'Floyd..." I can quickly write off gear, often in about 2-5 songs so I have more time to listen to the good stuff.

So that's my new approach to headphone & gear testing. Find out what's wrong with it first, then if the faults are acceptable, stick in the "audiophile grade" music and see what it can do. And yes, I do have a chart of every single headphone & speaker I've heard along with the songs that screwed them up along with some brief comments.

Thoughts? Comments?

 

[bigshot]

The problem is that if the music you are using to test has massive imbalances, equipment that happens to have imbalances in the opposite direction will sound better with them. That doesn't mean that the equipment is better... on the contrary, it's worse... but it just happens to have a complementary error.

I've found, like you, that heavily produced music like Pink Floyd isn't the best for testing. The reason is that the effects are what sound good, not the way they are reproduced. A synth bass glissando is going to sound pretty good on any system, and weird phase effects are too.

The best stuff to test with is direct recordings... one with a minimum of processing... and acoustic instruments. With those, you have a built in baseline in your head of what it should sound like, because you have heard an acoustic guitar or violin in person before and you're familiar with the sound.

I'm currently using Paavo Jarvi's Stravinsky collection on the Pentatone label to test with. It's a very direct and clear recording of acoustic instruments, and it shows up errors well.

See ya
Steve

 

[ephemere]

This approach works for me:

Norwitz/Qvortrup article on how to evaluate music playback systems

In a nutshell, they advise not to use reference recordings (comparison by reference), but instead to compare a variety of recordings against each other (comparison by contrast). In a nutshell, they propose that "the more accurate system is the one which reproduces more differences - more contrast between the various program sources."

Even though I try to follow their approach when I have the time to do it properly, it is still difficult to resist the temptation to use reference recordings, as well. But once you find a piece of gear that passes the "comaprison by contrast" test, you'll know it when you hear it.

 

[aerius]

Quote:

Originally Posted by bigshot The problem is that if the music you are using to test has massive imbalances, equipment that happens to have imbalances in the opposite direction will sound better with them. That doesn't mean that the equipment is better... on the contrary, it's worse... but it just happens to have a complementary error.

Sure it's a possibility, but not much of one. The recordings I use aren't the most pristine by any means, but there's no glaring errors on them. Even if there were, my methodology will take care of them. There's enough range & variation in the music selection such that imbalances will be revealed.

Quote:

The best stuff to test with is direct recordings... one with a minimum of processing... and acoustic instruments. With those, you have a built in baseline in your head of what it should sound like, because you have heard an acoustic guitar or violin in person before and you're familiar with the sound.

The Cowboy Junkies "Trinity Session" CD is a direct to DAT deck recording with no processing, and it's absolutely amazing sounding on a good system. Problem. Even on a mediocre system it still sounds pretty nice. Most of the Chesky Ultimate Demo CD is the same way, pristine recording, excellent sound, but next to useless for finding faults. It all sounds pretty good on my Senn 580, which in my opinion is a rather suck-ass headphone.

 

[bigshot]

What's wrong with HD-580s? I've heard they are pretty good.

See ya
Steve