[ffdpmaggot]

With some luck I'll have dragged a few of the more "sciency" head-fi'ers in with the title for this post.
 
I want to propose a project to the more scientifically inclined crowd on head-fi, the "absolute fidelity index" project, as I'm calling it for now. I'm kind of frustrated that when I'm looking for a new pair of headphones, I need to read highly subjective reviews on what various people think of the sound quality of the headphones, in order to determine which one is probably the best at a given price range. This shouldn't need to be the case, there are objective ways to measure the absolute fidelity of a piece of audio equipment, there is nothing in sound that can't be measured. What I'm proposing, as most of you have probably already guessed, is that we objectively measure, with a percentage, how close the electrical impulses or sound waves produced by a piece of audio gear are. I'd like a few people to respond agreeing that they'd be interested in participating before I type out the full description of how we'd be doing this. Anybody have some good meters and the time to figure out how close the sound reproduced by a pair of cans replicates the electrical impulses from an amp?

 

[Head Injury]

Contact Tyll Hertsens of Inner Fidelity. If you're proposing a new method for measuring headphones, he'll be interested.
 
Measuring transducers is a little more complicated than measuring amps and DACs. Amps and DACs are entirely electrical, so the signal at either end is easy to compare. Transducers are a little more complicated. You can't easily translate compare the electrical signal being fed to the headphones with the sound waves the headphone produces. Reflections in the cup and the seal of the pads, etc. have complex effects on sound decay. Aspects like "soundstage" are easily heard, but hard to measure because the brain perceives them through subtle cues in volume and decay. Headphones in particular don't want flat frequency responses, because of their proximity to the ear and limited use of the outer ear. If you go here, you'll see that a diffuse field equalized headphone doesn't have anywhere near a flat square wave response. I'm looking forward to free field results.
 
All in all I think it would be very difficult to judge headphones on purely objective "performance", unlike DACs and amps. Again, if you think you have a method for measuring it, Tyll would be interested.
 
Also, I suggest you change the title to something that is even remotely relevant to the topic at hand.

 

[ffdpmaggot]

All I'm proposing is a series of recordings played back at very hit bit rates / bit qualities to headphones on a binaural dummy head. Get both the DAC and the ADC to work using the same clock. Play back the 384 / 24 track over the headphones, then record it with the very carefully calibrated mics in the dummy head. Apply a reverse of the mics equalization curve to the recording that you produced, and then run it through a program to measure the difference between each "bar" of the new recording against the old one. Then, add up the values of all of the differences, and then divide that by the total of all of the values in the test track to find the percentage fidelity for the headphones. Shift the headphones around on the head, and repeat the process two to three dozen times. Then average the percentages, and give out a low percentage, a high percentage, a mean percentage, and a median percentage. This should account for varying placement of the headphones, as well as different head topography to get  a "typical" result.
 
You mention soundstage, those sonic cues come from the music itself, not the headphone. The ability to reproduce these cues is based entirely on the headphone ability to faithfully reproduce anything. As for the diffuse field equalization, then apply the reverse of that curve the recording as well.

 

[poikkeus]

 
 
Quote:

You mention soundstage, those sonic cues come from the music itself, not the headphone. The ability to reproduce these cues is based entirely on the headphone ability to faithfully reproduce anything.

Your testing methodology is promising, but your statement on soundstage doesn't fit in with my experience with a pretty wide range of phones. Soundstage relates directly to the distance your ears from the headphone diaphragm. That's why it's just about impossible to get soundstage when you're wearing a pair of IEMs. I've also noticed increased headspace while using some open headphone; the excess sound leaks from your phones, giving the impression of space.

 

[ffdpmaggot]

Quote:

 
 
Your testing methodology is promising, but your statement on soundstage doesn't fit in with my experience with a pretty wide range of phones. Soundstage relates directly to the distance your ears from the headphone diaphragm. That's why it's just about impossible to get soundstage when you're wearing a pair of IEMs. I've also noticed increased headspace while using some open headphone; the excess sound leaks from your phones, giving the impression of space.

I don't think the leakage is the issue, so much as the minimized reverb inside of the housing, anyway, that would be accounted for as any echos would be picked up on the mic and would severely impact percentage fidelity. And soundstage isn't about the distance between your ears and the diaphragm, its about a ton of very complex sonic cues that should not be added in by the headphones. Thats all there is to it, a headphone should not try and apply HRTFs to make the sound seem more distant, this is just very, very bad practice when we're talking about reproducing the sound as it was recorded. You might be talking about a sort of natural "cross feed" which could very easily be measured by playing sound through only one side at a time and taking measurements from the other. Anyway, again, I'm concerned about fidelity, not soundstage. If someone else would pitch a good way to objectively measure soundstage, I already shot tyll an email about this asking for his opinion, then once the details are rounded out I'll start looking around to see who has the gear / how much would it cost to start this project.
 

 

[Uncle Erik]

Amplification will be an issue.

You'll probably go for a flat solid state amp with a flat output impedance curve.

Problem is that many of us use tube amps with unique output impedance curves. Those very much affect headphone response.

Given the variety of tube amps, the index would be inaccurate for many of us.

 

[ffdpmaggot]

The headphone graph is compared with an original graph made with just the amp alone. You could zero the test using the most distorted piece of crap amp you could hope to find from 1980 if it could produce all of the frequencies in the test sweep. The idea isn't to tell you exactly what a headphone will sound like, just how accurately the headphone reproduces the signal fed to it by an amplifier. I don't see how the impedance curves on an amplifier will change how accurately the headphone reproduces the signal that is being thrown at it. If you could explain that, it would be greatly appreciated.

 

[Head Injury]

Quote:

The headphone graph is compared with an original graph made with just the amp alone. You could zero the test using the most distorted piece of crap amp you could hope to find from 1980 if it could produce all of the frequencies in the test sweep. The idea isn't to tell you exactly what a headphone will sound like, just how accurately the headphone reproduces the signal fed to it by an amplifier. I don't see how the impedance curves on an amplifier will change how accurately the headphone reproduces the signal that is being thrown at it. If you could explain that, it would be greatly appreciated.

High output impedance alters the frequency response of low impedance headphones. If you used an amp with a high output impedance, even though the amp isn't colored, low impedance headphones will produce an altered signal while high impedance headphones won't.
 
Benchmark DAC1 would make a good test amp. It has an output impedance of like 0.1 ohms.

 

[tvrboy]

Quote:

I don't think the leakage is the issue, so much as the minimized reverb inside of the housing, anyway, that would be accounted for as any echos would be picked up on the mic and would severely impact percentage fidelity. And soundstage isn't about the distance between your ears and the diaphragm, its about a ton of very complex sonic cues that should not be added in by the headphones. Thats all there is to it, a headphone should not try and apply HRTFs to make the sound seem more distant, this is just very, very bad practice when we're talking about reproducing the sound as it was recorded. You might be talking about a sort of natural "cross feed" which could very easily be measured by playing sound through only one side at a time and taking measurements from the other. Anyway, again, I'm concerned about fidelity, not soundstage. If someone else would pitch a good way to objectively measure soundstage, I already shot tyll an email about this asking for his opinion, then once the details are rounded out I'll start looking around to see who has the gear / how much would it cost to start this project.
 

Truly flat headphones (audiometric) do not give a perceived flat frequency response to the listener. Try a Beyerdynamic DT-48. It's the flattest frequency response I've ever seen. But when listening, it has some problems including seriously rolled off bass and treble. There are also things like tonality that we haven't found good ways to measure yet. A transducer can have perfect frequency response yet have timbre totally unlike a real instrument. I think measurements are necessary, but with the technology we have now it's impossible to "listen" to a transducer with measurements alone.
 
Keep in mind that the majority of people on Head-Fi do not seek neutral, realistic sound. Most members enjoy a colored, fun sound to some degree. If everybody had the OP's priorities, I suspect we would all be listening to HD-600s and Dynahis, or 4070s from KGSSs. Instead people buy highly colored cans like Denons and Grados and all kinds of tube amps. That's why only a few people would pay attention to a rating scale based on accuracy. Even if your idea could be done, very few people would care about it, because it's not what they want.

 

[castlevania32]

Quote:

Truly flat headphones (audiometric) do not give a perceived flat frequency response to the listener. Try a Beyerdynamic DT-48. It's the flattest frequency response I've ever seen. But when listening, it has some problems including seriously rolled off bass and treble. There are also things like tonality that we haven't found good ways to measure yet. A transducer can have perfect frequency response yet have timbre totally unlike a real instrument. I think measurements are necessary, but with the technology we have now it's impossible to "listen" to a transducer with measurements alone.
 
Keep in mind that the majority of people on Head-Fi do not seek neutral, realistic sound. Most members enjoy a colored, fun sound to some degree. If everybody had the OP's priorities, I suspect we would all be listening to HD-600s and Dynahis, or 4070s from KGSSs. Instead people buy highly colored cans like Denons and Grados and all kinds of tube amps. That's why only a few people would pay attention to a rating scale based on accuracy. Even if your idea could be done, very few people would care about it, because it's not what they want.
 

Really ? i think a good amount of people here care about having a hi--fidelity sound.
 
For the dt48, two graphs can be found :


 
i suppose you are talking about the first one,  i don't know why both graphs are so different, if someone could explain ?
Anyway i have an old dt48, and it's not flat, it is vocal centric because it is the way it was designed, for studio purposes and recording engineers (radio station, journalists,etc...)

 

[ffdpmaggot]

Quote:

Truly flat headphones (audiometric) do not give a perceived flat frequency response to the listener. Try a Beyerdynamic DT-48. It's the flattest frequency response I've ever seen. But when listening, it has some problems including seriously rolled off bass and treble. There are also things like tonality that we haven't found good ways to measure yet. A transducer can have perfect frequency response yet have timbre totally unlike a real instrument. I think measurements are necessary, but with the technology we have now it's impossible to "listen" to a transducer with measurements alone.
 

1. That is a consequence of head acoustics. That first graph above was likely done with just a regular spl meter held up to the cups. The latter graph specifically identifies that it was adjusted for head acoustics. Neither of these situations are desirable, as they involve too much assumption and compensation. The test I proposed uses a dummy head like headroom does to account for head acoustics. 
 
2. I'm proposing this test as a way to measure tonality. Admittedly, we can't measure exactly how the cans will sound with this test. I'm sure if someone wanted to, they could create a complex testing method that solves this issue, I just want to know how accurately a pair of cans reproduces the signal fed into them by an amp. If the cans score. say, a 98% on this test, instruments are going to sound **** realistic. A 98% means that you get pretty much flawless reproduction of the sound, and if your reproduction is "near flawless" the instruments should sound exactly as they were recorded. The only reason the instruments wouldn't sound realistic if the headphones produced every "bar" the amp fed them without error is because of a lack of a decent soundstage, the fact that you can't tangibly feel the sound with headphones, and the distortion created by the recording equipment / your dac and amp.
 

 

[Head Injury]

Quote:

2. I'm proposing this test as a way to measure tonality. Admittedly, we can't measure exactly how the cans will sound with this test. I'm sure if someone wanted to, they could create a complex testing method that solves this issue, I just want to know how accurately a pair of cans reproduces the signal fed into them by an amp. If the cans score. say, a 98% on this test, instruments are going to sound **** realistic. A 98% means that you get pretty much flawless reproduction of the sound, and if your reproduction is "near flawless" the instruments should sound exactly as they were recorded. The only reason the instruments wouldn't sound realistic if the headphones produced every "bar" the amp fed them without error is because of a lack of a decent soundstage, the fact that you can't tangibly feel the sound with headphones, and the distortion created by the recording equipment / your dac and amp.

If you do a simple null to determine how similar the signals are, you're not going to be taking into account head acoustics. Every headphone is colored, so the ones that score best would be flattest. The question isn't which headphone is flattest, the question is which headphone most closely matches the ideal equalization to simulate instruments a certain distance away. Most songs are mixed on speakers X feet away, and will sound realistic from X feet away. So doesn't a headphone needs to produce the signal and account for the effects of the outer ear and attenuation caused by the air between the head and ideal listening distance?
 
In other words, true perceived neutrality is only achieved with the same gear in the same position as the sound engineer used, unless the sound engineer is mixing specifically for a certain setup. The signal that's recorded is OKed by the sound engineer because it sounds good from where he's sitting.
 
A headphone that produces the exact signal it's fed will sound "realistic" if realistic is sticking a violin in your ear canal.
 
If you use binaural recordings you would get more accurate results. However, those results won't matter for most of the music out there.
 
P.S. As far as I know, Tyll at Inner Fidelity does use a dummy head. After all, he's the reason Headroom has graphs of their own. Those bottom curves are what the dummy perceives. The upper curve is the only thing that's compensated.

 

[ffdpmaggot]

"If you do a simple null to determine how similar the signals are, you're not going to be taking into account head acoustics. Every headphone is colored, so the ones that score best would be flattest. The question isn't which headphone is flattest, the question is which headphone most closely matches the ideal equalization to simulate instruments a certain distance away. Most songs are mixed on speakers X feet away, and will sound realistic from X feet away. So doesn't a headphone needs to produce the signal and account for the effects of the outer ear and attenuation caused by the air between the head and ideal listening distance?"
 
Thank you, this is an excellent point. The test would need to be done using a dummy head and reference studio monitors positioned exactly where the engineer usually puts them (which can vary from studio to studio) as the reference point to compare with the headphones. This is starting to get ridiculous, the testing method just has to compensate for too many variables to be truly effective. 
 
"P.S. As far as I know, Tyll at Inner Fidelity does use a dummy head. After all, he's the reason Headroom has graphs of their own. Those bottom curves are what the dummy perceives. The upper curve is the only thing that's compensated."
 
Ah, I saw "compensated" on the graph and assumed they ran a computer simulation. I guess not, thanks for clearing that up.

 

[Head Injury]

Quote:

Thank you, this is an excellent point. The test would need to be done using a dummy head and reference studio monitors positioned exactly where the engineer usually puts them (which can vary from studio to studio) as the reference point to compare with the headphones. This is starting to get ridiculous, the testing method just has to compensate for too many variables to be truly effective. 

You wouldn't have to account for every possible speaker position. No one does that with their speakers anyway, and I think engineers know that. The point is, you have to compensate for head acoustics and distance somehow. Even if a headphone measured perfectly flat in either the uncompensated or compensated graphs of Inner Fidelity, that still wouldn't be "neutral". Speakers are EQed flat because they're a certain distance away, not just because of head acoustics. After head acoustics, if a headphone measured flat, it would still sound like the musicians are in your ear.
 
It depends on what your reference for neutrality is. True flat response? Perceived flat response? Speakers in an imperfect room? Or musicians in an acoustically neutral room?

 

[ffdpmaggot]

Quote:

You wouldn't have to account for every possible speaker position. No one does that with their speakers anyway, and I think engineers know that. The point is, you have to compensate for head acoustics and distance somehow. Even if a headphone measured perfectly flat in either the uncompensated or compensated graphs of Inner Fidelity, that still wouldn't be "neutral". Speakers are EQed flat because they're a certain distance away, not just because of head acoustics. After head acoustics, if a headphone measured flat, it would still sound like the musicians are in your ear.
 
It depends on what your reference for neutrality is. True flat response? Perceived flat response? Speakers in an imperfect room? Or musicians in an acoustically neutral room?

I was going to compensate for head acoustics from the headphones with a dummy head. The distance, I presume, could be compensated for by applying an ideal diffuse field equalization curve to the control recording? This way when the original recording is compared with the headphones, the "better" the diffuse field equalization on the headphones, the close they would come to the control recording? My reference for neutrality is the exact sound the engineer intended for listeners to get. I was thinking that the only impact the use of speakers would have on the sound would be limited to a larger soundstage without any notable effects on the timbre of the instruments, but I'm probably wrong.