Toward comparing pairs of headphones
Apr 30, 2015 at 10:26 AM Thread Starter Post #1 of 27

ruthieandjohn

Stumbling towards enlightenment
(Formerly known as kayandjohn.)
Joined
Feb 2, 2014
Posts
4,125
Likes
2,692
Location
Poulsbo, WA, USA
The ability to compare, accurately and reproducibly, one headphone pair to another is a fundamental element of assessing headphones.
 
Ideally, we would be able to perform a blind test, where we listened to one pair, immediately listened to the other pair, could repeat, mark the results, and do this all without knowing the identity of either headphone.
 
Less than ideally, but still good, we would have some way of capturing and recording the sound of each headphone as it entered our ear canal, and then play back that sound with certainty that the recorded sound, interrupted by capture on its way to our eardrum, would sound just the same as the original. We could then switch back and forth between headphones (or more generally entire audio chains) without unplugging, removing and donning headphones, or knowing which we were listening to at the moment.
 
And if the interrupted sound did not sound just the same, perhaps it would maintain relative properties to the original, i.e., if in the original Headphone A had better soundstage and poorer bass than Headphone B, these replicated versions of their waveforms into our ear would have the same.
 
Perhaps the use of a calibrated head-and-torso simulator (e..g, the Bruel & Kjaer 4128C), in which we record the signal within each ear, could suffice.
 
hats-type-4128-c-with-headphones.ashx

 
In a blind listening test, if the headphones are indistinguishable, we would expect to correctly identify each one 50% of the time, as a result of random selection similar to flipping a coin.
 
If we are really capable of distinguishing them, we would correctly identify each 100% of the time.
 
But practically, our ability ranges somewhere between 50% and 100%.
 
I find as I listen to two headphones, I follow a hierarchy of ability to distinguish or identify them:
 
  • Hear a difference:  Listening to one immediately followed by another, I can tell that they sound different.
  • Characterize the difference:  Again, hearing one right after the other, I can describe the differences, e.g., "Headphone A has more bass, but less bass tone, while Headphone B has better treble detail, less bass, but more bass tonality (e..g, high frequency harmonics of the bass);"
  • Recognize the difference:  I can, in a blind test hearing one right after the other, correctly declare "That is Headphone A (because it has more bass, though less bass tone")
  • Remember the difference:  I can in a blind test with each comparison separated by time (and intermediate listening to other things) accurately make that same declaration of which headphone is which
  • Recall the difference:  After being away from the comparison for several hours, I can listen to just one of the headphones and say, without needing to hear the second, which headphone it is.
 
Perhaps these levels of distinction can map onto some score in a blind test, e.g., Stage 1 ("Hear") might score me 60% correct, a bit better than random but not much, while #5 ("Recall") might predict a 100% score in a blind listening test.
 
I find that in the binary headphone comparisons that I do, I regularly can get through Step 3.  Step 4 is harder and Step 5 is rare indeed.
 
After all of this comparing, remembering, and recalling, I still have not actually assessed the results (e.g., "Headphone B sounds better than Headphone A.")  That may not actually be a problem, because I find, particularly with high-end headphones, the differences are in sound quality and how that matches the preference of the listener (and the nature of the source material), rather than collapsing a number of variables (sound stage, bass, mid, treble, etc.) into a single "better / worse" value judgement.
 
How does your comparing of two headphones match my view above?  Thanks!
 
Apr 30, 2015 at 1:14 PM Post #2 of 27

bigshot

Headphoneus Supremus
Joined
Nov 16, 2004
Posts
22,547
Likes
4,462
Location
Hollywood USA
Blind tests are most useful in determining whether a difference exists between two very similar sounds. Headphones rarely sound very similar. It seems to me that careful sighted listening with clear criteria for judging would do fine for headphones.
 
Apr 30, 2015 at 6:52 PM Post #3 of 27

castleofargh

Sound Science Forum Moderator
Joined
Jul 2, 2011
Posts
9,363
Likes
4,748
headphones suck too much at reproducing signal fidelity with homogeneous FR, so while listening to them is good enough for me to tell if I enjoy them or not, and a few other massive characteristics, I don't believe it can substitute to measurements.
 
but when I can't find measurements, my way of testing is to use test tones to find out how far it extends(well at least within my own hearing abilities), use some sine sweep to try and find out if there are massive spikes(somehow I often fail at hearing drops, but spikes are usually easy to notice and to hate). then I proceed to try and EQ those spikes out and maybe the overall response if it's too colored for me. and if I can't seem to succeed I decide the headphone is useless to me.
so as you can see I consider FR as the highest criteria because that's what I am most sensitive to, and feel like most of all the talks about soundstage, texture and stuff are vastly a direct consequence of the FR(at least as long as distortion values don't go over 1% too much).
 
Apr 30, 2015 at 7:13 PM Post #4 of 27

bigshot

Headphoneus Supremus
Joined
Nov 16, 2004
Posts
22,547
Likes
4,462
Location
Hollywood USA
yup. agree with all of that.
 
Apr 30, 2015 at 7:53 PM Post #5 of 27

ruthieandjohn

Stumbling towards enlightenment
(Formerly known as kayandjohn.)
Joined
Feb 2, 2014
Posts
4,125
Likes
2,692
Location
Poulsbo, WA, USA
  headphones suck too much at reproducing signal fidelity with homogeneous FR, so while listening to them is good enough for me to tell if I enjoy them or not, and a few other massive characteristics, I don't believe it can substitute to measurements.
 
but when I can't find measurements, my way of testing is to use test tones to find out how far it extends(well at least within my own hearing abilities), use some sine sweep to try and find out if there are massive spikes(somehow I often fail at hearing drops, but spikes are usually easy to notice and to hate). then I proceed to try and EQ those spikes out and maybe the overall response if it's too colored for me. and if I can't seem to succeed I decide the headphone is useless to me.
so as you can see I consider FR as the highest criteria because that's what I am most sensitive to, and feel like most of all the talks about soundstage, texture and stuff are vastly a direct consequence of the FR(at least as long as distortion values don't go over 1% too much).


Problem I have is that the FR (Frequency Response) plotted is always the *magnitude* frequency response... no separation of real and imaginary components. 
 
Who cares?  Well, to exactly reproduce a signal, you need to accurately pass both its real and imaginary frequency components (i.e., you gotta get the phase as well as the energy right, at each and every frequency). 
 
Phase (I think) governs our perception of soundstage, imaging, and positional placement of instruments, as well as probably more subtle things.
 
If your headphone + amp system delays signals of different frequencies differently, via frequency-dependent phase delay, your resulting signal will be distorted, even if the frequency response is flat.
 
Apr 30, 2015 at 10:00 PM Post #6 of 27

RRod

Headphoneus Supremus
Joined
Aug 25, 2014
Posts
3,371
Likes
970
 
Problem I have is that the FR (Frequency Response) plotted is always the *magnitude* frequency response... no separation of real and imaginary components. 
 
Who cares?  Well, to exactly reproduce a signal, you need to accurately pass both its real and imaginary frequency components (i.e., you gotta get the phase as well as the energy right, at each and every frequency). 
 
Phase (I think) governs our perception of soundstage, imaging, and positional placement of instruments, as well as probably more subtle things.
 
If your headphone + amp system delays signals of different frequencies differently, via frequency-dependent phase delay, your resulting signal will be distorted, even if the frequency response is flat.

 
The hard part is that a narrow frequency adjustment in the main audible band can cause noticeable pre-ringing (ironic I'm the one bringing that up) if the phase is left untouched, so it's a give and take. But you're right in that, without a room of reflections muddying the works, headphones can make it easier to notice phase distortions over speakers. Still, you have to factor in the relative merits of accurate frequency vs. phase response. See:
 
http://ethanwiner.com/phase.html
http://www.silcom.com/~aludwig/Phase_audibility.htm
http://www.audioholics.com/room-acoustics/human-hearing-phase-distortion-audibility-part-2
 
Apr 30, 2015 at 10:28 PM Post #7 of 27

bigshot

Headphoneus Supremus
Joined
Nov 16, 2004
Posts
22,547
Likes
4,462
Location
Hollywood USA
The threshold of audibility for phase shift is between 1 and 3 ms. Maybe I'm missing something, but I don't know how headphones could get phase distortion of that magnitude.
 
Apr 30, 2015 at 11:48 PM Post #9 of 27

bigshot

Headphoneus Supremus
Joined
Nov 16, 2004
Posts
22,547
Likes
4,462
Location
Hollywood USA
See group delay: http://sound.westhost.com/ptd.htm
 
This one has some more info... http://www.audioholics.com/room-acoustics/human-hearing-phase-distortion-audibility-part-2
 
Feel free to let me know if I'm missing something, because a big chunk of this flies over my head.
 
Also keep in mind that whenever I talk about thresholds, it relates to listening to music in a home environment, not pink noise, tones or square waves. 1 to 3 ms is being generous.
 
May 1, 2015 at 1:16 AM Post #11 of 27

bigshot

Headphoneus Supremus
Joined
Nov 16, 2004
Posts
22,547
Likes
4,462
Location
Hollywood USA
We discuss speakers in Sound Science too.
 
May 1, 2015 at 8:31 AM Post #12 of 27

ruthieandjohn

Stumbling towards enlightenment
(Formerly known as kayandjohn.)
Joined
Feb 2, 2014
Posts
4,125
Likes
2,692
Location
Poulsbo, WA, USA
  See group delay: http://sound.westhost.com/ptd.htm
 
This one has some more info... http://www.audioholics.com/room-acoustics/human-hearing-phase-distortion-audibility-part-2
 
Feel free to let me know if I'm missing something, because a big chunk of this flies over my head.
 
Also keep in mind that whenever I talk about thresholds, it relates to listening to music in a home environment, not pink noise, tones or square waves. 1 to 3 ms is being generous.


Good references, again stating that except in certain contrived circumstances, listeners cannot hear the effect of phase reliably on what appear to be sustained multi-frequency signals, and when they can, it is easier done on headphones than with loudspeakers.
 
An article by (Susan) Flanagan et al. also agrees with this, using square waves, Fourier transforming them, then delaying phases of different frequency differently, inverse FT-ing them, and playing them.  They are no longer square waves, but having the same frequency component, they sound the same.
 
She then goes on to say that *transients* are influenced (fuzzed up) by frequency-dependent phase delays.  Makes sense... a transient is a change in the overall energy envelope of a signal where all frequencies need to increase in intensity at once... messing with their phases differently would seem to smooth out the transient a bit.
 
Should be easy to replicate this experiment.
 
But back to my topic (which I admit I threw us off track when I first mentioned phase fidelity)...
 
Do you use any "degrees of discrimination" as listed in my 5-point scale in the original post when you compare two headphones?  How do you capture your confidence of a difference?  Seems to me you can either apply them in an overall compare, or in a comparison of individual acoustic features ("Headphone A has wider soundstage than B; Headphone B has stronger subbass than A," etc.)
 
May 1, 2015 at 8:35 AM Post #13 of 27
Joined
Nov 24, 2011
Posts
437
Likes
524
Location
Buffalo, New York
Agreed. We can 'see' anomalies on the surface of the planar diaphragm that would no doubt cause phase shifts at various frequencies that do not show up in standard measurements yet are quite audible. B&W has used laser interferometry since the early 80's to see standing waves on driver and cabinet surfaces with the ultimate goal to reduce or eliminate.


Problem I have is that the FR (Frequency Response) plotted is always the *magnitude* frequency response... no separation of real and imaginary components. 

Who cares?  Well, to exactly reproduce a signal, you need to accurately pass both its real and imaginary frequency components (i.e., you gotta get the phase as well as the energy right, at each and every frequency). 

Phase (I think) governs our perception of soundstage, imaging, and positional placement of instruments, as well as probably more subtle things.

If your headphone + amp system delays signals of different frequencies differently, via frequency-dependent phase delay, your resulting signal will be distorted, even if the frequency response is flat.
 
ABYSS Headphones We engineer, machine, and build our headphones from scratch in New York, USA. Stay updated on ABYSS Headphones at their sponsor profile on Head-Fi.
 
https://www.facebook.com/AbyssHeadphones https://twitter.com/AbyssHeadphones https://www.instagram.com/abyssheadphones/ https://abyss-headphones.com/ info@Abyss-Headphones.com
May 1, 2015 at 12:20 PM Post #14 of 27

bigshot

Headphoneus Supremus
Joined
Nov 16, 2004
Posts
22,547
Likes
4,462
Location
Hollywood USA

  Good references, again stating that except in certain contrived circumstances, listeners cannot hear the effect of phase reliably on what appear to be sustained multi-frequency signals, and when they can, it is easier done on headphones than with loudspeakers. An article by (Susan) Flanagan et al. also agrees with this, using square waves, Fourier transforming them, then delaying phases of different frequency differently, inverse FT-ing them, and playing them.  They are no longer square waves, but having the same frequency component, they sound the same.

 
When you use the term transient, you need to define if it is being used to refer to the waveform or the dynamics of the music. Phase shift isn't even in the same universe as musical transients. Square wave performance doesn't matter. Speakers do poorly at reproducing square waves, but they sound excellent with music. I listen to music with headphones, so the meaningful specs are the ones that relate to music. For the purposes of listening to music in the home on headphones, phase shift seems to be pretty much of a non-issue. With room acoustics phase shift can be a big problem, so it isn't as much the transducers as it is the space around them.
 
As for the five points... I generally audition headphones using music I am familiar with and look for ones that come closest to sounding like my speaker system, which I've put a lot of work into getting to sound the way I want it to sound.
 
Accuracy is only important as it relates to our ability to hear and the purposes we use headphones for. It's nice to cross every T and dot every I, and I suppose some people enjoy thinking about that stuff, but I just want accurate reproduction of music.
 
May 1, 2015 at 12:55 PM Post #15 of 27

ruthieandjohn

Stumbling towards enlightenment
(Formerly known as kayandjohn.)
Joined
Feb 2, 2014
Posts
4,125
Likes
2,692
Location
Poulsbo, WA, USA
   
When you use the term transient, you need to define if it is being used to refer to the waveform or the dynamics of the music. Phase shift isn't even in the same universe as musical transients. Square wave performance doesn't matter. Speakers do poorly at reproducing square waves, but they sound excellent with music. I listen to music with headphones, so the meaningful specs are the ones that relate to music. For the purposes of listening to music in the home on headphones, phase shift seems to be pretty much of a non-issue. With room acoustics phase shift can be a big problem, so it isn't as much the transducers as it is the space around them.
 
As for the five points... I generally audition headphones using music I am familiar with and look for ones that come closest to sounding like my speaker system, which I've put a lot of work into getting to sound the way I want it to sound.
 
Accuracy is only important as it relates to our ability to hear and the purposes we use headphones for. It's nice to cross every T and dot every I, and I suppose some people enjoy thinking about that stuff, but I just want accurate reproduction of music.


Good question about "transients."  Most directly, I mean the same "transients" that folks tell me prevent my $30  100 mW - into - 16 ohms FiiO E6 from being a great headphone amp, even though it causes those headphones to achieve 115 dB SPL!
 
Folks say, "You need headroom for transients!"  I don't disagree... my car needs more power than the simple amount needed to climb a hill or maintain speed of 60 mph to be able to get from 0 to 60 mph in 5 sec (sort of a "transient!")  I just don't know of any study of the sound effect of such headroom or quantification of amount needed.
 
The "transients" I am referring to are sudden impacts that result in very rapid increase (or decrease) in sound level... the strike of a drum, the crash of a cymbal, the initial bite of the bow on the string.  These are more amplitude transients, though indeed they are the accumulation of waveform transients across frequencies, aggregated into an instantaneous power level.
 

Users who are viewing this thread

Top