Head-Fi.org › Forums › Equipment Forums › Sound Science › Headphone outputs : lots of measurments and one ABX
New Posts  All Forums:Forum Nav:

Headphone outputs : lots of measurments and one ABX

post #1 of 22
Thread Starter 
Last Saturday, Nitri and I gathered in Lyon, France, in order to compare several headphone outputs with hifi and in-ear headphones.

We compared the following outputs :

Marantz PM26 Integrated amplifier
Pro-ject Headphone amplifier
Arcam Diva A85 Integrated amplifier
Sony DTC55ES DAT deck Headphone output
Marian Marc 2 Soundcard

We thus had three types of outputs : integrated amplifiers headphone output, headphone output of a deck, and dedicated headphone amplifier.

We loaded them with the following headphones :

Beyer DT880 pro (250 ohms)
Creative In-ear earphones (same as Sennheiser cx300 (16 ohms))
Sennheiser HD600 (300 Ohms)
AKG K-400 (120 Ohms).

Three hifi headphones of rather high impedance, and one pair of low impedance earphones.

In 5 hours, we just had the time to measure everything with the RMAA software, and to perform an ABX test (with success) between the setups that showed the biggest differences according to the measurments.

We did not have the time to check the maximum listening level allowed by each output, nor the minimum level, nor the background noise.


Measurment results

Bare outputs

The level in dB shows, very approximately, the volume compared to a line output.
Thus, the noise level is to be compared with this volume. The quieter an output plays, the lower is the signal to noise ratio.



For comparison, the headphone.com website gives the harmonic distortion for three of our headphones :



If we assume that we can read, taking into account the fact that we must sum the different harmonic peaks in a logarithmic way, -70 db for the DT880, -73 dB for the HD600, and -85 db for the Creative, this translates into

DT880 : 0.03 %
HD600 : 0.02 %
Creative : 0.005 %

In all cases, this is well below the audible threshold.

The noise level is low on the Pro-ject (given the measurment level), the Marian and the DAT. The power amplifiers are noisier. A bit for the Arcam, more for the Marantz.

We also noted the level for each channel. The Pro-ject is quite unbalanced with 0.7 dB of difference between the left channel and the right channel at the listening level for hifi headphones, and 1.5 dB of difference at the level for earphones (11 db lower). And it doesn't feature a balance setting. Too bad for a dedicated headphone amplifier.

The Arcam and the Marc 2 soundcard are perfectly balanced (less than 0.2 db of difference at low and high level).
The Marantz and the DAT output have 1.2 db of difference between the left and the right channel at the listening level for hifi headphones (if we assume that the soundcard, that measured the DAT, is itself balanced) ! Oddly enough, the difference is only 0.6 db for the DAT and 0.2 dB for the Marantz at low level, for earphones.

Now, here is the behaviour of our outputs, loaded with the Creative earphones, at a listening level that seemed average to us, except on the Marantz, very loud (we have not had the time to carefully adjust the levels).



Very neutral in all cases. The lack of bass of the Marantz have stayed the same, but the Pro-ject falls at -1 dB at 30 Hz loaded with the earphones. This is very probably inaudible.

There have been a bug with the Marantz measurments. For an unknown reason, RMAA pretends that the recording are clipped, while they are not. And the crosstalk measurment is in contradiction with the test signal recorded (RMAA says -6 dB). I have masked the number in the table.

Let's go on with the Sennheiser HD600. The level was 5 or 6 dB lower on the Pro-ject (it was on its maximum !) than on the other outputs (except maybe the DAT, for which we have no level reference).



The deviation from neutrality is much bigger. But the headphone impedance is much higher ! Paradoxical ? No, here are the impedance curves given by Headphone.com :



The earphones impedance is indeed very low, but perfectly flat. Consequently, the neutrality is not affected at all. They exhaust the amplifier in exactly the same way at all frequencies. As a result, it is as we had lowered the volume, but with no quality loss. The Sennheiser is less demanding, but shows variations according to the frequency. Therefore it tends to turn the amplifier "less neutral".
THD or IMD are not comparable the same way, because they are affected by the volume, and it was very different between the two measurments.

IMD (InterModulation Distortion) becomes quite high on the Marantz, while it was set at a lower level than during the bare measurments. I could not say if this is audible. but I think that below 1%, if it is audible, it is probably not annoying.

Something interesting is that there is no separation by category for the frequency curves. The two worst are a deck output and an integrated amplifier (Marantz). The two best are the soundcard and the dedicated amplifier. The other integrated amplifier is somewhere in between.

Here are the results with the AKG K-400 :



We did not have the time to test it on the DAT.

We get the same kind of results as with the HD600, but in a stronger way. Its nominal impedance is indeed lower : 120 Ohms vs 300. It is therefore more difficult to drive.

Last, the DT880 :



Surprise : its nominal impedance, 250 Ohms, is between the two previous ones, but its effect on the amplifier is smaller. This is explained by a flatter impedance curve.


ABX Test

After having done all this, we had little time left for the listening. We chose the easiest : Pro-ject vs Marantz with the AKG K-400.
According to the measurments, we should expect on the Marantz -1 db at 30 Hz, +1 db at 100 Hz, -1 around 1000 Hz, and +2 dB at 15000 Hz.

Nitri chooses the easiest test : a pink noise.

I'll tell later about the test conditions, level matching, and especially the incredible mess needed to perform the test in double blind conditions without hardware switch !

The difference sounds important, but the delay between the listenings, more than 5 seconds, is very annoying in order to tell which is which blinded. We manage to reduce the delay to 3 seconds. Nitri is not sure about his ability. He tries one trial and fails.
I try too, a bit more confident. I run one trial and fail too.

Nitri wonders if a musical signal could help more. Normally, a pink noise is much more discriminating.
I find a sample where the difference seems obvious. i get 4/4 in a trial ABX, with 4 or 5 seconds betwen the listenings. I start an ABX in 8 trials, and get 8/8.

Nitri must leave, but he listens quickly. He says that he's really not sure to succeed with this sample either.

It is very interesting to note that I failed on a pink noise, but succeeded on a musical sample. Moreover, it was not an acoustic recording, but a plain, flat techno boom-boom (Front 242 - Neurobashing).

This can be explained by the fact that this sample features mainly frequencies affected by the frequency response of the Marantz. Here is its spectrum :



It features little medium frequencies, where both frequency responses are flat, and much bass and treble, where the Pro-ject is flat and not the Marantz.
A pink noise would have a lot more of unuseful medium frequencies, not relevant for this comparison.

Another explanation would be that the two pink noises, in the AKG headphones, sounded both abstract to my ears, while with the musical sample, I had a kind of concrete milestone in a musical language.
It seemed to me that I could remember the particular sound of the "tshookootshookootshook", heavier on the Marantz, and keep it in memory from a playback to the next, while I could not remember at all the sound of the pink noise.

Last, I realized in the evening that this ABX test was not valid at all ! I had recorded the signals delivered by the two outputs and made wav files of them in order to put them online, so that people can listen by themselves what was the biggest sound difference that we encountered.
Offsetting and normalizing the recordings, I realized that we had swapped the left and right channels between the two amplifiers !

I didn't even notice during the listening test.

I thus, with the two recordings, got the channels back in place, time-aligned the samples, and equalized the levels (in 16 bits). Then I ran the ABX again, on the PC, this time, since Nitri was not there anymore.
I got 8/8 again. Which proves that the difference between the frequency curves is indeed audible. Given this result, I think that I can say that during the real ABX, it was indeed the difference of frequency response, and not the channel swap, that allowed me to succeed.

Here are the samples, in flac :

Original
Recording of the Marantz loaded with the K-400
Recording of the Pro-ject, loaded with the K-400


Conclusion

In conclusion, we can say...
That the quality of a headphone output is, if not important, at least sometimes audible.
That the nominal impedance of the headphones can't tell us if the amplifier will deliver a neutral sound quality.
That integrated amplifiers outputs are not all equal.
That the dedicated Pro-ject headphone amplifier can perfectly feed difficult loads, but that it is a bit unbalanced between the left and right channels, and quite short in output level.
That the soundcard seemed to be able to output higher levels with similar quality.
That the Beyer DT880 is easier to feed with neutrality than the Sennheiser HD600, itself easier than the AKG K-400.
That integrated amplifiers seem to have more background noise than dedicated headphone outputs (save basic headphone outputs on computers, and maybe nomad player outputs).

...and that it remains, in order to get a general idea, to test the possible listening level, maximum as well as minimum, and the background noise according to the amplifier and headphones. It would also be interesting to test a nomad player output. Nitri had brought a Creative player, which can play wav files, but we lacked the mini-jack adapter in order to perform the measurments.

Since Nitri is not in Lyon, it won't happen in the short future.
post #2 of 22
Thread Starter 
Reserved space for protocol and other details.
post #3 of 22
Wow, thanks for this. It's neat seeing measurements show up clearly for once. It would be very interesting indeed to see how various dedicated headphone amps compare to each other with this kind of testing.
post #4 of 22
Did you measure the output impedance of each amplifier through the headphone socket?

Looking at the frequency response curves, I would guess that the Project amplifier has very low output impedance, while the Arcam is higher and the Marantz the highest output impedance.

Also remember, what is matters is not what the frequency response at the amplifier is, but what the frequency response of the amplifier + headphone combination is.
post #5 of 22
Pio2001 - I'm thoroughly confused by the creative FR measurements. My own experiments with this sort of thing has lead me to conclude that high impedance headphone outs (as shown with the speaker amps) causes a measurable alteration of frequency response. If we take a high output impedance (say 50 ohms), the distortion of the FR graph will be greater the lower the impedance of the headphones.

For the AKG K-400 and HD600 your results do not contradict this. But the results for the creative are totally at odds with my own findings. Even your DT880 results seem to be different from what I would have predicted.

This is very interesting for me! Thanks very much for doing it!

**edit**


Could you talk about how you matched volumes? Considering the deviations in the FR graphs I'm curious as to how you did it.
post #6 of 22
Quote:
Originally Posted by odigg View Post
My own experiments with this sort of thing has lead me to conclude that high impedance headphone outs (as shown with the speaker amps) causes a measurable alteration of frequency response.
Yes, and sometimes this is intentional. Instead of driving a headphone from a voltage source (very low output impedance), some are meant to be current driven (higher output impedance). Depends on the design of the headphone.

Quote:
Originally Posted by odigg View Post
If we take a high output impedance (say 50 ohms), the distortion of the FR graph will be greater the lower the impedance of the headphones.
This depends on the shape of the impedance graph of the headphone.
post #7 of 22
Quote:
Originally Posted by Donald North View Post
This depends on the shape of the impedance graph of the headphone.
Could you talk about this a bit? The creative phones have a flat 16 ohm impedance throughout the FR range. I would have predicted the FR graphs would be totally distorted but the measurements shown above demonstrate my prediction to be false.
post #8 of 22
The amplifier's output impedance forms a voltage divider with the impedance of the headphone. If a headphone has a flat impedance curve (same impedance at all frequencies), then the voltage delivered to it will be constant versus frequency, regardless of the amplifier's output impedance, thereby yielding a flat frequency response across it. Of course for a given generator voltage, the greater the output impedance, the less the voltage is across the headphone.
post #9 of 22
Thread Starter 
Quote:
Originally Posted by Donald North View Post
Did you measure the output impedance of each amplifier through the headphone socket?
We did not measure the impedance. Juste the frequency response. And yes, through the headphone output. We used a divider in order to plug the headphones and the measurment device at the same time in the same plug.

I just began to write about the setup in the original thread (french, but there is a picture) : Sorties casque : des tonnes de mesures et un ABX

Quote:
Originally Posted by Donald North View Post
Looking at the frequency response curves, I would guess that the Project amplifier has very low output impedance, while the Arcam is higher and the Marantz the highest output impedance.
Yes. For a given pair of headphones, the flatter the curve, the lower the output impedance (given that the curve is flat when no headphones are plugged).

Quote:
Originally Posted by odigg View Post
Could you talk about how you matched volumes? Considering the deviations in the FR graphs I'm curious as to how you did it.
The levels of the first ABX, spoiled by the channel inversion between A and B, were equalized reading the peak level on SoundForge peak meter (0.1 dB accuracy), while the amplifiers were fed with the RMAA calibration signal, which is a burst of short 1 kHz sines, and acting on the volume and balance controls of the Marantz.
Looking at the frequency responses, this turned the test easier than it should, because all frequencies were then louder on the Marantz.

The second test, between the wav files, on PC, was equalized with 0.01 dB accuracy, measuring the RMS level of the complete musical sample, after time-matching, which is a better method.

Quote:
Originally Posted by odigg View Post
Could you talk about this a bit? The creative phones have a flat 16 ohm impedance throughout the FR range. I would have predicted the FR graphs would be totally distorted but the measurements shown above demonstrate my prediction to be false.
This is because the frequency responses are level corrected. The bare measurments would have shown them one on top of the other, with the Creative one very very deep below the original level.
Actually, it would have looked a bit like the impedance graph :



Imagine that the original signal is represented by a flat line on top of the graph, on the 600 ohms mark.

The blue line of the cx300 is actually the one that is farthest from the original ! So you were right, all frequencies are terribly affected because of the low impedance.
But since the earphone impedance is itself flat, the result is also a flat curve, which can be restored at the original level just turning up the volume !
post #10 of 22
First of all: thanks for your efforts Pio!

I downloaded the files to ABX them, and after a few minutes I was able to tell the difference between all of them.
The results were 9/9 for all tests (original <-> marantz, marantz <-> pro-ject, original <-> pro-ject) which is a bit surprising! Looking at the K400 FR chart, we can see that the pro-ject's biggest difference to the flat original is about +0.25db between 10 kHz and 20 kHz.
But +- 0.25db should not be distinguishable for the human ear, right?

So I don't really know whats going on here. Can anyone confirm this? (you can do ABX tests easily in foobar, just select two files, right-click -> utils -> abx)
post #11 of 22
0.25 dB level difference is expected to be audibly detectable in ABX testing - the usual recommendation is to match better than 0.1 dB

ABX Amplitude vs. Frequency Matching Criteria
post #12 of 22
Hmm the graph shows that at ~15 kHz only the 3-octave line is near 0.25 db.
10 kHz to 20 kHz would be only one octave?
post #13 of 22
how about the >0.5 dB Project-Marantz difference from 400-4K?, > 3 octaves

absolute SPL and level matching can be argued over too - A weighting of the pink noise might be better for level matching with "conversational" ~80 dB SPL

another confounding factor in many ABX tests is extraneous clues like start/stop cliks, delays - underlying noise level when test signal is silent

I haven't tried it but Audio DiffMaker looks like a tool that could help locate spurious audio clues in test files
post #14 of 22
Quote:
Originally Posted by jcx
how about the >0.5 dB Project-Marantz difference from 400-4K?, > 3 octaves
I was talking about project<->unaltered original.

Quote:
another confounding factor in many ABX tests is extraneous clues like start/stop cliks, delays - underlying noise level when test signal is silent
I used start/stop markers at several positions. Also I can hear the _difference_, it's not a feeling or something.
Besides, it wouldn't make much sense to do this personal test if I manipulated it from the beginning.

Quote:
I haven't tried it but Audio DiffMaker looks like a tool that could help locate spurious audio clues in test files
Thank you very much! Will try this out.
post #15 of 22
The original has 44.1 kHz while the others have 48 kHz sample rate. :/
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › Headphone outputs : lots of measurments and one ABX