Blinded or controlled listening tests - do they exist?
Jul 17, 2009 at 5:12 PM Thread Starter Post #1 of 13

SmellyGas

100+ Head-Fier
Joined
May 20, 2006
Posts
261
Likes
12
Does anyone know if there are any published level-matched double-blind (or even single-blind) listening tests comparing headphone amps (like there are for power amplifiers)?
 
Jul 18, 2009 at 4:48 PM Post #2 of 13
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
Does anyone know if there are any published level-matched double-blind (or even single-blind) listening tests comparing headphone amps (like there are for power amplifiers)?


...yeah, I haven't seen any either. Oh well.
 
Jul 19, 2009 at 12:13 AM Post #3 of 13
Generally speaking, we do not discuss DBT outside the Sound Science forum, and, in fact, DBT is strictly prohibited in most other forums.

This is due to the contentious discussions that arise from such, and simply, best avoided here.
 
Jul 19, 2009 at 1:18 AM Post #4 of 13
It's much more common to test amplifiers using common test equipment. You can easily demonstrate differences using oscilloscopes and other typical test gear.

The reason you see DBT as such a contentious issue is because you cannot demonstrate differences in cables using ordinary test gear. They all measure the same. The argument is that differences in cables can only be detected with listening tests. However, those differences do not appear with DBT. Which is why there are continuous arguments over whether DBT is effective.
 
Jul 19, 2009 at 12:02 PM Post #5 of 13
Quote:

Originally Posted by Uncle Erik /img/forum/go_quote.gif
The argument is that differences in cables can only be detected with listening tests. However, those differences do not appear with DBT


I am trying to compile a thread with links to BDT.

Could you please provide a couple of links for your above reference on DBT.

Thanks
 
Jul 19, 2009 at 10:41 PM Post #6 of 13
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
Does anyone know if there are any published level-matched double-blind (or even single-blind) listening tests comparing headphone amps (like there are for power amplifiers)?


It is not comparing headphone amps between them, but we recently compared headphone amp vs integrated amp : http://www.head-fi.org/forums/f133/h...ne-abx-429619/

The original report, in French, has more details : Sorties casque : des tonnes de mesures et un ABX

Quote:

Originally Posted by 883dave /img/forum/go_quote.gif
I am trying to compile a thread with links to BDT.


You can find some links here : Post-it: Annuaire des tests ABX
 
Jul 22, 2009 at 1:29 AM Post #7 of 13
Quote:

Originally Posted by Uncle Erik /img/forum/go_quote.gif
It's much more common to test amplifiers using common test equipment. You can easily demonstrate differences using oscilloscopes and other typical test gear.

The reason you see DBT as such a contentious issue is because you cannot demonstrate differences in cables using ordinary test gear.



My understanding is that the electrical properties (inductance/capacitance/resistance) CAN be measured. However, whether the small differences found translate to an audible difference (which is ultimately what is relevant) requires a listening test...preferably one that is not influenced by bias.

Quote:

They all measure the same. The argument is that differences in cables can only be detected with listening tests. However, those differences do not appear with DBT. Which is why there are continuous arguments over whether DBT is effective.


That's basically saying that cables DO make an audible difference and that if any test shows that there is no difference, then there's something wrong with the test. I hope everyone sees why this is ridiculous.
 
Jul 22, 2009 at 3:13 AM Post #8 of 13
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
My understanding is that the electrical properties (inductance/capacitance/resistance) CAN be measured. However, whether the small differences found translate to an audible difference (which is ultimately what is relevant) requires a listening test...preferably one that is not influenced by bias.

That's basically saying that cables DO make an audible difference and that if any test shows that there is no difference, then there's something wrong with the test. I hope everyone sees why this is ridiculous.



Inductance, capacitance and resistance are not the relevant properties for amplifiers. You'll want to look at linear response, distortion, the output impedance curve, among other values. Those are all measurable. Especially with tube amps, differences often fall within measured regions of human audibility and conform with what people hear.

And I think you misunderstand my statement about cables. I'm thoroughly skeptical - having bought and tested cables myself. I can neither hear nor measure differences.

I was stating the other side of the argument. The believers take enormous issue with testing of any type because all testing - aside from sighted listening tests - indicates that there is no difference between cables.
 
Jul 22, 2009 at 3:24 AM Post #9 of 13
Quote:

Originally Posted by Uncle Erik /img/forum/go_quote.gif
Inductance, capacitance and resistance are not the relevant properties for amplifiers. You'll want to look at linear response, distortion, the output impedance curve, among other values.


Inductance, capacitance, and resistance are relevant properties for cables, which is what was responding to.

Quote:

Those are all measurable. Especially with tube amps, differences often fall within measured regions of human audibility and conform with what people hear.


I believe that tube amps can have nonlinear frequency response, depending on the load they're driving, and this nonlinearity occurs in a fortuitous way that sounds more pleasing. I also know that based on work by Geddes, it is the dynamic component of harmonic distortion (and not the traditionally measured summed % or dB) that correlates with pleasing/displeasing sound. This is my explanation for the widespread preference for tubes.

Quote:

And I think you misunderstand my statement about cables. I'm thoroughly skeptical - having bought and tested cables myself. I can neither hear nor measure differences.


Quote:

I was stating the other side of the argument. The believers take enormous issue with testing of any type because all testing - aside from sighted listening tests - indicates that there is no difference between cables.


I agree. I found this subforum after I posted this thread in the headphone amp forum.
 
Jul 22, 2009 at 6:53 PM Post #10 of 13
Quote:

Originally Posted by Uncle Erik /img/forum/go_quote.gif
I was stating the other side of the argument. The believers take enormous issue with testing of any type because all testing - aside from sighted listening tests - indicates that there is no difference between cables.


Could you please back this statement up with links to the documented "Tests" that are either blind or sighted.

Thanks again
 
Jul 29, 2009 at 1:34 AM Post #11 of 13
Quote:

Originally Posted by Pio2001 /img/forum/go_quote.gif
It is not comparing headphone amps between them, but we recently compared headphone amp vs integrated amp : http://www.head-fi.org/forums/f133/h...ne-abx-429619/



That's very cool of you to post your findings! So basically you found that the Marantz developed HUGE frequency response deviations when it was driving the K400. Looking at your plots, a boost from 6kHz to >14kHz with a peak >+2dB is very likely audible because it covers more than an octave. The +1dB boost at 100Hz with a range of 50Hz to 200Hz is also probably audible.

Your listening test basically compared a non-dedicated headphone amp (built-in to an integrated amp) vs. a dedicated headphone amp. This is probably because the non-dedicated headphone amp circuit is not designed well to handle headphone loads. I wonder how different dedicated headphone amps would compare, given that they all should do a reasonably good job driving the loads they are designed for. I bet the big frequency response aberrations would be minimized.

Are you aware of any more tests (in English)?
 
Jul 30, 2009 at 8:49 PM Post #12 of 13
Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
Are you aware of any more tests (in English)?


No. At least no DBT. I didn't look for simple listening test or bare measurments.

Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
you found that the Marantz developed HUGE frequency response deviations


Well, that's huge compared to the frequency response without headphones, but that's just 1 or 2 dB.

I recently started a thread called "ABX for dummies" in a french forum, with pairs of files to ABX, from the easiest (10 to 15 dB in the frequency response for a vinyls vs CD test) to the hardest (The Digidesign challenge : analog mix vs ProTools mix).

I did include the recordings made on the Marantz and on the ProJect outputs while the amps were feeding the K400. I put them in fourth position from the easiest to the hardest, just before the Digidesign challenge.
However, one of the listeners passed the Digidesign challenge but could not hear the difference between the two headphone outputs.

So far, two listeners could ABX it, and two listeners couldn't.

So I'd rather qualify it as "barely audible", rather than "huge".
 
Aug 18, 2009 at 3:54 PM Post #13 of 13
@Pio: cool, thanks.

results from the jazz samples:
Total: 9/9 (0.2%)

equip used: px100's connected to my laptops onboard realtek soundcard
biggrin.gif



Quote:

Originally Posted by SmellyGas /img/forum/go_quote.gif
...yeah, I haven't seen any either. Oh well.


This can have two reasons:
a) people are too lazy / lose patience too easily to do these tests
b) abx only works if you can hear a difference between A and B

Quote:

Originally Posted by pabbi1 /img/forum/go_quote.gif
Generally speaking, we do not discuss DBT outside the Sound Science forum, and, in fact, DBT is strictly prohibited in most other forums.

This is due to the contentious discussions that arise from such, and simply, best avoided here.



I know websites w/ forums that have rules like this:
".. members that put forth a statement concerning subjective sound quality, must -- to the best of their ability -- provide objective support for their claims. Acceptable means of support are double blind listening tests ..."
and I don't think that's a bad idea, at least in the scientific forums.
 

Users who are viewing this thread

Back
Top