[dgardner]

Last night I built up an A/B Switch box with two headphone jack inputs and one headphone jack output. Nothing new here. Then I started to go through my inventory of headphone amps and try to confirm/deny the "sonic differences" between them all.

A created a stereo test tone waveform a few minutes long in Adobe Audition and burned it to a CD. This way, I can play a calibrated test tone through the source unit and get the two amplifiers under test to exactly the same listening level using an AC Voltage Meter (HP34401A). After the calibration is set, I don't move the volume controls on either test amp. Now, I start listening to those favorite music selections that I tell myself are "the ones" I know so well that I can hear differences between equipment, etc.

Source: Benchmark DAC-1
Phones: HD-650's

Amps compared:
a) Benchmark DAC-1 headphone jack
b) Dynalo with ELPAC wallwart supply
c) Dynalo with Dynahi supply
d) Dynahi with Condor supply
e) Dynahi with Dynahi supply

I constantly hear people talking about the OMG differences between these items and I must say the differences start washing away under more scientific testing. I will spare everyone any attempts to write colorful prose on the differences, however subtle.

Does anyone else out there feel that A/B Switch box testing really narrows the gaps that the brain tends to create for itself?

I would expect to be able to readily detect differences like:
a) background noise
b) frequency response
c) tonal colorations
d) distortion
e) dynamics

Funny, how difficult is is to separate things when listening to two amps at precisely the same listening level.

What are some tips to start detecting the nuances? What are some of the best track selections to reveal these gaps?

 

[wistily]

And perhaps that if you build yourself an ABX box (or something that don't show what you are hearing), you won't here any difference anymore.

 

[aerius]

I once A/B tested headphones. After a fair number of switches I couldn't tell apart my Senn 580 and Grado 225 headphones by sound alone. I learned that the more times I switch, the less I'll be able to tell the differences. With the headphones mentioned above, the first couple switches were blindingly obvious, after 6-7 I start getting confused, after that I was like "screw this monkeycrap". It can be useful but you gotta be careful doing it.

 

[amb]

Quote:

Originally Posted by dgardner I constantly hear people talking about the OMG differences between these items and I must say the differences start washing away under more scientific testing.

Ain't that the truth!

Such A/B testing on amps really puts a perspective on things. There are differences to be heard, but very, very subtle (if the amps being compared are all of good quality).

 

[atx]

Quote:

Ain't that the truth! Such A/B testing on amps really puts a perspective on things. There are differences to be heard, but very, very subtle (if the amps being compared are all of good quality).

How about A/B tests on interconnects.

I certainly can't tell the difference between a Zu cable and a generic brand. But for those using a Zu Mobius, I presume they can... ( or so they believe

)

 

[shimage]

Quote:

Originally Posted by aerius I once A/B tested headphones. After a fair number of switches I couldn't tell apart my Senn 580 and Grado 225 headphones by sound alone. I learned that the more times I switch, the less I'll be able to tell the differences. With the headphones mentioned above, the first couple switches were blindingly obvious, after 6-7 I start getting confused, after that I was like "screw this monkeycrap". It can be useful but you gotta be careful doing it.

It's called fatigue, and you are right about that. On the one hand, you want to be careful about that sort of thing, so perhaps only do a couple trials per day for a week or two (to get better statistics). On the other, if you have to concentrate that hard to hear the differences (if you didn't have to, then you wouldn't get fatigued so quickly), then maybe the difference isn't worth bothering about. Oh wait, is this blind, or not? It's the same effect, and all, but if it isn't blind, then there aren't any statistics to improve.

Quote:

Originally Posted by atx How about A/B tests on interconnects.

I did that before, but only between crappy cables (RadioShack and ... RadioShack). Oddly enough, it was pretty easy to tell them apart (I guess different kinds of crappy are pretty easy to tell apart?). Now I need to get some decent cables to test out ... hmm ... Larry's just across the lake ...

 

[john_jcb]

I have compared many amps using a switch box and the differences are far less than most would imagine. One of the problems we face I believe has nothing to do with the audio qualities of the gear at all. The problem is trying to describe these small differences in writing. A most difficult task made even harder when you factor in preconceived prejudices.

When I evaluate two amps in this way I will use music that I know very well but only very short passages where I will intently listen for a difference in a voice or instrument. I will also pick passages where background noise can be assessed. This is not an easy process and it requires detailed note taking over an extended period. Longer listening I believe clouds your mind and your impressions have more to do with what you want to hear rather than the actual differences.

The night and day differences that many reviewers report are just are not their and I weigh their review accordingly. I have, however, heard larger differences where the amp manufacturer has taken measures to noticeably alter the sound. I have heard amps that I felt were overtly tubey for instance. To my ears the music just does not sound like a faithful reproduction of the original music. This is not to say that it cannot be pleasing to the ear, it is just not what the original performance sounded like. I also believe that it is important to know what live music sounds like so that you know for instance what a string bass, a piano or Diana Krall's voice should sound like. It is for this reason that I will never use synthesized or recordings that I know have been altered for evaluations. I tend to use simple ones with instruments and voices that I can concentrate on.

 

[PhilS]

Quote:

Originally Posted by john_jcb When I evaluate two amps in this way I will use music that I know very well but only very short passages where I will intently listen for a difference in a voice or instrument. I will also pick passages where background noise can be assessed. This is not an easy process and it requires detailed note taking over an extended period. Longer listening I believe clouds your mind and your impressions have more to do with what you want to hear rather than the actual differences.

That's interesting, because I've found it easier to hear differences with longer term listening. I find short term switching back and forth to be very difficult -- whether it's due to fatique or just the fact that trying to hear differences makes it hard to hear them, I don't know. But if I have listened to something over the long term many times, it's as if the precise sounds are memorized by my brain (assuming no changes have been made in the system during the long term listening). Then if something changes, it's easier to recognize, at least to me. That's why I find A/B testing of a short duration to be not very probative, although it's really the only practical way to A/B anything.

 

[dip16amp]

Looks like you're comparing amps of increasing power capability but limiting the volume levels. You need to crank them up to see how they perform at higher levels and which ones can go louder and cleaner. Then it will be easier to notice the difference even if you don't normally listen to them at loud levels. Of course you shouldn't turn them up for long periods of time.

 

[gpalmer]

Quote:

Originally Posted by john_jcb I have compared many amps using a switch box and the differences are far less than most would imagine. One of the problems we face I believe has nothing to do with the audio qualities of the gear at all. The problem is trying to describe these small differences in writing. A most difficult task made even harder when you factor in preconceived prejudices.

This of course assumes the amplifers are roughly the same quality level, if you A/B a Boosteroo against an HR-2, I'm guessing you would be able to tell the differences pretty easily.

If I remember correctly, I believe that we've discussed this before and we agree on this point. Then again, I think at least implied in the basic premise is the assumption that people measure quality of sound reproduction in a linear fashion, but drawing an analogy to how people experience pain, even a small amount of pain can become perceived to be huge if it persists long enough. I find this is my character when listening to amps also. I have a certain threshold of quality in sound reproduction that makes me happy and if the sound is below that it will aggravate me until it becomes perceived to be much larger than it really is.

I won't even mention the fact that I believe most would agree to, that the amplifer makes the smallest amount of difference when weighed against the headphones and source.

I personally prefer longer term listening myself, though your methodology strikes me as quite sound and pretty much what I try doing. I try to pick logical sections of the music I am using, for instance a guitar solo or some other section of music that illustrates a point of interest and then do a comparison. Sometimes it will be subtle, such as a soft strumming of strings in the background other times it will be very overt, such as the aforementioned guitar solo. I agree it's a long process, I need to listen to 50 or more CD's of varying types of music and qualities of recording before I can really get a feel for an amplifier. Many times I can listen to a number of CDs without having a real epiphony on the sound signatures and then I put in the right one and Bingo!, the differences are there. I guess this might also support the premise that amps do sound very much alike.

Things I will tend to listen for (one at a time, you need to concentrate):
1) Depth of apparent sound.
2) Small details in the background (for instance, does a guitarist sliding a cord waver with one of his fingers). How clearly can you hear the sound seperated from the rest of the music?
3) Track each instrument in the music, is the sound reproduction of the two amplifers equally clear and well defined?
4) Frequency extremes, both low and high, do the amps reproduce them equally well? Cymbols to Synths... Are fast low bass notes well defined? There are some Beethoven piano Sonatas I like for testing this, again this is a frequent difference between amplifers.
5) Human voices, both male and female - In the current solid state market, it's only the low end that can't get these right, but humans are very attuned to voices. In tube land, this is one of those vital indicators as to whether or not you are right in your selection of tubes.
6) Reverberation of big drums, as much as audiophiles usually sneer at the Blue Man Group, their DVD-Audio has some big drums being hit and the differences in the reverberation and decay can tell you a lot about an amplifer.
7) Lastly, I like pulling back and not paying attention to the details but rather just savoring the overall ambience.

 

[Beauregard]

Quote:

Originally Posted by dip16amp Looks like you're comparing amps of increasing power capability but limiting the volume levels. You need to crank them up to see how they perform at higher levels and which ones can go louder and cleaner. Then it will be easier to notice the difference even if you don't normally listen to them at loud levels. Of course you shouldn't turn them up for long periods of time.

That's the ticket!

dgarnder - for me, the dynamics that result from adequate power is the make-or-break for an amp - with speakers or phones. My test pieces are selections of full orchestra/chorus crescendos played at close to ear-splitting volume - which is what occurs at a live concert if you're seated most anywhere but the back of the balcony at most venues.

With a good recording, it shouldn't be too hard to hear which amps poop out and which reproduce such a passage without sounding weak, congested, distorted. The true acid test is when a bass drum, recorded at a high level, is struck repeatedly during the full volume crescendo. A really good amp will have you sitting up in your seat, delivering the impact with authority over the noise of the orchestra and chorus; I expect differences in amps will be readily apparent in the test paradigm you're using.

If you're not a classical fan, I suggest you grab a well-recorded Carmina Burana for your experiments. The Shaw/Atlanta Symphony version from Telarc is an oldie but goodie - sonically, still one of my favorite recordings of all time. The 'O, Fortuna' should meet your needs nicely.

Keep us posted on your progress. I think controlled listening tests are sorely needed in this business; IMO, your approach has infinitely more validity than the fan-boy "reviews" that pass for critical evaluation around here.

Quote:

Originally Posted by gpalmer I won't even mention the fact that I believe most would agree to, that the amplifer makes the smallest amount of difference when weighed against the headphones and source.

Not me. Especially with high impedance phones, the phone/amp combo is significantly more important to my enjoyment of music. If I was forced to sacrifice components in my recently upgraded headphone setup, I'd let the DAC1 and beloved new SACD player go before my HD650s/MOH(R) - and be pretty damn satisfied with my 13-year old portable CDP as a source.

Best,
Beau

 

[Ferbose]

Dgardner, your meticulous efforts at comparing amps is quite admirable.
There is so much nonsense in modern audio nowadays.
If only audio reviewers can start doing "some" objective listening....
Hey, cable manufacturers don't even provide reviewers with electrical specs or a sample of cut-open cable to show the construction.
Reviwers don't have any means of A/B testing, not to mention ABX.
Is cable reviewing a beuty contest or what?
The pretty and the expensive gets the nod--really makes sense in a totally capitalistic society.
Some people will say A/B testing causes listening fatigue and prevents discrimination.
There may be some truth about this theory and does it also mean after 30 minutes of listening to music we might as well switch a inferior amp?
So audiophile gear only provide a 30 min window of enjoyment? (Just kidding)
Benchmark DAC1's headphone out is already considered one of the better jacks in pro-audio gears.
I looked into the super-extensive Sweetwater catalog and realized that the only better headphone output may be Grace.
Many people have reported that Benchmark's XLR driving headphones sounds even better than its headphone jack.
Here is my personal experience on that: http://www6.head-fi.org/forums/showthread.php?t=104041

 

[gpalmer]

Quote:

Originally Posted by Beauregard dgarnder - for me, the dynamics that result from adequate power is the make-or-break for an amp - with speakers or phones.

Actually another equally valid comparison (and one that is more useful to me and I suspect the majority of users) is the low volume performance. That's where I do the bulk of my listening and it's one of the areas that really seperates the men from the boys.

Quote:

Originally Posted by Beauregard Not me. Especially with high impedance phones, the phone/amp combo is significantly more important to my enjoyment of music. If I was forced to sacrifice components in my recently upgraded headphone setup, I'd let the DAC1 and beloved new SACD player go before my HD650s/MOH(R) - and be pretty damn satisfied with my 13-year old portable CDP as a source.

I'm not sure we're addressing the same issue. I was stating that the variance in sound from changing headphones and sources are typically much greater than those of changing an amplifer. Not necessarily that changing one would make the system unacceptable or bad just different. I also didn't imply the amplifer was being removed from the system just substituted. Your response seems to state that you have two equally acceptable sources and you would rather use the inferior source than lose the amplifer, which is not really what I was saying.

 

[john_jcb]

Quote:

Originally Posted by gpalmer Actually another equally valid comparison (and one that is more useful to me and I suspect the majority of users) is the low volume performance. That's where I do the bulk of my listening and it's one of the areas that really seperates the men from the boys...............

This is really a challenge for any amp be it headphone or speaker. I also listen at lower volumes most of the time and where you can really tell if you have a good amp or not. There have been many articles on how louder music sounds better. Some of this I believe is do to the amplifiers performance at lower volumes.

 

[dip16amp]

This leads back to the original problem of this thread in that comparing the five amplifiers of increasing power capability, they all sound very similar. Listening at lower volume levels does not really help in determining the difference between these amps. This also shows that increasing the power capability is less noticeable when listening at lower volume levels.

Turning the volume up when the music is quiet or silent lets you determine the lower noise floor amplifier. Turning the volume up on heavy bass passages lets you determine which amplifier has better power, speed, and control under load. I also find it easier to determine the differences when using low impedance Grado headphones than Senns.

Six months ago when I completed and compared my balanced Dynahi (Dynamight) to my balanced Dynalo (Dynamid) they sounded similar at low volume levels but the biggest difference is at higher volume levels. My Dynamight gets used almost everyday at both lower volume levels as well as higher levels. The Dynamid just can't keep up with the Dynamight at all levels.