[bigshot]

thanks for that bit of information that has literally nothing to do with anything.

Actually he's attempting to use Appeal To Authority.

I'm getting closer to my Bingo!

The general advice for the edge-case amp problems is sometimes just to turn the volume down a little.

Excellent advice in general. Let's see if he takes it.

 

[KeithEmo]

I was prepared to argue with you when I realized that, this time, you actually did get my point.

Claiming that "it would be too difficult to test kindergarten students" is pretty absurd.
(And the best way to point that out seemed to be by comparing it to an even more absurd assertion.)

However, in fact, I may have failed to provide a sufficiently absurd counter-sample...
I recently read a summary of the results of a test to determine "whether cats recognize their own names"...
It was in a recent edition of Scientific American...
They actually seemed to consider furthering our knowledge of how cats think as useful science.
(Science can actually be fun after all.)

Let's play the Head-Fi Sound Science Home Game! Just quote one line and cite the logical fallacy that goes with it. Fill your card and call out BINGO! when you've had enough. I'll start...



Reductio ad absurdum

 

[bigshot]

If what you think you hear seems to disagree with the measurements...
You should consider taking a few more measurements...
Or even, perhaps, examining the situation a little more closely, and attempting to figure out what you might have missed...
Instead of just discounting what you heard as "something that couldn't possibly be true"...

Stereo Review in the introduction to the article "Do All Amps Sound The Same?" says there is nothing in the measurements of amps to make one think that they would sound different. And at the end of the test they concluded that according to the results of their listening tests, if there are differences between amps, we aren't likely to be able to hear them. So they basically say that the measurements and the listening tests agree. Why are you talking about disparities that don't exist here?

 

[KeithEmo]

I wasn't specifically referring to that article...

However, I do give the authors of that articular article a lot of credit for NOT making overly grandiose claims about their results.
Note how they used phrases like "that would make one think"... and "aren't likely to hear"... both of which convey elements of personal opinion...
Which make it clear that they don't expect their results to be taken as generalizations that apply to every amplifier, every listener, or every situation...
They did a small scale comparison, under limited circumstances, between a few popular audiophile products, and reached some interesting conclusions...
(Which many people might well find interesting, informative, and useful...)

Stereo Review in the introduction to the article "Do All Amps Sound The Same?" says there is nothing in the measurements of amps to make one think that they would sound different. And at the end of the test they concluded that according to the results of their listening tests, if there are differences between amps, we aren't likely to be able to hear them. So they basically say that the measurements and the listening tests agree. Why are you talking about disparities that don't exist here?

 

[analogsurviver]

AS and KE. . . Why don’t you go over and join the Audio Science Review forum and try your ideas over there. They are not so regimented in separating the objectivist and subjectivist camps because they are not so wedded to commercial interests.https://www.audiosciencereview.com/forum/index.php

There are some head-fiers who bounce back and forth. My guess is that they would be slightly more sympathetic to your philosophical bent. For example, they are sympathetic to the idea that amps will sound different when you push them to the edge of their operating limits and will investigate exactly what is going on, and they do test audiophile equipment and enjoy impressive performance beyond audibility for its own sake. The general advice for the edge-case amp problems is sometimes just to turn the volume down a little. They have some moderate “believers” and moderate “non-believers” who have common ground in the middle and the forum has a very cordial posting style. They do lots of testing and measuring. Some of them have experience designing and building big-league stuff. Why not try your hand at it?

Audio Science Review is, of all present options, perhaps the most interesting option. About the best info on objective performance of any currently available gear.

However, despite what you correctly observed, they do not do acid tests - pushing equipment to and beyond the limits. When operated within design envelope, most equipmewnt today will be reasonably good. When pushed beyond ( a single peak into clipping, for example ) can wreck havoc with some amps - and sustained clipping can still sound acceptable on another. Usually, both subjectivists and objectivists find their jaw on the floor - IF presented with the listening to their music at their normal listening level - and observing the input and output of the power amp. Only very high efficiency speaker systems manage to avoid clipping ... - and when the level is reduced for zero clipping on high dynamic range recordings using normal efficiency speaker system (below 90 dB /w/m), the resulting sound is usually simply too low in volume. This is to cite only the most obvious problem with real world use of amplifiers.

With DACs and ADCs, the focus at ASR is on 20Hz-20 kHz performance almost exclusively. I would much prefer distortion 0.01% well into the ultrasonics ( in fact, to and beyond the frequency the device is capable of ) than zero % within 20-20 k, but skyrocketing at any frequency the device still produces output. Real world digital devices have quantization noise from certain frequency up, depending on the sample rate the device is capable of. Above 20 kHz - and, by that I mean at least 40 kHz - there are generally no measurements. Some digital devices can have noise and distortion going trough the roof above 50 kHz - and some, although not perfect, can remain relatively unaffected. The second kind will usually be preffered in listening - despite being worse in the 20-20K band. Objective faction would insist it is the other way around ...; true, but only if and when fed strictly with a RBCD signal. Any analog or hirez digital would prove them wrong. Ever attempted testing digital devices for intermodulation very close to their upper limits of frequency response - like 45 and 46 kHz at1:1 , with amplitude close to 0 dBFS ( say at not more than -3 dBFS ), with the sampling frequency of 96 kHz ? Usually, the difference sum - 1 kHz, most definitely audible - will be rather higher in level than any single tone THD measurement. It is true that high levels at so high frequencies are extremely rare, but not impossible. And the device that shows less deviation from perfect, even within so stressful conditions, will usually perform better in the long run of actual listening to music. No testing with upper limit of 20 kHz would reveal such differences among digital devices - but listening to any source exceeding RBCD would generate impressions; regardless how true or false interpreted, they would at lest suggest the difference DOES EXIST.

I have learned troughout my time with audio those hard to do feats , which in theory should never occur, do in fact produce audible differences in perceived sound. Just to cite the most widespread problem in amplifiers prior and up to say 1980 - power supply. Almost everything had great spec for noise levels, but with one, unfortunately fatal flaw ; that power supply has only been quiet and ripple free with no or low level signal(s). Push it above say 25% of the rated power, the power supply will be modulated by the audio signal it is trying to reproduced. Push it close to , to or above the clipping - it might momentarily reduce channel separaton to close to zero.... and so on and so forth. I have adopted the policy the power supply(es) have to remain stable no matter what, even into gross clipping - and amp better recover from this gross overload instantly once the too high input amplitude is removed. This type of problems in amps would NEVER be revealed in conventional testing. If it measures perfectly within specs, that does not mean it will never see "out of spec" signals in real world of listening to music.

Even if we go to the most basic whole chain consisting of microphone, preamp, power amp and headphone/speaker , reasonable performance both below and above 20-20k range is required. Should anything other than RBCD be used as a source, the amplification will be facing various artefacts, from phono stylus mistracking ( can be almost in MHz range ...) to quantization noise of digital. 20-20k conventional tests may well fail to weed the chaff from the grain.

And, like it or not, better and more or even unconditionally stable and noise free power supplies DO contribute to better perceived sound quality. That may well mean regulated supplies for input, for driver and maybe even for output stage - and you can double that for dual mono or mono block operation designs.

If anyone of you thinks I am exaggerating with clipping in home audio systems - attach a dual trace oscilloscope to input and output of the power amp, play some reasonably low sine signal ( usually 1 kHz at - 20dB ), adjust the sensitivity of the scope so that input and output have the same amplitude in the display. Invert the polarity of one of the channewls and add the two signals - ideally, the trace on the scope should not show any movement at all, not even noise. Then, at the same settings, start playing music. Make sure your settings do not overload the inputs of the oscilloscope; that would give you false readings. I DO NOT RECOMMEND HIGH LEVEL TESTING CLOSE TO MAX OUTPUT OF THE AMP WITH ANY (PARTICULARLY DYNAMIC ) SPEAKER - if you can, verify that your power amp clips before inputs of the oscilloscope for your settings with dummy resistor load, which is hard to damage and can sustain steady state full output for the few seconds required to do the verification. Needless to say, playing steady state signal ( 1kHz ) trough loudspeakers close to full amp power is extremely LOUD; and could lead to permanent hearing damage. Speakers can be replaced, ears can not. Use common sense.

After you have ascertained above, just play music, at your normal listening level setting. In case of no clipping, output on the scope display should be ideally perfect horizontal line. With today's amps, no noise should be observable on that line. If you are lucky nothing remains to be seen even during the loudest climaxes of the (most) demanding recordings you chose fdor the test - congrats, you have a really well chosen system according to your requirements.

But most WILL get to see peaking to occur some percentage of the time... - and are likely to find the volume insufficient for the desired loudness if the input is reduced below peaking 100 % of the time.

That is when the "beauty" of electroacoustics sets in ... output power of amplifiers is MUCH more meaningful if instead of simple Watts is expressed according how humans perceive loudness : https://www.rapidtables.com/electric/dBW.html A 100 W amp has 20dBW, a 200 amp only 23dBW - just 3 dB louder. For another 3 dB increase in loudness, 26dBW are required - or 400 W. Converted into money, those increases in performance ( provided the speakers used can take the power cleanly without damage to begin with ) does cost dearly indeed.

And those who have some experience will tell you that designing a decent sounding say a 70 watt amp is easier than a say 200W amp - which, in turn, is easier than designing say equally good sounding 400 or more W amp. You can add this claim as yet another "audiophile" myth - but this will be confirmed by anyone who has ever attempted or built any real world amp. In cases where the same - or at least similar - circuit topology has been used for approx above mentioned power levels, the best sounding amp that came on top during listening at low levels and/or with loudspeakers with above than average efficiency has been usually the least powerful one. Which succumbed to power shortage when used for high level music and/or loiw(er) speaker efficiency. An aptly named range has been from Great American Sound: the original big one Ampzilla, its smaller sibling Son of Ampzilla and the final addition to the family, called Grandson . With careful matching to the room and speakers used, it has been thus catered to optimally choose the amp regarding both cost and sound quality - with many Grandsons replacing its more powerful predecessors in case the power of the smallest amp has been found sufficient. Similar stories can be observed with many more brands, but none had more aptly named amplifiers.

 

[bigshot]

I think the Stereo Review article is a MUCH better guide for the best way to select an amp than audiophile sales pitch and anecdotal impressions. However, on the planet Mars if the sun isi at just the right angle and the entire universe is in alignment, perhaps it would be better to go with the BS advice from stereo salesmen. Thankfully, I have no plans to go to Mars, so I can just take it as an absolute and act accordingly.

I keep my amps below clipping so none of that flurry of made up pseudo-scientific blather applies to me either.

 

[KeithEmo]

Wow......
I have to admit that I didn't read the entire test until now..... and, after reading a bit more, I did notice some interesting things.

For example.... they used a tube preamp.
Do we all agree that a tube preamp is "neutral enough to reveal subtle differences in other components"?

Then, they seem to have only used a single set of speakers.... Magneplanars.
Do we all agree that one set of speakers is sufficient to represent all speakers, of all different brands, and designs, completely?
And do we all agree that, even though Magneplanars aren't especially sensitive to amplifier damping factor, we can discount this?
And, for that matter, do we all agree that Magneplanars (from 1987) are "revealing enough that we trust them not to obscure differences in other components"?

And, for that matter, do we all agree that the provenance of the CD player they chose ensures that it can deliver a perfectly neutral signal?
(If we're going to pick out minute colorations in other gear we should at least start with a signal that is as close to uncolored as we can find... right?)

Something else worth noting...
I've also seen one paragraph repeated and quoted multiple times:

"But for now, the evidence would seem to suggest that distinctive amplifier sounds, if they exist at all, are
so minute that they form a poor basis for choosing one amplifier over another. Certainly there are
still differences between amps, but we are unlikely to hear them."

HOWEVER, nobody bothered to quote the preceding paragraph:

"So for these units, under these conditions, we believe the question has been resolved. But whether or -
not another group of amplifiers in a different situation would yield dramatically different results is still
open to question. This is just the beginning-few scientists would place a great deal of weight on the results of a single
experiment, however extensive, and particularly not an early one. The testing techniques must, and will, be
refined, and a larger body of data will be collected as more such tests are performed in the years to come."

THAT paragraph seems to place the results of this test, and the conclusions the authors derived from them, in a slightly different context.
(For starters - rather than suggesting that further research was unnecessary - they seemed to believe that their results suggested the exact opposite.)

Interesting....

I think the Stereo Review article is a MUCH better guide for the best way to select an amp than audiophile sales pitch and anecdotal impressions. However, on the planet Mars if the sun isi at just the right angle and the entire universe is in alignment, perhaps it would be better to go with the BS advice from stereo salesmen. Thankfully, I have no plans to go to Mars, so I can just take it as an absolute and act accordingly.

I keep my amps below clipping so none of that flurry of made up pseudo-scientific blather applies to me either.

 

[Sgt. Ear Ache]

they were testing amps. Not speakers. Please stop. There is no need to introduce different variables. All components except the amp remain the same and the comparison is made. You know that.

And yes, the test is not the final word. Go ahead and present your evidence that proves it wrong.

 

[bigshot]

I'm getting listening fatigue and I'm not even listening to them much any more. Time for me to post my put up or shut up again...

Every amp, player and DAC that I have ever tested has been audibly transparent. Many other people here in this forum have had the same experience, yet anecdotal comments continue to insist that equipment exists that sounds different than every other piece of similar equipment. Here is my request...

If you yourself have done CONTROLLED TESTING (blind, level matched, direct A/B switched) on a piece of solid state home audio equipment of recent manufacture and have determined that it is not completely AUDIBLY TRANSPARENT, please let me know the make and model and consider lending it to those of us in Head Fi Sound Science who are interested so we can do measurements and listening tests to confirm your results.

Until we can identify a piece of home audio equipment that is colored, we can't speak about why it might be colored or what is causing the difference in sound, because to do so would be pure speculation about something that might not even exist. Let's find out.

Thank you!

 

[Steve999]

Audio Science Review is, of all present options, perhaps the most interesting option. About the best info on objective performance of any currently available gear.

However, despite what you correctly observed, they do not do acid tests - pushing equipment to and beyond the limits.

Quite untrue. In pretty much every review they push the unit to and if possible beyond the limits and tell you if and when the review unit goes bonkers. They even tell you how far back to dial the volume to get things under control.

Go there and have fun!

Just two quick examples from AudioScienceReview.com:

"Now let us talk about another gotcha with this unit. The volume control goes from 0 to 80 in .5 steps. But they aren't db steps. Further, set to 80 (max) you get more than 30% THD. Set to 70 which is about 3 db less you get a few % THD. Set to 69 THD drops to .009% or about – 80 db. 69 is 2.23 volts output and 80 is only 3.23 volts output. A difference of 3.2 db. Why did they allow you to do this? You don't get any real gain for low levels. Why leave it so you could experience 30% THD?"

-OR-

"Intermodulation test was totally nutz with volume control at max as I normally test. Distortion skyrocketed early in both low and high gains. I dialed the volume control back to about 2:00 o'clock and that fixed most of that at the expense of slightly higher noise floor. . .Strangely, high gain did not do anything better than raise distortion level above .3 volt input (pink). There is some design error here. . .Measuring power versus distortion and noise did not do much to make me happy. . .These are dismal power output levels. That aside, what the heck is wrong with low gain mode? Why is it clipping? Low gain mode is supposed to stress the amp a lot less with lower voltage output so there is no reason for it to badly clip. . .High gain has much elevated noise likely due to the mains noise shown in the dashboard. The gap to our reference $99 JDS Labs Atom is massive. . .The same ugliness continues with 33 ohm load. . .I plugged in my Sennheiser HD-650 headphones into [deleted] expecting the outcome to disagree with the measurements. That was not to be the case. Up to 12:00 o'clock on the volume control all was OK. But going much past that caused severe distortion in both low and high gain modes. The latter was not remotely sufficient for this headphone. The distortion was not subtle at all. . . how did listeners not notice the lack of power and quick onset of distortion? I searched and the only reviews I found were the fluff kind with soundstage this, and detail that, missing the major weaknesses in this product. . . To continue selling headphones amps with far less power and more distortion and noise for same or much higher price is like trying to sell flip phones for more money than smartphones."

They gave the Apple headphone dongle a very favorable mini-review, FWIW.

 

[analogsurviver]

Quite untrue. In pretty much every review they push the unit to and if possible beyond the limits and tell you if and when the review unit goes bonkers. They even tell you how far back to dial the volume to get things under control.

Go there and have fun!

Just two quick examples from AudioScienceReview.com:

"Now let us talk about another gotcha with this unit. The volume control goes from 0 to 80 in .5 steps. But they aren't db steps. Further, set to 80 (max) you get more than 30% THD. Set to 70 which is about 3 db less you get a few % THD. Set to 69 THD drops to .009% or about – 80 db. 69 is 2.23 volts output and 80 is only 3.23 volts output. A difference of 3.2 db. Why did they allow you to do this? You don't get any real gain for low levels. Why leave it so you could experience 30% THD?"

-OR-

"Intermodulation test was totally nutz with volume control at max as I normally test. Distortion skyrocketed early in both low and high gains. I dialed the volume control back to about 2:00 o'clock and that fixed most of that at the expense of slightly higher noise floor. . .Strangely, high gain did not do anything better than raise distortion level above .3 volt input (pink). There is some design error here. . .Measuring power versus distortion and noise did not do much to make me happy. . .These are dismal power output levels. That aside, what the heck is wrong with low gain mode? Why is it clipping? Low gain mode is supposed to stress the amp a lot less with lower voltage output so there is no reason for it to badly clip. . .High gain has much elevated noise likely due to the mains noise shown in the dashboard. The gap to our reference $99 JDS Labs Atom is massive. . .The same ugliness continues with 33 ohm load. . .I plugged in my Sennheiser HD-650 headphones into [deleted] expecting the outcome to disagree with the measurements. That was not to be the case. Up to 12:00 o'clock on the volume control all was OK. But going much past that caused severe distortion in both low and high gain modes. The latter was not remotely sufficient for this headphone. The distortion was not subtle at all. . . how did listeners not notice the lack of power and quick onset of distortion? I searched and the only reviews I found were the fluff kind with soundstage this, and detail that, missing the major weaknesses in this product. . . To continue selling headphones amps with far less power and more distortion and noise for same or much higher price is like trying to sell flip phones for more money than smartphones."

They gave the Apple headphone dongle a very favorable mini-review, FWIW.

Wow... - I have to check them out again !

Two or so years ago all of their testing has been limited to 20.000 Hz ( or, maybe, to 22050 ).

And, yes, they have to be applauded for giving good reviews to inexpensive stuff and exposing some ridiculously expensive stuff as being practically broken.

Now, time to check if there is any life on ASR above 22050 Hz .

 

[analogsurviver]

they were testing amps. Not speakers. Please stop. There is no need to introduce different variables. All components except the amp remain the same and the comparison is made. You know that.

And yes, the test is not the final word. Go ahead and present your evidence that proves it wrong.

Testing amps must involve testing them with different speakers. Real world speaker load is anything but a purely resistive load of constant value - such as used for establishing specs and consequent testing. The published spec of an amp is the most rosiest of conditions an amp will almost NEVER be faced to work under in real world use.

Stereo review chose to test with the most benign real world load on the planet - Magnepan. It is the closest approximation of resistive load ( around 6 ohms IIRC ) with a first order crossover - and there is next to none impedance rise in high frequency, such as examplified by any normal dynamic tweeter. Depending on exact model, that impedance may well be within less than one ohm from nominal , across 20 -20 K range - a FAR cry from any more conventional speaker on the market.

Any other speaker used would have presented more challenging load - with bigger differences in sound.

There have been any number of attempts to make the life of an amplifier easier - both commercial and DIY. The first commercial normal box dynamic speaker that adressed the impedance problem has been Cizek ( Model One ) - which, if the large peak in impedance around the port frequency ( unavoidable ) is exempted, really had a very constant and smooth impedance curve across the rest of the spectrum. The Speaker Builder magazine later also published a series of articles addressing the same problem. Catch is that additional elements that have to be added to conventional crossovers slightly reduce efficiency and - predictably - cost more money. So much of the potential cost savings on amplifier are swamped by the more expensive and less efficient speaker.

 

[Steve999]

Wow... - I have to check them out again !

Two or so years ago all of their testing has been limited to 20.000 Hz ( or, maybe, to 22050 ).

And, yes, they have to be applauded for giving good reviews to inexpensive stuff and exposing some ridiculously expensive stuff as being practically broken.

Now, time to check if there is any life on ASR above 22050 Hz .

In some reviews they state that they prefer to see equipment measure well at least up to 40 kHz, in at least one review they stated that they like to see the gear measure well up to 40 kHz in this age of hi-rez. It’s much more of an an amalgam of subjectivist and objectivist, with a huge dose of audio playback engineering expertise there. You might well feel much more at home there. They have two sets of measurement criteria for audible transparency, a lenient set and a strict set, though they state that even with the lenient set the equipment will be audibly transparent. They like to see gear measure way beyond audibly transparent, but if it is so-so they will talk about if the problems are likely to be audible. They can find gear to be audibly transparent and still be very unhappy with the measurements. There are no big disagreements about audible transparency. They will challenge purely subjective impressions with no data or testing to back it up though. It’s quite a big tent and very active. You may well find your own niche there.

Here they make fun of a cable manufacturer in a short review and there is a funny discussion thread afterwards:

https://www.audiosciencereview.com/...o-cable-business-is-getting-out-of-hand.7712/

 

[Sgt. Ear Ache]

Testing amps must involve testing them with different speakers. Real world speaker load is anything but a purely resistive load of constant value - such as used for establishing specs and consequent testing. The published spec of an amp is the most rosiest of conditions an amp will almost NEVER be faced to work under in real world use.

Stereo review chose to test with the most benign real world load on the planet - Magnepan. It is the closest approximation of resistive load ( around 6 ohms IIRC ) with a first order crossover - and there is next to none impedance rise in high frequency, such as examplified by any normal dynamic tweeter. Depending on exact model, that impedance may well be within less than one ohm from nominal , across 20 -20 K range - a FAR cry from any more conventional speaker on the market.

Any other speaker used would have presented more challenging load - with bigger differences in sound.

There have been any number of attempts to make the life of an amplifier easier - both commercial and DIY. The first commercial normal box dynamic speaker that adressed the impedance problem has been Cizek ( Model One ) - which, if the large peak in impedance around the port frequency ( unavoidable ) is exempted, really had a very constant and smooth impedance curve across the rest of the spectrum. The Speaker Builder magazine later also published a series of articles addressing the same problem. Catch is that additional elements that have to be added to conventional crossovers slightly reduce efficiency and - predictably - cost more money. So much of the potential cost savings on amplifier are swamped by the more expensive and less efficient speaker.

sorry. not buying it. They were testing to see if any difference could be noted between two amps. no need to test them under a wild array of conditions. here's two amps. nothing else is changing in the test except the amp. do you hear a difference? Simple. And again, if you have a problem with the test, go ahead and present the evidence you have that proves it wrong...

 

[KeithEmo]

There is a famous quote - often attributed to Einstein (probably incorrectly).
It goes: “Everything should be as simple as it can be, but not simpler”

The simplest way to test amplifiers is by connecting them all to a nice simple resistor load...
It's very standard, big load resistors aren't very expensive, and it's a convenient way to compare things equally....
And, for all those reasons, measuring them with a resistive load is in fact the most common way to measure amplifiers.

The problem is that, while purely resistive test loads are the most common type, virtually ALL real world speakers are nothing like that.
Almost all speakers are the exact opposite - in fact, most represent very reactive loads, with very complex impedance characteristics.
And, yes, different amplifiers do act VERY differently when connected to reactive loads.
In fact, being asked to drive a speaker that is almost purely resistive, like a Magnepan, IS "the wild and wacky exception" for an amplifier.
You might as well compare the handling of several cars - by testing them on a straight road, with a gentle but steady downhill angle, and a perfectly smooth dry surface...
And then claim to be comparing how they handle and how fast they go under "real world driving conditions".
In fact, a large part of designing a useful test is to choose test conditions, and test samples, representative of real world usage conditions.

You can drive a VW Beetle, and a Formula 1 racer, straight down the street in front of your house at 20 mPH...
And then gleefully claim: "Wow... we tested a VW Beetle and a Formula 1 racer - and they handled just the same."
Your statement won't be a lie - but it won't contain any useful information either.

You aren't going to be able to hear the differences in distortion between two even really poor quality amplifiers while playing fuzz guitar music...
And you aren't going to be able to hear differences in frequency response by playing vocals that contain a range of frequencies that any telephone could easily handle...
And you aren't going to find out how they sound with real typical consumer speakers... unless you test them with a variety of real typical consumer speakers.

In that Stereo Review test, they almost certainly chose Magneplanars because they are highly favored by certain audiophiles...
However, they are not at all representative of "the load offered by most typical loudspeakers"...
And neither are they "the loudspeaker most likely to reveal a wide variety of flaws or limitations if they're present"...
- Magneplanars have a very resistive and "benign" impedance characteristic
- Magneplanars have poor low bass response (which is obvious if you look at their specifications)
- Magneplanars are also dipoles (so they tend to highlight certain types of errors very well - while totally obscuring others)
- Also, because of their low mass, and weakly coupled drive mechanism, Magneplanars are extremely INSENSITIVE to variations in amplifier damping
(while virtually all other speakers in common use are the exact opposite)

In short, other than being well liked by a certain group of audiophiles, Magneplanars are a very POOR approximation of "a typical speaker".
They are in fact a very ATYPICAL speaker.
Therefore, unless you happen to own or plan to own Magnepan speakers, the results of their test are quite likely to be useless to you.

There is no need for evidence to "prove the test wrong".
Based on the methodology of the test itself it is simply inadequate to support the convlusions you're looking for.

If you look at the products they chose...
And actually read the verbiage about how they chose what to test...
It becomes obvious that they chose products to satisfy certain requirements.
They chose products that were currently popular among audiophiles.
They also SPECIFICALLY chose products that many audiophiles would be likely to EXPECT to sound very different.
They compared a low cost, solid state, consumer receiver to an expensive tube amplifier of rather unique design.
They didn't even pick a typical tube amplifier - they instead chose a unique and atypical "output transformer-less" one.
Rather than chose typical products - they chose products that would "appear to be as different as possible".
And, as such, they succeeded quite well.

sorry. not buying it. They were testing to see if any difference could be noted between two amps. no need to test them under a wild array of conditions. here's two amps. nothing else is changing in the test except the amp. do you hear a difference? Simple. And again, if you have a problem with the test, go ahead and present the evidence you have that proves it wrong...