Originally Posted by Willakan
The concept of them sounding the same is not reliant upon null testing - it simply is based on the measurements. It is true that there is no set boundary where we say "Yes! This is a good solid state amp" (leaving tube amps and designs that alter the sound out of it, as they are designed to do so). However, when every measurement is vastly beyond the threshold of audibility, it is not unreasonable to say it is a well engineered amp. Take the hypothetical amp I now place before you - I have chosen measurements which are commonplace for good solid state amps:
Hence I feel relatively confident, when faced with sets of excellent measurements, to conclude that the systems from which they are taken produce sound that is effectively identical. The reason null tests do not abound is that the sort of people with the equipment and expertise to perform them likely feel satisfied that they are not required.
I am not a professional amp designer, just a hobbyist EE who does sometimes do high-end speakers. I have to say there is a lot of interesting information in this discussion, and hopefully it will avoid going off the rails like the silly DAC thread did, so thanks everyone for such interesting reading.
Willaken, while much of what you write is interesting you make a fundamental assumption in "measuring" which is not correlated to real-world performance of systems: load.
Virtually all "measurements" of amps rely on specific, usually totally resistive loads. Headphones and speakers often have non-linear impedance, and this couples with factors like the output impedance of an amp to actually create variations in power-spectrum of up to 3dB in variance. That's a factor of two in power. In other words, unless you are driving an ortho with a pretty ruler-flat impedance curve, just the nature of the headphone interaction with a given amp will absolutely change the sound. Some people can pretty easily hear 1dB in bass boost, others can not. Now if you level match at 1KHz, one system will be perceived as having slightly more impactful or extended bass, even if the amps test identically into a resistive test load.
There are other factors related to feedback, transient intermodulation, and a few other factors that vary as a function of load.
Also, I have clearly heard in my amps that the less feedback I used globally, the more open and relaxed the sound becomes, but it's NOT from distortion, which I can usually pretty easily hear. Even on the fanciest test gear the lower feedback or no feedback version sounded better but tested worse.
Another example was when I was designing some speakers and my ex wife had been listening because she was curious. Later that day I swapped out a single quality polystyrene cap with a cluster of parallel caps from a crazy expensive source. She walked in and said "What did you change? The treble sounds so much better..." I had been unable to measure any difference, but was thinking I heard something, and was worried about placebo clouding my perception because I couldn't easily A/B.
Another interesting example is the cult of single-transistor amps in Japan. Someone once took me to a "show" where these guys were swapping the transitor in their circuits out and trying different ones. Some circuits had feedback, others didn't, but you could hear the difference with some of the transistors pretty easily. Nobody was doing any measurements, though...
So to sum up my point, the challenge with testing is it can only show certain results under certain conditions and it is possible we just don't know what to measure that correlates to perception, vs to some simple metric like THD. Measurements are the beginning but not the end of understanding.
As a result, when I do do designs, I test then listen, and sometimes blind A/B. Sometimes it's been clear I could hear something different but couldn't measure it, and other times I could not hear one. BTW, we did use null testing too, including analog when we wanted to try to measure against ...