eucariote
1000+ Head-Fier
- Joined
- Oct 15, 2009
- Posts
- 1,474
- Likes
- 76
People have been arguing about amps for a while, whether they all sound the same so long as they don't suck and add distortion (a 'wire with gain') or if better designed, more powerful amps really do sound better. With the help of a friend, I ran blind tests of three different headphone amplifiers- a Magni 3 (160mW @ 300 Ω ), an original uDac (4.3mW @ 300 Ω ) and a Yamaha HTR-5630 integrated amplifier (?) to see if we could hear a difference without the help of sight and expectation. We used HD6XX headphones (300 Ω ), since they would be less susceptible to damping due to the probably high output impedance of the Yamaha.
We used hypothesis testing, where you assume the null hypothesis, that experimental outcomes are random, unless you can show that the outcomes are so consistent that they would be very rare if they were random, occurring below a 5% threshold (P < 0.05) assuming a probabilistic distribution of random outcomes, here a binomial for two correct/incorrect choices. In that case you reject the null hypothesis and accept the experimental hypothesis that there is a real systematic effect (choice) due to an independent variable (amp).
The first test was a blind comparison of the Magni vs the uDac. Sighted, I swear that the Magni has noticeably more slam, soundstage, plankton, I love this thing. I had my back turned to the gear while my friend flipped a coin, unplugged the headphones and plugged them in the Magni (heads) or the uDac (tails). I took all the time I wanted to guess which amp I was listening to, but usually guessed after 5-10 seconds. Since the Magni makes the sound of a resonant box when you plug in headphones, we used a short headphone adapter cord for it (picture). The song was a lossless recording of Olé by John coltrane, which has fairly crisp and constant bass, drums and saxophone to hear all the frequencies. The uDac was the dac for both amps and they were volume matched with pink noise.
In 30 trials, I correctly guessed the amplifier I was hearing 14 times (P = 0.71). For the second test, we switched roles and my friend correctly guessed the amplifier 13 times (P = 0.82). For the third test, I tried to hear the difference between the Magni and the Yamaha. Since they have different inputs we had to use Spotify/Chromecast for the Yamaha, which has its own dac and a matching 256 kbps file for the Magni/uDac and used headphone extension cords for both amps since both make a distinct plug/unplug sound. The test song was M83's Skin Of The Night since it gives me chills when listening on the Yamaha because of its wider, deeper (sighted) sound. In 25 trials, I correctly guessed the amp 15 times (P = 0.21). If these last results were consistent for ~105 trials we might have reached P < 0.05 but there would be a 55% chance of a type 2 error (a false positive).
The null hypothesis was not rejected for any of the three tests (P > 0.05) so in conclusion, neither of us could hear a difference between any of the three amps when relying on hearing alone. It might be the case that with different songs, headphones or better listening ability the results might have been more consistent. On that last point I consider myself to have pretty experienced ears, owning a massive library of lossless music files and having owned/heard many (100+) expensive headphones, dacs and amps. Even though I literally hear differences when not blinded, more than half the inputs to primary auditory cortex come from within the brain and not the cochlea, so in this case it is almost certain that vision and expectation, and not sound alone are making the Yamaha and Magni sound better.
Edit- TL;DR- a buddy and I listened to three amps without seeing what we were listening to and we could not tell them apart at all. Their sound is so similar that any difference is very very subtle to the point of being imperceptible.
We used hypothesis testing, where you assume the null hypothesis, that experimental outcomes are random, unless you can show that the outcomes are so consistent that they would be very rare if they were random, occurring below a 5% threshold (P < 0.05) assuming a probabilistic distribution of random outcomes, here a binomial for two correct/incorrect choices. In that case you reject the null hypothesis and accept the experimental hypothesis that there is a real systematic effect (choice) due to an independent variable (amp).
The first test was a blind comparison of the Magni vs the uDac. Sighted, I swear that the Magni has noticeably more slam, soundstage, plankton, I love this thing. I had my back turned to the gear while my friend flipped a coin, unplugged the headphones and plugged them in the Magni (heads) or the uDac (tails). I took all the time I wanted to guess which amp I was listening to, but usually guessed after 5-10 seconds. Since the Magni makes the sound of a resonant box when you plug in headphones, we used a short headphone adapter cord for it (picture). The song was a lossless recording of Olé by John coltrane, which has fairly crisp and constant bass, drums and saxophone to hear all the frequencies. The uDac was the dac for both amps and they were volume matched with pink noise.
In 30 trials, I correctly guessed the amplifier I was hearing 14 times (P = 0.71). For the second test, we switched roles and my friend correctly guessed the amplifier 13 times (P = 0.82). For the third test, I tried to hear the difference between the Magni and the Yamaha. Since they have different inputs we had to use Spotify/Chromecast for the Yamaha, which has its own dac and a matching 256 kbps file for the Magni/uDac and used headphone extension cords for both amps since both make a distinct plug/unplug sound. The test song was M83's Skin Of The Night since it gives me chills when listening on the Yamaha because of its wider, deeper (sighted) sound. In 25 trials, I correctly guessed the amp 15 times (P = 0.21). If these last results were consistent for ~105 trials we might have reached P < 0.05 but there would be a 55% chance of a type 2 error (a false positive).
The null hypothesis was not rejected for any of the three tests (P > 0.05) so in conclusion, neither of us could hear a difference between any of the three amps when relying on hearing alone. It might be the case that with different songs, headphones or better listening ability the results might have been more consistent. On that last point I consider myself to have pretty experienced ears, owning a massive library of lossless music files and having owned/heard many (100+) expensive headphones, dacs and amps. Even though I literally hear differences when not blinded, more than half the inputs to primary auditory cortex come from within the brain and not the cochlea, so in this case it is almost certain that vision and expectation, and not sound alone are making the Yamaha and Magni sound better.
Edit- TL;DR- a buddy and I listened to three amps without seeing what we were listening to and we could not tell them apart at all. Their sound is so similar that any difference is very very subtle to the point of being imperceptible.
Last edited:
even for me who's complete garbage at handling stats.
