I agree that trying to quantify the audibility of distortion with single figures of merit is impossible, for several reasons.
Distortion-causing mechanisms are nonlinear, and the resulting degree of distortion is level dependant. Listening for distortion with fixed level sine waves is therefore unrealistic because the result is steady-state, not varying. Distortion becomes more audible with time, and a distorted music signal doesn't remain at a constant degree of distortion for anything but brief moments. The audibility vs time aspect of distortion was a major factor in determining the meter ballistics of a PPM (Peak Program Meter), with a deliberately slower than true-peak characteristic.
Distortion is nearly always harmonically related to the undistorted signal. With music, most harmonic content that forms the timbre of a sound is also directly related to the fundamental. It becomes much more difficult to hear distortion with complex waveforms vs pure sine waves. Music is almost never composed of pure sine waves.
Masking, the phenomenon of one sound masking another close in frequency but separated in level, is always active. Music's spectrum results in effective masking of harmonically related distortion products, but is also highly dynamic. Masking changes constantly, and depends on the music signal spectrum vs level vs the spectrum and level of the masked signal. It's another reason there can be no single figure of merit.
Related to the above, distortion products that make up a single figure of merit (like .5% THD, for example), can be any combination of the number of harmonics and their relative level, and can be any mix of even and odd-order harmonics, all depending on the specific type of nonlinearity that caused the distortion. An example might be the spectrum of a hard-clipped sine wave resulting in 1% THD (odd-order harmonics) vs the spectrum of a soft-clipped sinewave with the same resulting THD (also odd-order, but a different distribution) vs a non-clipped sinewave with crossover distortion at the same level (even order harmonics). All three examples have the same THD figure, but sound quite differently. Clipping distortion increases with level, and at a rate that changes with how the hard clipping effect is, where crossover basic distortion increases as level goes down, becoming less audible with higher signal levels.
Distortion audibility is very complex, and depends on many factors. Attempting to draw conclusions about distortion audibility by the use of tone mixes, like the mix of 40 Hz and 3 kHz, would be incorrect as that signal does not represent the results of any kind of nonlinearity, is steady-state, and entails no masking. Using tone mixes alone (without a musical masking signal) is an exaggerated and unrealistic test condition. Mixing fixed, non-harmonically related tones below music or speech is also inconclusive because distortion products caused by nonlinearity are always harmonically related and level-related to the original signal, resulting in greater masking and less distortion audibility.
David Clark's paper, "High Resolution Double-Blind Testing", (JAES Vol. 30 #5, May 1980, page 337) describes ABX testing attempts made using a distortion generating device named the "Grunge Box", and a schematic is provided. With musical signals, between 1% and 3% was stated as possibly audible, but clearly there is much more to know about what was going on than the stated single figure of merit. But he does indicate that distortion audibility using nonlinear distortion generators and music as a test signal result in a much higher audible threshold than when using fixed pure tones.
