Measurement vs. listening. What is the relationship between good/bad measurements and listening tests? What role does bias play in both?
This "issue" is the same as point #1 below.
[1] The post is try to understand what we can learn from the measurements (vs SQ?).
[2] Nothing is perfect, all we need to do is find the issues and fix them. These including the flaws of the designs, as well as the limits of measurements.
1. Assuming SQ to mean audible differences, the problem we have here (in this thread so far) is what do we mean by "listening tests"? Any difference at any level can be audible; take say a jitter artefact at -110dB, increase the gain by say 50dB or so and it's audible! Amirm (and me sometimes) will find a few secs of low level signal somewhere in a music track (that exemplifies a potential issue, say a reverb tail for example), loop it, whack the volume up and DBT two bits of gear or processes. On the other hand we have bigshot (and me sometimes), who conducts listening tests without picking a low level bit of signal and "whacking the volume up", IE. The listening test is conducted under the conditions of "normal listening", where the output level is set so that the peak output level is "reasonable" (comfortable to loud) and not changed. In other words, Amirm's test is not "real world", if he played whole tracks at that test level he would damage his transducers and/or his ears. The problem (if we can call it that) with bigshot's test is that for 20 years (or so/more) digital technology has become so good, at such a low price point, that when comparing most/all digital gear/processess, the differences are all below audibility (at normal/reasonable listening levels). In fact, when comparing most digital gear/processess, this process of picking a low level bit of signal and whacking up the volume (or in some situations, doing multiple loopbacks instead) is the only way of audibly differentiating differences and is therefore the de facto standard testing procedure.
In answer to SoundAndMotion: Under normal listening conditions, at any reasonable playback level level, there is NO relationship between good/bad measurements and listening tests! Even the bad measurements of modern DACs are outside audibility. However, it's of course entirely possible that bias could play a role. Outside of a controlled test, knowing that one DAC (for example) has peak jitter atrefects at -100dB, while another is at -130dB, may cause a biased listener to believe/perceive the latter as sounding better.
2. The limits of measurements are typically thousands to many millions of times beyond audibility, so that's not an issue and hasn't been for several decades. The other problem is that although "nothing is perfect", the "issues" which exist, even with cheap modern consumer DACs, are all below audibility (under normal listening conditions), so why do we "need to fix them"? The PSAudio DAC is an interesting example though, some of the measurements are by far the worst I've seen, even if it were only a $100 DAC! The balanced output voltage is effectively faulty and I've never seen intermodulation distortion that bad, even on 25 year old DACs but even given this atrocious measurement and faulty output voltage, still the differences would probably only be audible under normal listening conditions with certain amp/transducer combinations, specific recordings and good/trained listening skills!
G