Measurements - Reliable way to gauge audio accuracy?

Quote:

Originally Posted by Dibster 

What's been bothering me lately is the fact my current soundcard, the Asus Xonar STX, boasts incredibly good measurements for something that costs a fraction of the price of external devices that have a hard time matching it. I have extensively compared it to a Musical Fidelity V-DAC and a Heed Canamp. Subjectively, the STX offers a very dry and often shrill representation of the recording. The former duo is very smooth and lush in comparison, however - again subjectively speaking - the STX does seem to be more resolving.

 

Now, we all have our preferences of how our albums should sound. Some like tubes, some solid state. Some like "warm", some "analytical" etc. However, if we're speaking purely of how close to the original recording we can get, do you feel these measurements are a reliable way to see it? Would the Xonar STX truly beat a 1300€ DAC/AMP combo geared towards accuracy in being true to what is on the CD as measurements would suggest?

If you define accurate as being the least deviation from the source, then suitable measurements (if done accurately) can indicate the level of deviation. A high fidelity item that is accurate (by the above definition) should neither add nor take away. It should not add (or only add very small amounts of) distortion (harmonic or intermodulation) or noise or crosstalk or time-based variations, it should not take away , there should be no frequency response dips or roll-offs before say 20K and it should maintain the same dynamic range across audible frequiencies.

We have measurements that indicate the level of deviation for these parameters.

That said, the difference between an SNR of 120db and one of 110db is pretty irrelevant, they are both subjectively silent until you crank the volume to absurd levels, jitter at anything below 10ns P-P rms has never rigorously been shown to be an audible problem, distortion at 0.05 and 0.025 percent are subjectively indistinguishable. But such measurements can tell you some useful things, for instance a 24 bit DAC with an SNR of 110db is not giving you all 24 bits, you are getting about 18/19.

Very nice Nick.

Dibster, have you taken (a look at) measurements where a dummy load was connected to the device?

Loopback (line-out -> line-in) or "unloaded" measurements are pretty useless if you intend to use the device under test to drive headphones.

*deleted as I responded about speakers rather than DACs*

Quote:

Originally Posted by Dibster 

Which measurements do you believe are the most difficult for modern dacs and amps to master, and cause the largest audible differences between devices?

Modern DACs properly designed and not bizarre NOS tweako cultist designs can manage very high performance on any of the standard measurement parameters, Digital audio is a relatively mature technology, it is possible to buy fundamentally technically awful kit and pay a bucketload of money for it into the bargain (look at a few Stereophile reveiews) but a bit of due diligence will allow you to avoid this.

As for audible differences, the single most common cause of perceiving differences between DAC A and B is that they are just not level matched, the line-out on DACs can vary enormously, enough to make comparisons difficult.

The 2nd most common cause of perceiving differences between DAC A and B is a delay between comparisons, our memory for gross details  (such as your mother's telephone voice or background noise on LP) is very good over time, for smaller differences such as a fraction of a db treble lift not so much and delays again make comparisons flawed.

The 3rd most common cause of perceiving differences between DAC A and B is expectations, somebody tells you that A and B are different and so you hear differences, this is very basic human Psychology (the subject of my 1st two degrees).

Then there are real differences. If there is a real reliably detected under controlled conditions audible difference then there is a real physical cause. This may be then due to such things as differences in Noise, FR, Distortion, Crosstalk. There is no good evidence at present to suggest jitter is normally an audible issue in competent digital kit. Others speculate about as yet undiscovered physical properties that may be responsible, who knows , but there are so many other more likely causes that I would worry about these undiscovered things rather less until better explanatory models and testable hypotheses for these phenomena arrive (IMO).

A good question to ask yourself is "why should these things sound different ?"

N.O. ...

Yes

The more I read and the more kit I listen to the more the connection between measurements and actual audibility is weakening.

But then again, even when two different things measure the same, some will still say they sound different but then deny it is all in their head.

Even if measurements can make differences look BIG they can be completely inaudible because, like nick_charles said, modern equipment achieves very high performance, even some really cheap devices achieve performance that can be considered transparent.

But that doesn't make measurements unreliable or inaccurate. Humans are.