[raddle]

Measurements of audio equipment can never be the ultimate arbiter of the accuracy of reproduction of a system; only listening can be that. Measurements are models.
 
An example of a model is a map that Google Maps produces. It shows you roads and buildings. If you use the map to get to a building shown on Google Maps, and when you arrive there it turns out that the building isn't there, what would you conclude?
 
A. Google Maps is wrong.
B. You are hallucinating the absence of a building.
 
Generally (A) would be correct. That's because the map is just a model; it's not reality itself, but "stands in" for reality. It's not a complete description of reality and it can differ from reality. People know that about maps, but for some reason a lot of people are taking option (B) when it comes to audio when they put their faith in measurements.
 
The reality of the listening experience is a subjective experience that results from the behavior of the audio equipment and the listener's brain. The measurements are just models of that situation. E.g. the frequency response curve is a model calculated assuming the device is linear. It doesn't capture nonlinearities at all. It's incomplete. It corresponds roughly to what people hear (that is, it roughly models the listener's brain), but doesn't capture very much of the behavior of a brain.
 
Measuring harmonic distortion at a given frequency is a *very* small slice of reality. It takes a number of complex mechanisms, reduces them to a steady state (causing a loss of information about reality), and then reduces the entire nonharmonic energy to a single number, further losing information.

 

[Speedskater]

So if in making internal parts for a automobile race engine, it's better to have someone look at the parts rather than measure them to determine if they are accurate?

 

[raddle]

  So if in making internal parts for a automobile race engine, it's better to have someone look at the parts rather than measure them to determine if they are accurate?
 

 
The quote at the top of the sound science forum says: "The purpose of science is not to analyse or describe but to make useful models of the world. A model is useful if it allows us to get use out of it."
 
This actually says it all.
 
It sounds like you've never thought about this critically before, so I'll pose a question to you: what is being modeled? What is the system whose behavior we are interested it?
 
Answer that question for race cars.
 
Then answer that question for audio.
 
Consider then, what use we are trying to get out of models?
 
This will answer your question.

 

[raddle]

  So if in making internal parts for a automobile race engine, it's better to have someone look at the parts rather than measure them to determine if they are accurate?
The purpose of science is not to analyse or describe but to make useful models of the world. A model is useful if it allows us to get use out of it.

 

[dvw]

  Measurements can never be the ultimate arbiter of the accuracy of reproduction of a system; only listening can be that. There are multiple reasons why measurements can't determine accuracy. Only one of them is that measurements are models.
 
The reality of the listening experience is a subjective experience that results from the behavior of the audio equipment and the listener's brain. The measurements are just models of that situation. E.g. the frequency response curve is a model calculated assuming the device is linear. It doesn't capture nonlinearities at all. It's incomplete. It corresponds roughly to what people hear (that is, it roughly models the listener's brain), but doesn't capture very much of the behavior of a brain.
 
Measuring harmonic distortion at a given frequency is a *very* small slice of reality. It takes a number of complex mechanisms, reduces them to a steady state (causing a loss of information about reality), and then reduces the entire nonharmonic energy to a single number, further losing information.
 

Are you saying measurement is a calculation. Frequency response is a calculation? Please explain how the frequency response is calculated. Please also explain the non-linearity and the complex mechanism that you are talking about. I do not understand how THD is a "small slice of reality".
 
With your beginning statement; all amplifiers and cable should sound the same if they are accurate. So based on the reviews here, you are saying there is no such thing as an accurate cable since people can hear difference. Difference usually mean they are not the same meaning inaccurate. What may I ask, is the complex mechanism that changes the signal.

 

[proton007]

I'll begin with the OP's assumption.
Here are the factors on the basis of which a model is evaluated:
-- A model is evaluated first and foremost by its consistency to empirical data; any model inconsistent with reproducible observations must be modified or rejected
-- Ability to explain past observations
-- Ability to predict future observations
-- Cost of use, especially in combination with other models
-- Refutability, enabling estimation of the degree of confidence in the model
-- Simplicity, or even aesthetic appeal

I'd say our current way of measurement does a pretty good job of ticking most of the points.
About the Hearing part, its a question of aesthetics.
When people appreciate a painting, does the fact of it being the original or a replica change the feelings it evokes? In most cases, yes.
The biggest point here is a priori information. The fact that one's feeling towards the art changes based on knowing before, not after the viewing, that the object in question is a fake, makes this model useless.

In a nutshell, the perception of our senses of sight and sound are inconsistent. The model of perception is incomplete, or flawed.
So, among the two models, which one should we choose?

 

[dvw]

The hearing part is not modelled as far as I know. When I was in school, I worked for a professor that was doing model of the auditory cell. But that will still not account for the emotional response of listening to a song. Are there psychological or social model that can predict a response? Maybe profiling is the right term.

 

[raddle]

  The hearing part is not modelled as far as I know. When I was in school, I worked for a professor that was doing model of the auditory cell. But that will still not account for the emotional response of listening to a song. Are there psychological or social model that can predict a response? Maybe profiling is the right term.

 
Frequency response is a simple model of hearing as well as a model of devices. Think about this question: why do we measure frequency response? Why is that included among the important measurements of a device?

 

[raddle]

  Are you saying measurement is a calculation. Frequency response is a calculation? Please explain how the frequency response is calculated. .

 
This is a very good question because it really is important to understanding measurements. Here's how frequency response is calculated:
 

1. you take a device under test
2. you put a signal into it (you don't know exactly what signal due to distortion and noise in the test equipment, but you have a fair idea)
3. you measure the signal coming out it (your measurement is itself a model due to distortion and noise in the test equipment)
4. with these two signals you can compute the frequency response - I'm forgetting the exact mathematics involved, and I'm sure one of the scientists here could inform us, but it's something like a reverse Fourier transform

 

[raddle]

I'll begin with the OP's assumption.
Here are the factors on the basis of which a model is evaluated:
-- A model is evaluated first and foremost by its consistency to empirical data; any model inconsistent with reproducible observations must be modified or rejected
-- Ability to explain past observations
-- Ability to predict future observations
-- Cost of use, especially in combination with other models
-- Refutability, enabling estimation of the degree of confidence in the model
-- Simplicity, or even aesthetic appeal
 

 
Thanks for this, these are all good points and actually critical to my point. But I would ask you to answer several questions:
What is the system being observed? (This may have more than one answer.)
What are the observations?
What is being predicted?
How is the model used to predict that?
 

About the Hearing part, its a question of aesthetics. When people appreciate a painting, does the fact of it being the original or a replica change the feelings it evokes? In most cases, yes. The biggest point here is a priori information. The fact that one's feeling towards the art changes based on knowing before, not after the viewing, that the object in question is a fake, makes this model useless.

 
A more accurate analogy would be this: you have an original painting, and you have two reproductions, and you are evaluating which one is more accurate. How would you propose to do this?
 
"Knowing before" has nothing to do with my points, because you can carry out all experiments blind and you are still dealing with these fundamental factors (such as that measurements are models, not reality).

 

[mikeaj]

To me, when talking about electronics, the system being observed is the electronics. Hence you take electrical measurements (observing the output electric signal for some different inputs), which are accurate within some known limitations and degree, to evaluate the performance. There would be some different metrics for accuracy. For loudspeakers you'd also take acoustical measurements.
 
Because these systems are largely close to linear, especially in the ranges you want to operate them, you can apply the most basic systems theory models to the performance and then account for nonlinearities when appropriate (more models). How good these models are depends on the systems, the inputs you're interested in, and so on. The higher performance the system, the less deviation there should be from the models because they're built to emulate what the models describe. There are also stochastic models for the noise if necessary. 
 
 
If you want to evaluate human listener preferences or some notion of perceptual accuracy for the electronics or whatever else, you measure the human preference or response in listening tests. That's a different issue. You could even use brain scans if you wanted; in any one of these scenarios there is some amount of concern whether or not the test conditions are invasive or non-ordinary in some sense and skewing the results. Animal behavior in captivity may be different than animal behavior in the wild, so if the latter is what you're interested in... Then again, for audio one would hope that these nuisance factors and level of invasiveness are relatively small compared to the kind of claims made in showrooms, magazines, and forums (if A sounds better than B when normally listening, it should be apparent in a listening test as well).
 
In practice there is some research on psychoacoustics and some things known about the relationship between sounds and signals produced and then their response on human listeners, so for some applications or general ideas the machine measurements are useful for describing what people perceive.

 

[xnor]

  Measurements can never be the ultimate arbiter of the accuracy of reproduction of a system; only listening can be that.

When you talk about listening, you mean blind listening tests right?
 

There are multiple reasons why measurements can't determine accuracy. Only one of them is that measurements are models.

I don't see how "model" excludes "determining accuracy".
 
 
 

An example of a model is a map that Google Maps produces. It shows you roads and buildings. If you use the map to get to a building shown on Google Maps, and when you arrive there it turns out that the building isn't there, what would you conclude?  
A. Google Maps is wrong.
B. You are hallucinating the absence of a building.

No, given your constructed example some possibilities are:
- Reality changed, e.g. the building was torn down in the past and the map has not been updated yet.
- Some guy made a mistake typing in the data or developing the algorithm that creates the virtual map.
- The original map data was wrong.
 
None of these points translate well to something like a headphone amp FR measurement.
 
B) is just a non sequitur, a cheap attempt trying to discredit the fact that biases change our perception.
 
 

Generally (A) would be correct. That's because the map is just a model; it's not reality itself, but "stands in" for reality. It's not a complete description of reality and it can differ from reality. People know that about maps, but for some reason a lot of people are taking option (B) when it comes to audio when they put their faith in measurements.

Since B is a joke answer you leave no other option than A. Also your example has like no relevance to audio.
When it comes to audio then we have evidence in the form of blind listening tests combined with measurements. If you know you can hear a difference between cable A and B then prove it! People can't when measured performance is close enough.
 
What people are taking on faith are subjective listening impressions made by others.
 
 

The reality of the listening experience is a subjective experience that results from the behavior of the audio equipment and the listener's brain. The measurements are just models of that situation.

You could argue the same for graphics cards and the output on a computer monitor. You could argue that any kind of benchmarks are just models and you only know how the graphics card performs by putting it into your computer and subjectively experiencing it...
 
 

E.g. the frequency response curve is a model calculated assuming the device is linear. It doesn't capture nonlinearities at all. It's incomplete. It corresponds roughly to what people hear (that is, it roughly models the listener's brain), but doesn't capture very much of the behavior of a brain.  
Measuring harmonic distortion at a given frequency is a *very* small slice of reality. It takes a number of complex mechanisms, reduces them to a steady state (causing a loss of information about reality), and then reduces the entire nonharmonic energy to a single number, further losing information.

Ask yourself the question: how does nonlinearity change the frequency response? Right, within an acceptable range it doesn't...
 
Yes, a single THD number is less useful than a THD vs. frequency plot, but not useless.
If you're interested in high fidelity then THD should be very small to begin with, so the precise distortion profile doesn't matter.
 
 

By the way, regarding options A and B above, I predict that someone is going to tell me that any differences in wire are placebo effect. This is a complete non sequitur, although it's one the favorite topics pro-measurement guys like to harp on.

Not any difference, but any difference between roughly similarly measuring cables. And you can get good cables for a couple of cents/m.
If you think otherwise: provide evidence.
 
And btw, I'm not pro-measurement, I'm pro reality. What you perceive is a version of reality that was delayed and altered (bias!) by your brain and senses.

 

[dvw]

   
This is a very good question because it really is important to understanding measurements. Here's how frequency response is calculated:
 

1. you take a device under test
2. you put a signal into it (you don't know exactly what signal due to distortion and noise in the test equipment, but you have a fair idea)
3. you measure the signal coming out it (your measurement is itself a model due to distortion and noise in the test equipment)
4. with these two signals you can compute the frequency response - I'm forgetting the exact mathematics involved, and I'm sure one of the scientists here could inform us, but it's something like a reverse Fourier transform

This is not accurate. A frequency response is just that the bandwidth of a device. You can take the frequency as a transfer function. This function will dictate what the output will be with a certain input. The frequency response curve is simply a compi;ation of data, there is  no calculation involved.

 

[proton007]

Thanks for this, these are all good points and actually critical to my point. But I would ask you to answer several questions:
What is the system being observed? (This may have more than one answer.)
What are the observations?
What is being predicted?
How is the model used to predict that?

You tell me. Which one do you wish to investigate? I presented two models, one of measurements, the other of perception.
If you're saying theres another way of representing reality, wouldn't it be a model as well?

A more accurate analogy would be this: you have an original painting, and you have two reproductions, and you are evaluating which one is more accurate. How would you propose to do this?

"Knowing before" has nothing to do with my points, because you can carry out all experiments blind and you are still dealing with these fundamental factors (such as that measurements are models, not reality).

Stop twisting and paraphrasing other's words into your own arguments. Use your own logic to refute the counter arguments, otherwise its just a waste of everyone's time.

I'll refute your last point with this:
Do you have another way of consistently explaining reality? If you do, wouldn't it be a model as well? Aren't you choosing between models then?

 

[dvw]

   
Frequency response is a simple model of hearing as well as a model of devices. Think about this question: why do we measure frequency response? Why is that included among the important measurements of a device?

Frequency response is not a model of hearing. The frequency response is a parameter of physical limitation. For example you can design a product to have a GHz bandwidth. But do you need all the extra cost? So to design a device you need to figure out what your parameter is? If you can only hear up to 20KHz, you don't really need bandwidth of more than 20KHz. The same is true with video. The eye can't really see past 60 frames. Ever wonder why your phone has lousy frequency response? It's simply because it designed for human voice.