Testing audiophile claims and myths
May 18, 2015 at 4:17 PM Post #6,001 of 17,336
  Could somebody recommend me a good book to read about the science behind audio?

 
I thought this was pretty good.
 
 
5f60ab2c_expert_cover.jpeg
 
 
May 18, 2015 at 4:18 PM Post #6,002 of 17,336
  I am not over-joyed at the superfluous negativity from both of you but the dialog is an excellent opportunity to review some basic critical thinking skills and the methodology of science.   Let me ask both of you to comment on the following.
 
Occam's razor says among competing hypotheses that predict equally well, the one with the fewest assumptions should be selected (to test first).  How does each of your positions stack up against the razor?
 
As a practical matter, it's often the case that we test several hypotheses and often pick the easiest or least expensive or least time consuming first for obvious but illogical reasons.  My question to you both is:
 
1.  what are all the reasonable available hypotheses that explain the facts available to us?
 
2.  which has the least assumptions?
 
3.  How can we test it?
 
Let's stick to the logical analysis of the phenomenon at hand, I really don't care which planet either of you comes from.

One must be careful as I can make something sound simple that is completely wrong but looks good to many people that might not have the proper knowledge. I do happen to like our dear friend Occam.
 
May 18, 2015 at 4:42 PM Post #6,003 of 17,336
  That would be a very defective amp. It is supposed to amp the signal without degrading the frequency response. I know that some amps can alter, but by +-5DB it is a lot to change in response, i would not think that any amp producers would really make one that is so distant from the perfect linear frequency response.

Exactly! 
 
May 18, 2015 at 4:44 PM Post #6,004 of 17,336
   
I honestly can't imagine where you learned that....
 
In science class, what we learned was that science was a process....
 
1) you make observations
2) you discuss your observations and come up with a theory to explain them
3) you devise and run experiments to test those theories
4) either the experiments confirm your theories or not
5) if not, then you modify your theories based on your results
6) then you run new experiments to test your new theories
(basically repeat until the results of your experiments agree with the results your theories predicted)
 
I've NEVER seen a science book that claimed science consisted of "deciding something" - then ignoring any observations that seem to disagree with it.
 
And, in real-world science, it is often NOT the person or group that first comes up with a theory that ends up running the experiments (although it also often is).

 I am a working scientist- and that is my modus operandi.
 
May 18, 2015 at 6:06 PM Post #6,006 of 17,336
   
Hmmmmmm.  Let's see. I've actually compared how the Lyr and the Asgard sound - and I say they sound different. The manufacturer says they sound different. Virtually everyone I know who owns either, but has auditioned both, agrees that they sound different. Yet you, who don't even claim to have actually heard either one, "just know" that they "can't possibly" sound different based on the specs (which neither one of us has confirmed anyway).
 
That's really impressive.... but it's getting to be a bit boring.
 
Perhaps it's not really rolled off - and it's some other sort of "psychoacoustic effect" - or perhaps their specs are just wrong (I didn't measure it).

 That is a conundrum. How are the tools used to measure calibrated? In analytical biochemistry we have some standards we use to make sure that a measurement taking by person A on day one is the same (within excepted variation) with the measurement taken by B on another. And depending on how critical the sample is- possibly across labs.
Is there a universal standard that all instruments use? If you take 5 such devices for measuring the usual suspects do they all give the same value? First order, second order, third order harmonic distortion at 1000 hz (obviously measuring the 2nd harmonic of 12 Khz is outside of audibility) . Intermodulation distortion? Group delay? What else? Even vs. Odd harmonics? Square wave response? Ringing?
Are we measuring everything that can affect the quality of sound? 
 
May 18, 2015 at 6:30 PM Post #6,007 of 17,336
Following on, all the measurements we make are important.  It's sometimes not hard to show what happens or doesn't happen when one or more of them is way out of line.  Tke 10% distortion at 1KHz for example.  Most of us would notice that (another assumption). But there are a lot of instances where specs aren't all that different yet people claim they hear a difference between two pieces.  In this case it's two pieces of Schiit.
 
This thread has been down this path many times.  I can't tell you how many times I have evaluated stuff before buying it and had a different reaction to that of others who reviewed it.  This points to a need for objective testing. Since we know some of the numbers that tell us important things about gear, it's valuable to have them even if we can't explain why we hear a difference.  To go back to Occam's razor, the first and most obvious question to ask is "is there a difference in sound better the two (even if everyone agrees there is)?" 
 
You know where this is going.  If you don't do a DBT of some kind, maybe ABX, maybe not--there are other designs that can be used, you don't really know if the gear sounds  different, or how it sounds different.  Another way to say this is if you make the assumption that all the reports are correct and they are not, you will never ask the next right question that will lead you to why they are different.
 
In the case being discussed I don't think it's unlikely that there is a difference in sound between the two pieces.  I'm not contesting that.  Just saying that good methodology is to independently verify that listeners can tell the two pieces of gear apart.

Why do we always end up here?  And why is this so seldom done?  Why can we only talk about this here?  Life has so many imponderables.
 
May 18, 2015 at 6:38 PM Post #6,008 of 17,336
  Following on, all the measurements we make are important.  It's sometimes not hard to show what happens or doesn't happen when one or more of them is way out of line.  Tke 10% distortion at 1KHz for example.  Most of us would notice that (another assumption). But there are a lot of instances where specs aren't all that different yet people claim they hear a difference between two pieces.  In this case it's two pieces of Schiit.
 
This thread has been down this path many times.  I can't tell you how many times I have evaluated stuff before buying it and had a different reaction to that of others who reviewed it.  This points to a need for objective testing. Since we know some of the numbers that tell us important things about gear, it's valuable to have them even if we can't explain why we hear a difference.  To go back to Occam's razor, the first and most obvious question to ask is "is there a difference in sound better the two (even if everyone agrees there is)?" 
 
You know where this is going.  If you don't do a DBT of some kind, maybe ABX, maybe not--there are other designs that can be used, you don't really know if the gear sounds  different, or how it sounds different.  Another way to say this is if you make the assumption that all the reports are correct and they are not, you will never ask the next right question that will lead you to why they are different.
 
In the case being discussed I don't think it's unlikely that there is a difference in sound between the two pieces.  I'm not contesting that.  Just saying that good methodology is to independently verify that listeners can tell the two pieces of gear apart.

Why do we always end up here?  And why is this so seldom done?  Why can we only talk about this here?  Life has so many imponderables.

" In the case being discussed I don't think it's unlikely that there is a difference in sound between the two pieces.  I'm not contesting that.  Just saying that good methodology is to independently verify that listeners can tell the two pieces of gear apart.

Why do we always end up here?  And why is this so seldom done?  Why can we only talk about this here?  Life has so many imponderables."
Well said!
 
May 18, 2015 at 7:04 PM Post #6,009 of 17,336
I am not over-joyed at the superfluous negativity from both of you...


Who asked you? :p


Let me ask both of you to comment on the following.


Sure.


Occam's razor says among competing hypotheses that predict equally well, the one with the fewest assumptions should be selected (to test first).  How does each of your positions stack up against the razor?


Sorry, I don't buy Occam's razors. I belong to Dollar Shave Club. Their razors are less expensive and they have much better commercials. :D


As a practical matter, it's often the case that we test several hypotheses and often pick the easiest or least expensive or least time consuming first for obvious but illogical reasons.  My question to you both is:


Ok.


1.  what are all the reasonable available hypotheses that explain the facts available to us?


Given history, given what we know about the unreliability of human subjective perception, and what we know about the two amplifiers in question, I can't think of a reasonable hypothesis that explains all of the facts other than humans being human.

The only other hypothesis put forward has been a rolloff at high frequencies and second harmonic distortion.

However the amp in question has a spec'd F3 of 500 kHz. THD+N is spec'd at what, 0.01%? Second harmonic is pretty well masked at levels much higher than that.


2.  which has the least assumptions?


Humans being humans.


3.  How can we test it?


Properly controlled listening tests.

se
 
May 18, 2015 at 7:28 PM Post #6,010 of 17,336
   
Is there a universal standard that all instruments use? If you take 5 such devices for measuring the usual suspects do they all give the same value? First order, second order, third order harmonic distortion at 1000 hz (obviously measuring the 2nd harmonic of 12 Khz is outside of audibility) . Intermodulation distortion? Group delay? What else? Even vs. Odd harmonics? Square wave response? Ringing?
Are we measuring everything that can affect the quality of sound? 

 
This is something I have been working on very hard of late. The short answer is no. But it depends on how stuff is measured.
 
I had an antique sound technology 100A.  Absolutely filled with opamps.  made thd measurements completely analog
with complex filters to remove the fundamental. Even though it could go down to .001%, it always generated much
higher numbers than the digital things.
 
A while back lots of people were measuring stuff with audio sound cards that were 16 bit and getting numbers that
could not have possibly been real. (it should be obvious why)
 
But I am getting pretty close now, getting everything to agree.
 
1) spice synthesis of the circuit, with my own spice models for the parts generated by a network analyzer
 
2) QuantumAsylum 400  (basically a 24 bit sound card)
 
3) brand new Audio Precision system 2 cascade dual domain
 
I have now been able to get all 3 to agree to within about 1db which is as close as I ever hope to get.
 
The idea is that I can test out my new designs and be sure that when I actually build them, the numbers
will be close to the calculations. I can do 2nd harmonic vs frequency and 3rd harmonic vs frequency
calculations that are very close to the sweep numbers of the audio precision.
 
Getting pretty happy with this.
 
May 19, 2015 at 12:17 AM Post #6,011 of 17,336
This is something I have been working on very hard of late. The short answer is no. But it depends on how stuff is measured.

I had an antique sound technology 100A.  Absolutely filled with opamps.  made thd measurements completely analog
with complex filters to remove the fundamental. Even though it could go down to .001%, it always generated much
higher numbers than the digital things.

A while back lots of people were measuring stuff with audio sound cards that were 16 bit and getting numbers that
could not have possibly been real. (it should be obvious why)

But I am getting pretty close now, getting everything to agree.

1) spice synthesis of the circuit, with my own spice models for the parts generated by a network analyzer

2) QuantumAsylum 400  (basically a 24 bit sound card)

3) brand new Audio Precision system 2 cascade dual domain

I have now been able to get all 3 to agree to within about 1db which is as close as I ever hope to get.

The idea is that I can test out my new designs and be sure that when I actually build them, the numbers
will be close to the calculations. I can do 2nd harmonic vs frequency and 3rd harmonic vs frequency
calculations that are very close to the sweep numbers of the audio precision.

Getting pretty happy with this.

Hi. I'm not quite sure about paragraph 2. I think you are upgrading Opamps and testing the results with an old analog tester and a new digital tester?
 
May 19, 2015 at 12:25 AM Post #6,012 of 17,336
 
   
Hmmmmmm.  Let's see. I've actually compared how the Lyr and the Asgard sound - and I say they sound different. The manufacturer says they sound different. Virtually everyone I know who owns either, but has auditioned both, agrees that they sound different. Yet you, who don't even claim to have actually heard either one, "just know" that they "can't possibly" sound different based on the specs (which neither one of us has confirmed anyway).
 
That's really impressive.... but it's getting to be a bit boring.
 
Perhaps it's not really rolled off - and it's some other sort of "psychoacoustic effect" - or perhaps their specs are just wrong (I didn't measure it).

 That is a conundrum. How are the tools used to measure calibrated? In analytical biochemistry we have some standards we use to make sure that a measurement taking by person A on day one is the same (within excepted variation) with the measurement taken by B on another. And depending on how critical the sample is- possibly across labs.
Is there a universal standard that all instruments use? If you take 5 such devices for measuring the usual suspects do they all give the same value? First order, second order, third order harmonic distortion at 1000 hz (obviously measuring the 2nd harmonic of 12 Khz is outside of audibility) . Intermodulation distortion? Group delay? What else? Even vs. Odd harmonics? Square wave response? Ringing?
Are we measuring everything that can affect the quality of sound? 


I really wonder what the manufacturers use, I suspect(more like I hope) that big houses get anything they need, but what about all the smaller brands? some stuff are expensive and only useful for 1 kind of test. so I really wonder who bothers spending on those stuff and making sure they stay calibrated?
on my poor consumer level, I can't get to know the max power output@1%THD into the load of my headphones for a good 90% of the DAPs I'm interested in. meaning I just cannot know if the stuff I will buy exclusively to drive my headphone is able to drive it. pretty funky level of "you don't need to know" IMO. so obviously good luck making hypothesis about why a device sounds different.
 
May 19, 2015 at 12:53 AM Post #6,013 of 17,336
I think people spend WAY too much time on refining the measurements of sound their ears can't hear, instead of getting realistic measurements that relate to the sound they CAN her.
 
May 19, 2015 at 1:04 AM Post #6,014 of 17,336
I think people spend WAY too much time on refining the measurements of sound their ears can't hear, instead of getting realistic measurements that relate to the sound they CAN her.
You are the Bigshot- tell ME! Sorry could not resist. Did you ever post a picture of the said outfit? :)
 

Users who are viewing this thread

Back
Top