Head-Fi.org › Forums › Equipment Forums › Sound Science › Let's Prove The Null Hypothesis
New Posts  All Forums:Forum Nav:

Let's Prove The Null Hypothesis - Page 2

post #16 of 186
Thread Starter 
Quote:
Originally Posted by Uncle Erik View Post
#3: go back through the Stereophile archives. You will find about 30 years of objective amplifier testing that wasn't done by the manufacturers. They've been independently testing reviewed amps for many years. You might want to contact their staff for more information.
Hi UE

What is your recollection of the results of those tests?

USG
post #17 of 186
Thread Starter 
Quote:
Originally Posted by Publius View Post
You're looking for a noninferiority test.

noninferiority test - Google Search

I believe the common wisdom is that they're possible, but they require insane amounts of testing to achieve any degree of confidence. I believe the number I heard bandied about was thousands of trials.

Hey, it's only a headphone forum.......
post #18 of 186
Quote:
Originally Posted by upstateguy View Post

This $200 Pioneer is a solid state . . .

and these $12,000 Futterman monoblocks are tube amps.
But the proposition at issue is that all amps sound the same. The fact that one particular tube amp sounds the same as one particular SS amp (assuming arguendo that the test results prove that, and that is open to debate), doesn't prove that all amps sound the same, anymore than a test showing two amps sound different would prove that all amps sound different.

And most people, even the "skeptics," acknowledge that some amps sound different.
post #19 of 186
Quote:
Originally Posted by upstateguy View Post
What is your recollection of the results of those tests?
I don't want to answer for Uncle Eric, but I think you will find that the tests show the amps measure differently. Isn't that evidence that tends to support the proposition that amps can sound different? I'm not saying every difference in measurement translates to an audible difference, or that every amp that measures different sounds different to the human ear, but if there is a measurable difference, would you not concede there is the possibility of an audible difference?
post #20 of 186
Thread Starter 
Quote:
Originally Posted by PhilS View Post
But the proposition at issue is that all amps sound the same. The fact that one particular tube amp sounds the same as one particular SS amp (assuming arguendo that the test results prove that, and that is open to debate), doesn't prove that all amps sound the same, anymore than a test showing two amps sound different would prove that all amps sound different.

And most people, even the "skeptics," acknowledge that some amps sound different.
Do me a favor and read the entire David Clark stereophile amp test and then read the Richard Clark Amp Challenge. It won't take long.

Take from it what you will and we will be done.

USG
post #21 of 186
Thread Starter 
Quote:
Originally Posted by PhilS View Post

I don't want to answer for Uncle Eric...
That is something we can agree on.

USG
post #22 of 186
USG,

I don't think you're understanding my point. You're talking about statistical hypothesis testing, but you seem to miss the logic behind it.
The null hypothesis is what is currently believed to be true.
You don't prove it true. You can't prove something true..
What you do after stating the null hypothesis is determine what statistics you need to see to say that assuming the null is true those observations are statically improbable/impossible. If you see those observations then you reject the null.
In this case you might say I will randomly pair 100 amps together, and then have "golden ears" try to tell them apart. If they can only tell them apart less than 5% of the time then you reject the null.
Sorry, but that's how it works. Asking random people on an internet forum to prove something true is as pointless as it is philosophically misguided.
post #23 of 186
Meh. I'm all for double blind testing, to see if people can tell the difference between two systems. Perception isn't my field, and the perception guy in my dept. works in visual, not audio, but still, it's not controversial (among experimental psychologists, anyway) that an experiment can go a long way to deciding the matter. I don't really understand why some people are against DBT.

But let's not be naive.

We do not prove or disprove anything. Neither the null hypothesis, nor the alternative hypothesis. But what about Popperian falsification, you say? On his good days, even Popper would admit that falsification is not disproof in the strong sense of the word. After all, he was familiar with the Duhem-Quine thesis. Though at the time, it might just have been Duhem, really. For the uninitiated: Duhem–Quine thesis - Wikipedia, the free encyclopedia

So what does this mean? I think these kinds of tests proposed are worth doing, but they won't garner universal assent even if they are conducted with rigour. But - assuming methodological rigour - it would be terribly inconsistent of naysayers to trust other scientific results, but not this one.
post #24 of 186
I've closed the other thread, as this one covers the same bases, and is in the right forum:

http://www.head-fi.org/forums/f5/all...d-same-410314/ Figured I'd post a link here before it slips down into the abyss.
post #25 of 186
Thread Starter 
Quote:
Originally Posted by jonathanjong View Post

Meh. I'm all for double blind testing, to see if people can tell the difference between two systems. Perception isn't my field, and the perception guy in my dept. works in visual, not audio, but still, it's not controversial (among experimental psychologists, anyway) that an experiment can go a long way to deciding the matter. I don't really understand why some people are against DBT.

But let's not be naive.

We do not prove or disprove anything. Neither the null hypothesis, nor the alternative hypothesis. But what about Popperian falsification, you say? On his good days, even Popper would admit that falsification is not disproof in the strong sense of the word. After all, he was familiar with the Duhem-Quine thesis. Though at the time, it might just have been Duhem, really. For the uninitiated: Duhem–Quine thesis - Wikipedia, the free encyclopedia

So what does this mean? I think these kinds of tests proposed are worth doing, but they won't garner universal assent even if they are conducted with rigour. But - assuming methodological rigour - it would be terribly inconsistent of naysayers to trust other scientific results, but not this one.
Hi Jonathan

Silly stuff deleted.

USG
post #26 of 186
Ah, pardon the jargon. The proposition is that science neither "proves" nor "disproves." One problem with "proof" is that science (mostly) works by induction: Generalizations are made based on limited, specific observations; predictions are generated based on previous experience. But this is problematic. The fact that every swan we've ever seen is white does not entail that every swan is white. The fact that the sun has risen from east to west every day (as far as we know) does not entail that it will do this tomorrow. At least not in the strong sense implied by "proof." But then I've neglected to define proof. A proposition is proved to be true if it's negation is showed to be impossible.

What of disproof? You'd think this is possible. All we need to disprove the proposition "All swans are white" is to find a black swan. However, Pierre Duhem showed that it's really not so simple. Every experiment, every observation, every test involves assumptions. We assume that our eyes are functioning properly, etc. So, when we have a falsification event (e.g., an experiment that allegedly falsifies a hypothesis), we (almost?) never have to falsify the hypothesis in question. We could blame one of our assumptions. Perhaps we saw wrongly, and it wasn't a black swan, etc.

The history of science contains many cases, which illustrate both problems. I'll describe one to do with the second problem, from the phlogiston v. oxygen theory controversy during the Chemical Revolution. Without going into too much detail, Phlogiston Theory (PT) seems to imply that the calcination of a metal will lead its weight loss (as phlogiston leaves the metal). As it happens, calcination leads to a gain in weight! So, how did phlogiston theorists explain this phenomenon? Rather than abandoning their theory (which was a very good, very explanatorily powerful theory), theoriests rejected one or more of the following auxilary assumptions that must obtain for the prediction (viz., calcination -> weight loss) to obtain: 1. Phlogiston has positive weight; 2. Nothing is added to the metal during calcination as phlogiston (according to theory) leaves; 3. Processes involving something (positively) weighty leaving and nothing (positively) weighty entering necessarily leads to weight loss. Rejecting 1. and 3. seem silly, but the idea of massless (or weightless, anyway) entities are not ridiculous. Indeed, the idea of entities with negative weight are not all that ridiculous either. And back in the day, some people were ready to reject 3. on the basis on the weight difference between a balloon filled with air and an empty one. Put the former on a scale, and it weights less than the latter. Now we know that this test is flawed (ironically, some of its auxilary hypotheses are false. Most scientists rejected 2. Most people rejected 2., and came up with some pretty fascinating ways to do so.

Fast forward a few hundred years. The existence of sterile castes in eusocial insects were a great thorn in Darwin's side. Did this stop scientists from believing? No, and thank goodness not, because later genetic findings revealed that the existence of such castes was perfectly Darwinian. Too tired to rehash the entire story here, I'm afraid...

Edit: Gah, I think I went too far into tutor mode again...
post #27 of 186
Quote:
However, there is also no quantitative data that cables sound the same.
Sure there is... the simple fact that no one can tell them apart in blind tests.

It's really very easy folks... either you can tell the difference under controlled conditions (meaning those designed to eliminate one's imagination from the process), or you can't. Everything else is psychobabble.
post #28 of 186
Is anyone aware of experiments that have attempted to measure and compare directly the impulses along the aural nerves to see if different cables/amps/pebbles/potplants result in different aural stimuli being presented to the brain? This, to me, would be far, far more convincing than purely subjective (DBT or not) for about a million reasons.
post #29 of 186
Quote:
Originally Posted by b0dhi View Post
Is anyone aware of experiments that have attempted to measure and compare directly the impulses along the aural nerves to see if different cables/amps/pebbles/potplants result in different aural stimuli being presented to the brain? This, to me, would be far, far more convincing than purely subjective (DBT or not) for about a million reasons.
Every DBT experiment does that. If there are differences in impulses along the aural nerves that the brain can detect then the test subject reports it... and if the brain can't detect it then it's irrelevant.
post #30 of 186
Think there're quit some tests where they show the measurable differences between cables/amps (for example: 1st hit on google: Speaker Cable Reviews - Faceoff 2 — Reviews and News from Audioholics). Based on that you could say there are differences, rejecting H0 (assuming H0 = no difference and H1 = difference).

However, do you want to look at the difference in numbers or tests on people to see if they can hear the difference?

I think there are some articles written about the 2nd involving DBT's, but most of them cant be easily accessed (at least the more serious ones) :<

Anyway, not an expert on this
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › Let's Prove The Null Hypothesis