[stewtheking]

Quote:

Originally Posted by Publius /img/forum/go_quote.gif Tell ya what. Why don't you have the new objectivist forum merely be a link to HydrogenAudio? Everybody who wants to discuss their latest fuse-rolling exploits can still post here. Everybody who wants their audio questions answered correctly can head over there.

It's this kind of ridiculous statement that inflames the argument further. If you don't have something more intelligent to add, why fan the flames of an already rather tense thread?

 

[digger945]

Quote:

Originally Posted by upstateguy /img/forum/go_quote.gif It seems to me that the scientific approach of accurate measurement and testing should easily override hearsay, opinion and the recollections of dim auditory memory. USG

I agree!

 

[Publius]

Sorry, sorry. I couldn't resist.

Still, the question presents itself: there's already a forum out there that does the whole skeptic thing really well. It also prohibits subjective discussion.

There is another forum out there called "Propellerhead Plaza" on Audio Asylum. It allows all manners of discussion. It's also a troll-infested hellhole, and nothing of substance ever gets discussed there nowadays, simply because the bickering is so intense.

I'd argue that for this sort of forum to ever work out at all, the mods are going to have to go down the HA route rather than the AA route. That is, you'll need to set up a different moderation system entirely for that one forum. Otherwise bigshot's fears will be mostly realized.

 

[upstateguy]

Hey Guys

I would like to make a suggestion.

Try not to answer the trolls at all.

You know who they are and the futility of engaging them.

You are but one, and the cyber-bullies are many.

My advice is to Ignore Them.

USG

 

[MD1032]

Statement withdrawn.

 

[acidbasement]

Quote:

Originally Posted by JadeEast /img/forum/go_quote.gif I really liked this post and would love if you could expand on this particular statement. I am wondering if you are speaking about Popper's falsification.

Yes, I was referring to falsification. Falsifiability - Wikipedia, the free encyclopedia

In the context of this discussion, one cannot prove by performing blind tests that two pieces of equipment are functionally identical. The closest thing would be to show that there is no statistically significant difference between actual blind test results and random or uniform results. If there is only a small group of people taking part in the blind test (a likely scenario at most audio meets given the niche/cult nature of the hobby), statistical power - our ability to detect a significant difference between actual test results and random/uniform results - may very well be too low to make a negative result meaningful in the slightest. For this reason, I would not trust anyone who tried to quash my curiosity about a product by simply citing a DBT, especially if they didn't know any details about the DBT aside from the result.

Now this has me thinking. Would anyone who is stats-inclined (wavoman perhaps?) have time to do a power analysis to show how large a group of test subjects we would need to have 80% statistical power to detect a significant difference between two pieces of equipment?

This thread is indeed fun if you ignore the trolls.

 

[wavoman]

Quote:

Originally Posted by Zanth /img/forum/go_quote.gif ...During a DBT, one might feel uncomfortable, nearly on trial, being tested to "hear" or "not hear" certain things. In the comfort of one's own home, at one's leisure, if one is able to listen at different times of the day, then differences may be more readily apparent...

My proposals specifically avoid this problem. The subject (the only one blinded) is a Head-Fi member, and we are at a meet, A/B testing two alternative pieces of equipment (cables, amps, whatever) that he does not own and has no vested interest it. After careful balancing of levels, he listens to pairs of the same musical passage, and declares "No Difference" / "Difference, but no Preference" / "Pefer A to B" / Prefer "B to A".

Many pairs are played for him, and the answers recorded. We repeat this with different passages, and different HPs if we are not testing HPs. And many different people.

The experiment turns on the tricks: ringers are introduced, some comparisons are false on purpose, etc. The idea is these trials benchmark the subjects ability to listen objectively in the setting at hand.

The results are recorded without the subject's name. No results are announced until after. No one is embarrased, since we have tested enough different people to make the results anonymous. No one is on trial.

It would actually be great fun and we would learn things.

 

[wavoman]

Quote:

Originally Posted by acidbasement /img/forum/go_quote.gif Science is generally based on presence/absence of statistical significance given variability in responses displayed by a group of experimental subjects. Audio, and especially headphone audio, is an individual thing, and when n=1 all the statistics in the world based on a perfectly randomly drawn group of test subjects with a whole bunch of controls don't change what you experience personally...

Yes, but "sample size one" experiments done over a universe and pooled correctly (not simply averaged) are perfectly valid. Each person acts as a "block", to use the correct statistical term (if you are a statistician, think "paried t-test" vs "regular t-test"). As long as each subject is given a pair, we can do plenty of stats even though there is tremendous variability across subjects. (Look up "sample size one" in your books).

 

[wavoman]

Quote:

Originally Posted by acidbasement /img/forum/go_quote.gif ...statistical power - our ability to detect a significant difference between actual test results and random/uniform results - may very well be too low to make a negative result meaningful in the slightest.... Now this has me thinking. Would anyone who is stats-inclined (wavoman perhaps?) have time to do a power analysis to show how large a group of test subjects we would need to have 80% statistical power to detect a significant difference between two pieces of equipment?

Well you have to make a bunch of assumptions about the magnitute of the difference and the chance that a random person hearing it actually detects it, but I ran lots of values and, very roughly, to have 80% power, i.e., 80% chance of detecting the difference with 95% certainty that it really IS a difference, you need about 15 subjects.

Allocating 30 minutes to each test, we can do this in one full one day at a meet, taking 15 volunteers who feel they have "golden ears". I would love to take a top-of-the-line digital system, and play the same passage using MP3's of various bit rates, then full redbook 16/44.1, then 24/192 LPCM. This mimics a test the pros did and published. They claimed no one could hear the difference between redbook and hi-res LPCM.

I bet we can. Would love to know.

 

[acidbasement]

^ That was quick! Good on ya.

 

[oicdn]

I noticed that forum "coming soon" is no longer there...

 

[bigshot]

The problem here has nothing to do with objectivism and subjectivism. It has to do with people's reactions to being challenged in their beliefs. Some people respond with a reasoned response to support their position. Some people get mad and lash out with insults.

The solution to this problem is to encourage discussion and discourage ad hominem attacks. Certain folks should work on developing a sense of humor too.

I really don't have much interest in discussing the ins and outs of conducting double blind tests. There are plenty of tests and studies out there already. We don't need to conduct them ourselves. What we need to do is focus on sharing the most efficient and effective ways to achieve great sound. That's what we are all here for.

See ya
Steve

 

[oicdn]

Quote:

Originally Posted by bigshot /img/forum/go_quote.gif The problem here has nothing to do with objectivism and subjectivism. It has to do with people's reactions to being challenged in their beliefs. Some people respond with a reasoned response to support their position. Some people get mad and lash out with insults. The solution to this problem is to encourage discussion and discourage ad hominem attacks. Certain folks should work on developing a sense of humor too. I really don't have much interest in discussing the ins and outs of conducting double blind tests. There are plenty of tests and studies out there already. We don't need to conduct them ourselves. What we need to do is focus on sharing the most efficient and effective ways to achieve great sound. That's what we are all here for. See ya Steve

Probably one of the best posts here!!!

 

[upstateguy]

Quote:

Originally Posted by bigshot /img/forum/go_quote.gif The problem here has nothing to do with objectivism and subjectivism. It has to do with people's reactions to being challenged in their beliefs. Some people respond with a reasoned response to support their position. Some people get mad and lash out with insults. The solution to this problem is to encourage discussion and discourage ad hominem attacks. Certain folks should work on developing a sense of humor too. I really don't have much interest in discussing the ins and outs of conducting double blind tests. There are plenty of tests and studies out there already. We don't need to conduct them ourselves. What we need to do is focus on sharing the most efficient and effective ways to achieve great sound. That's what we are all here for. See ya Steve

 

[upstateguy]

Quote:

Originally Posted by bigshot /img/forum/go_quote.gif The problem here has nothing to do with objectivism and subjectivism. It has to do with people's reactions to being challenged in their beliefs. Some people respond with a reasoned response to support their position. Some people get mad and lash out with insults. The solution to this problem is to encourage discussion and discourage ad hominem attacks. Certain folks should work on developing a sense of humor too. I really don't have much interest in discussing the ins and outs of conducting double blind tests. There are plenty of tests and studies out there already. We don't need to conduct them ourselves. What we need to do is focus on sharing the most efficient and effective ways to achieve great sound. That's what we are all here for. See ya Steve

Just a thought...

While you are absolutely right that there are plenty of tests and studies out there, conducting some of them ourselves, like Wavoman wants to do, moves the prevailing views of our fact based forum into the Verifiable and Reproducible arena.

Really glad to have you aboard.

USG