Need help with a test involving cables

Design and run an experiment by the end of the day Tuesday? Yow!

The course is in experimental design, so if the teacher is any good a tight design is going to be more important than execution, particularly in a limited time frame.

So, step 1 is to frame your experimental hypothesis very concretely. Do you want to look at the influence of cables? sources? amps? headphones? Each variable you add increases both the complexity of the experiment, and the number of subjects you'll need to do it right.

Your instructor is absolutely right that multiple subjects are critical. Remember that your subjects are supposedly representative of a larger population. The more subjects you have, the stronger your statistical analysis will be, up to a point.

Why do you need to have people who you believe will hear cable differences? That's actually biasing your sample. Grab anyone who will participate.

DON'T make anyone try to remember the sounds of four or eight different setups, or even try to identify them. You're pushing audio memory past its limits. How about a forced choice type of design? The subject doesn't know what system he's hearing. One each trial, the same piece of music is played twice. The subject has to decide which setup he likes best (he's blinded, although the experimenter isn't), or if they sound the same. So, the subject has three choices: A>B, A<B, A=B. You can try to have your subject assign ratings, rather than a simple choice. You're going to need to repeat various pairings to test reliability of the rating, but you need to test repeatablility of the choice in any event.

Throw in some dummy trials so that your subject really is listening to the same system on both trials (this is an important control). Run through a randomized sequence of trials, and see if a systematic preference emerges from the series of choices.

Does this help any?

Got it. It makes a difference when you've expressed the hypothesis that way. So, your dependent variable is going to be the ability to detect cable differences. You're going to use two different independent variables: test system and cables. You a priori prediction would be that cable differences would be indetectable in the cheap system, but detectable in the good system.

Be careful analyzing a small N! You're going to need to insure that you've got enough subjects for the analysis to be valid. Multiple trials within a single subject has its limits.

Yes, that's good. As long as what you're looking at is the ability to detect differences, rather than which one is better, there's no need for any kind of ranking.

Yes, but bear in mind that your real comparison is going to be between the cables. If the systems aren't exactly level matched, you've got a confound: are cable differences more apparent in louder systems than quieter ones? But this may not be serious. The key is going to be to set the volumes of the systems as close as you can, and then not touch the volume levels of the systems until the experiment is done. If you can borrow a Rat Shack SPL meter, you can approximate db, but bear in mind that actual volume levels are going to depend on each subject's ears. It's going to be something you can't measure anyway.

I'd run it as a 2x2. Good system/Bad system (well, less good system, anyway.) Good cable/bad cable. Some trials you change the system, some trials you change the cable, some trials you don't change anything.

Note that you're already doing a more complex experiment than one that would normally be used to detect cable differences.

You may want to try looking at this with a signal detection analysis, rather than percentage correct. You would calculate, for each condition, the number of correct hits (you change the cable and the subject identifies it), false alarms (subject says you changed the cable when you didn't), correct rejections (subject says cables are the same when they are in fact the same), and misses (subject says the cables are the same when in fact you changed them). In this type of analysis, you can pick up subject bias. A real perceptual change would show up as a change in hits and correct rejections (or misses and false alarms). If those parameters change together, you've got a perceptual change. If hits and false alarms change together (as would misses and correct rejections), that would show a change in subject bias rather than perceptual change.