[Ferari550]

need help! got science-fair project do 1 month!
project is "SACD vs. CD" i need help!
using connected to sony's 5-disc SCD-CE 775 SACD player. i've got 6 SACD titles - all hybrids
1) Train-Drops of Jupiter
2) Berlioz-Cincinnati symphony
3) Turina- Cincinnati symphony
4) The Rolling Stones-12x5
5) Dvorak Smetana-Cleveland symphony
6) Brahms- no.4 Columbia symphony

problem: on some like the "Berlioz" I can barely hear a difference. my dad was amazed at the sound, he said "WOW" and then I said "this is CD!". I asked myself, How is the average test subject going to tell a difference?
My teacher said I'll get screwed by the judges unless
I test only one sex (women cause they're easier to get)
and you can't blame the judges for being hard but I can only have 1, only 1, no more than 1 variable. or I will lose!

NOW HERE IS MY QUESTION:
How do I carry out this project correctly? My dad goes to this site. He says, If anyone would know how to design a test, you guys would.

 

[aeberbach]

What is your aim? I don't understand whether you are trying to prove that there is a huge difference or no difference.
If you are selecting judges yourself try to get younger ones, younger people tend to have more acute hearing.

 

[Onix]

Okay, let's see. And I am talking as a journalist, and this is the way I would conduct such a test.

- First, be sure you know what do you want to demonstrate. I am assuming you want to probe that SACD is superior to CD, but remember that it could be proven on subjective values like musicality or pleasantness of sound, or objective values like resolution and reproduction quality. For that, I think you should be able to play each layer separately on the same player, or get a CD player of at least as good quality as the SACD, so you can play the CD layer. If you can afford it, it would be even better if you can get those releases on normal CD format.

- Second, define what you are going to ask to each one of your subjects. I would part from the notions exposed above, like this:

- What sound do you like the most?
- Why?
- Which one you think is more detailed?
- Would you get this kid of hardware/software
- Why yes, why not?

On this regard, I would have at least 3 groups of subjects (women are a great choice, since they have better hearing than men), sorted like this:

- I would play either CD or SACD to one group, not telling them what format they are listening to, and then switch to the same track using the other format. The I would ask them which one sounded better to them.

To another group, I woud tell them what format they are listening to, and then switch to the other, letting them know what format they are listening.

And then, I would have another group, and I would LIE to them, playing a CD and telling them it's a SACD and viceversa. That way, you could filter out the "wow" factor from the previous group.

Define the envinroment for this test: Are you going to use headphones, amper or unamped, or speakers, for example. Think on terms of isolation, quality of headphones and speakers, and try to use the same equipment on each subject, one by one. Also, play the SACD on stereo, not multichannel, so it won't have an "unfair advantage" over CD. That would make mounting the speakers, if you want to use them, way easier.

Keep in mind that your subjects should be educated on the differences between the two formats, in terms of hardware, availabitity, portability (this is important, as portable SACD players are not available), price, resolution, decoding and all that stuff. You can prepare either a text or a small lecture on the subject.

After that, I would bassically compare the porcentages of people who liked each format and use them to support or disprove the initial hypothesis.

Now, keep in mind all this is just a very general idea on how I would do it. The process would have to be refined by you, according to the subjects you would use, your preparation skills and many other factors. Besides, I haven´t done anything similar for some time, so perhaps I am being vague. Anyway, I hope I helped you at least a little.

By the way, who's your daddy?

 

[Ferari550]

sorry, my aim is to disover "is SACD really that good?" I dont care if the subjects say CD sounds better. I just want to conduct a fair correct study. I would greatly appreciate your knowlege in my question. My dad is kevinsinnott he's an audiophile he rarely leaves messages but reads here all the time.

 

[x1lexure]

Do some Blind testing with them. Put the CD or SACD in randomly and have them decide which is better or which they believe is the SACD. Try and get atleast 7 people to judge. But that's probably what you had planned anyways.

 

[yage]

I would have to argue in favor of *not* informing your test subjects which is the technically "superior" format. Tell them nothing. Informing them beforehand would either confuse them or bias them towards what they think sounds better. Just ask if they perceive a difference when you switch between sources and which one they liked better.

To conduct double-blind tests, (which, IMHO, is the fairest test) both the person switching the sources and the person listening to the sources do not know which is CD and which is SACD. So you'll have to have someone else set up everything and you'll just be switching back and forth between SACD and CD. If you have a single player and can switch between the two with a push of a button, this part is relatively easy.

Make sure, though, that the output levels between SACD and CD are matched.

Hope this helped and good luck with your project!

 

[kerelybonto]

Grr -- my first ever double post.

See below.

kerely

 

[kerelybonto]

The main problem with doing this sort of test is that it's completely subjective. In essense, regardless of what difference your study participants hear, you're asking them which they prefer. If you're in fact trying to find out which is subjectively better, well okay then -- but I assume you're trying to figure out which is objectively better, i.e., closer to the true representation of the recording (as mixed and mastered). Unfortunately there's no way to do that ... the closest you could get would be to have your subjects compare CD and SACD audio to a reference anaolg setup, but that's probably not at all practical.

So let's assume you're just trying to find out which is subjectively better. The problems here are that 'better' means different things to different people. So the only valid conclusion you can draw from your test is that more people prefer one format to the other. It says nothing about which is 'better,' even subjectively. That said, there's nothing wrong with doing a test to determine which format people prefer -- just make sure you state that that is what you're attempting.

But there are still problems with preference audio tests -- even if people can hear differences, they may not know (or care) which they prefer. Imagine someone coming up to you, showing you two shades of green, and asking which you prefer. Even if you give him an answer, it doesn't really mean anything. This guy's results will probably approach randomness. Basically, in a study like this, you're not controlling for preference. To do that in an audio test, obviously people with a lot of experience playing or listening to live music (and presumably know what things should sound like) and "audiophiles" (who also presumably know what things should sound like -- questionable, I know) make better test subjects.

Make sure you know the proper methodology for double-blind testing, too. Or at least avoid the major flaws.

kerely

 

[yage]

hmmm...

yes kerely i suppose you're right...

i guess if you really wanted to test whether SACD or CD was more faithful to the original signal, one would have to set up something to be recorded using both DSD and PCM technology, then run some sort of frequency/time domain analysis on the original, SACD, and CD signals.

of course you get into the whole thing about what signals to record, etc. etc...

still, i say the subjective listening test project would be pretty cool. i really do wonder if regular people with no "audiophile" experience whatsoever could detect a difference between SACD and CD, or if they prefer one over the other.

 

[kerelybonto]

Yes, a subjective 'preference' test would still be very worthwhile. I'd be interesting in hearing whether people who don't normally care about such things think SACD sounds 'better' without any prompting.

kerely

 

[Ferari550]

I thank you all for your posts, i will ask them if they "hear a difference" and for personal fun i will ask them which they perfer or sounds "better." And i like the idea of not informing them on which format is better. if you have any more ideas or corrections please send a post. MY GOAL IS 20 FEMALE SUBJECTS, preferably younger.
P.S. I will report my results when I'm done testing, you will be the first people to know!!!

 

[kerelybonto]

Good luck with your experiment. I look forward to seeing what you find out.

kerely

 

[Joe Bloggs]

If you want to test whether they hear a difference the best method is an ABX test.

Play the CD, tell them they're listening to A; play the SACD, tell them they're listening to B; then randomly choose between the two (and record your choice of course) and ask them which do they think they're hearing, A or B.

Then perform some statistical analysis to see what is the chance that they are purely guessing given that they got x correct choices out of y trials

 

[dgs]

Ferari,

I would suggest that you restructure your premise a bit. It is much harder to demonstrate a positive hypothesis than it is to disprove a negative one. In other words, to demonstrate that there is a significant difference between SACD and CD can be quite difficult and plagued by a number of what are called "confounding variables." It would be much easier to do the study this way:

There is no audible difference between CD and SACD.

Then, try to DISprove the hypothesis. You take a group of people, play both sources for them (randomnly), and ask them if they hear a difference. Ideally, you would want to go one step further and not tell them when you are switching sources--if you had an inaudible way to do this, you could ask them to identify WHEN the source changes. If a percentage of people can tell the difference, then you have shown that there is likely a difference in sound quality between the two.

A word to the wise: when people are given choices like this, they almost invariably will prefer one or the other, EVEN IF BOTH CHOICES ARE SECRETLY KEPT THE SAME. This has been demonstrated multiple times using the Coke/Pepsi/RC Cola test. You ask people to choose which they prefer, coke or pepsi. You secretly put RC cola in both glasses. Most people will choose one over the other.

This is a tough topic to choose, but if you are determined to do it, you could also do it like this:

Make a survey that the examinee fills out while they listen to the music on each source. Play the music first on one source, then the other, playing the same music on each source. The survey could ask questions like: Does this music sound more lifelike to you? Do the drums sound like you imagine they should? and so on. This way, you can avoid the problem of not having a "gold standard" to which they would have to be comparing the sound. Just make sure that for half the people you play the SACD first and for the other half you play the CD first.

Good luck.

 

[kevinsinnott]

Hi,

Ferrari is my son. We are father and son audiophiles. I have two other sons but they don't even appreciate tubes!

Last night we tested four audiophile salespeople. Each of them prefered SACD during two of three samples. The test was structured to determine preference, if any, not whether SACD was better, per se. It was interesting to note that the preferences were not identical, in fact they were equally diverse.

Another interesting note: The audiophile sales group seemed to have done almost no previous comparing. One comment was, "it's not vinyl, I know there will be less information. It's just another format."

They had SACD discs for sale, but in their upscale room, no SACD player. We brought our own Sony unit.

Selections:
Train, Track 10 (redbook cd/SACD)
Rolling Stones (dual layer)
Cobos Conducting Berlioz Fantastic Symphony (dual layer)

The next day, he began testing at his school, all female. One mom prefered SACD in each instance. I was not present by Ferrari said she picked instantly and matter--f-factly.

We've taken Head-fi'er's advice and mixed order in each case.

More later.