[johncarm]

  as opposed to the higher theoretical limitations. For example, the ultimate high frequency limit of human hearing is not defined by blind testing, it is defined by the anatomical and physiological limits of the human ear. Where blind testing is useful is in identifying a generalised maximum limit in practise, rather than the anatomical/physiological limit which is only a theoretical maximum (and unachievable in practise).
 

 
I can't respond to all of this, but let me focus on the main question. We've got two devices, A and B. We want to determine if there is an audible difference. There are some situations where you would want to do blind listening tests, right?

 

[Pio2001]

   
The question is whether you can hear any difference between device A and device B with one second of sound, no matter what signal you pick for that one second. Is that what you are claiming?

 
 
Oh no, certainly not. The sample must be carefully chosen. For mp3 vs wav, for example, there may be no audible differences for minutes, and then suddenly, a difference appears just at a given time, because the sound of an instrument that plays just here is badly encoded.
An extreme example : clipping. It can happen just at one point in a whole album. There won't be any clipping if any other part is chosen to perform the comparison.
 
For my part, in many cases, I'd prefer having 10 to 20 seconds of audio to perform an ABX test, but it's better if I can choose these 10 to 20 seconds at will from a given set of tracks.

 

[johncarm]

   
 
Oh no, certainly not. The sample must be carefully chosen. For mp3 vs wav, for example, there may be no audible differences for minutes, and then suddenly, a difference appears just at a given time, because the sound of an instrument that plays just here is badly encoded.
An extreme example : clipping. It can happen just at one point in a whole album. There won't be any clipping if any other part is chosen to perform the comparison.
 
For my part, in many cases, I'd prefer having 10 to 20 seconds of audio to perform an ABX test, but it's better if I can choose these 10 to 20 seconds at will from a given set of tracks.

 
Let me point out that the main reason you need time in an audition is that one's perception of music changes over time. This is the so-called "test of time" on the largest scale, years or decades or centuries. But it also happens on a smaller time scale, say seconds or minutes. Hearing a pattern, and then hearing it repeated a few moments later gives you a window into how that pattern is experienced.
 
You also point out that an audition can contain different musical passages which show different aspects of technique. That is true as well. It doesn't seem all that different in audio from what you are saying. If you want to hear the difference between A and B, you want to pick something (i.e. a passage) that reveals what you need to hear.
 
If there is more than one difference between A and B, you need more than one passage.
 
Now, here is a critical question. Let's say we have A and B and want to check for audible differences. We need to pick the right passages. But how do you pick the passages if you don't know yet what the difference between A and B is?

 

[johncarm]

By the way I'm still waiting for an answer to this question:
 
If we call the set of differences audible in a comparison via echoic memory C1, and those audible in any context C2, has anyone done work to establish that C1 equals C2?

 

[castleofargh]

 

   
 
Oh no, certainly not. The sample must be carefully chosen. For mp3 vs wav, for example, there may be no audible differences for minutes, and then suddenly, a difference appears just at a given time, because the sound of an instrument that plays just here is badly encoded.
An extreme example : clipping. It can happen just at one point in a whole album. There won't be any clipping if any other part is chosen to perform the comparison.
 
For my part, in many cases, I'd prefer having 10 to 20 seconds of audio to perform an ABX test, but it's better if I can choose these 10 to 20 seconds at will from a given set of tracks.

 
Let me point out that the main reason you need time in an audition is that one's perception of music changes over time. This is the so-called "test of time" on the largest scale, years or decades or centuries. But it also happens on a smaller time scale, say seconds or minutes. Hearing a pattern, and then hearing it repeated a few moments later gives you a window into how that pattern is experienced.
 
You also point out that an audition can contain different musical passages which show different aspects of technique. That is true as well. It doesn't seem all that different in audio from what you are saying. If you want to hear the difference between A and B, you want to pick something (i.e. a passage) that reveals what you need to hear.
 
If there is more than one difference between A and B, you need more than one passage.
 
Now, here is a critical question. Let's say we have A and B and want to check for audible differences. We need to pick the right passages. But how do you pick the passages if you don't know yet what the difference between A and B is?

it is relatively trivial to detect differences by measurement. anytime you detect something that could be of interest for some reason, you can isolate that passage and test it.

  By the way I'm still waiting for an answer to this question:
 
If we call the set of differences audible in a comparison via echoic memory C1, and those audible in any context C2, has anyone done work to establish that C1 equals C2?

my bad. I thought that was answered with my poor example of seeing or not the interpret. no hope in my opinion, because anything introduced in a test has potential to alter said test. so a different context can certainly change a lot of things, thus our obsession with isolating the tested part from everything. but there is nothing wrong with trying different testing methods and see if one seems to give better results, and then question why. it's a very open process IMO where we never stop to look for variables, even those inadvertently introduced by the testing method itself.

 

[gregorio]

  I can't respond to all of this, but let me focus on the main question. We've got two devices, A and B. We want to determine if there is an audible difference. There are some situations where you would want to do blind listening tests, right?

 
No, from a scientific point of view there are few/no situations where one would want to do a blind test! Blind tests are often very difficult to set up without flaws, flaws which can end up with variables other than the desired devices/variables being tested, not to mention that even a flawless blind test would still be subject to potential statistical aberrations. Scientifically, blind tests are therefore never absolutely conclusive, they provide supporting evidence rather than proof. If we've got two devices, A & B and want to determine if there is an audible difference, the first thing I personally would probably do is conduct a null test. If they null, end of discussion, we have absolute proof there is no difference period, let alone an audible difference. If they don't null, what we are left with is the difference: If that difference is below say -100dB we can safely say it's inaudible, as it will be several/many times below the threshold of audibility. If that difference is higher, it might be audible, depending on how much higher and where in the frequency spectrum the differences are. In which case, we might be forced to use a blind test because despite the fact they have potential flaws, they have far fewer flaws than the remaining alternatives (sighted tests or anecdotal evidence for example).
 
This scientific approach is of course invalid when evaluating a musician, we take it as given that there will always be differences (within the range of audibility) between musicians, even a musician deliberately trying to copy another musician and even between different performances by the same musician. When evaluating a musician we are not therefore trying to find out if there are any differences but identifying what the differences are and making qualitative judgements about them (relative to some abstract/subjective concept).
 
G 

 

[Pio2001]

Now, here is a critical question. Let's say we have A and B and want to check for audible differences. We need to pick the right passages. But how do you pick the passages if you don't know yet what the difference between A and B is?

 
You can't. You have to learn the difference before finding the right passages. It can take time.
But this is only the case if you don't know the difference. It means that you are in a situation where you hear no difference immediately, but you suppose there may be one.
Quite uncommon, but it can happen. For example when you compare a recording of a vinyl made with your current worn stylus vs a brand new one (mounted on the same cartridge). How to be sure that there will be a difference and what should it sound like ? In this case, yes, you'll have to take the time to listen to many records before you find something clear. Only then you can choose the right sample.
 
Quote:

  By the way I'm still waiting for an answer to this question:
 
If we call the set of differences audible in a comparison via echoic memory C1, and those audible in any context C2, has anyone done work to establish that C1 equals C2?

 
By "echoic memory" you mean the short term memory used in fast switching ABX tests ?
It seems normal that Short term memory > Long term memory, otherwise it would mean that memory doesn't decrease with time, or, more surprisingly, that a given memory becomes more faithful as time passes !
 
Here you are refering to the observation that subtle differences may be better assessed with more time. That's not exactly the same thing. I don't think that such a thing has been investigated in itself. It seems more logical just to proceed with enough time when there is need to.
If you setup a blind listening test aimed at detecting small differences, to arbitrary decide of a musical sample, its duration and to play it only once per trial should lead to very poor results in difference recognition. Let the listeners train, choose their samples, and let them listen to them as many times as they want, and the test should provide much more information, both in case of success (would have probably failed otherwise) or failure to recognize the difference (much less possible explanations).

 

[johncarm]

   
No, from a scientific point of view there are few/no situations where one would want to do a blind test! Blind tests are often very difficult to set up without flaws, flaws which can end up with variables other than the desired devices/variables being tested, not to mention that even a flawless blind test would still be subject to potential statistical aberrations. Scientifically, blind tests are therefore never absolutely conclusive, they provide supporting evidence rather than proof. If we've got two devices, A & B and want to determine if there is an audible difference, the first thing I personally would probably do is conduct a null test. If they null, end of discussion, we have absolute proof there is no difference period, let alone an audible difference. If they don't null, what we are left with is the difference: If that difference is below say -100dB we can safely say it's inaudible, as it will be several/many times below the threshold of audibility. If that difference is higher, it might be audible, depending on how much higher and where in the frequency spectrum the differences are. In which case, we might be forced to use a blind test because despite the fact they have potential flaws, they have far fewer flaws than the remaining alternatives (sighted tests or anecdotal evidence for example).
 
This scientific approach is of course invalid when evaluating a musician, we take it as given that there will always be differences (within the range of audibility) between musicians, even a musician deliberately trying to copy another musician and even between different performances by the same musician. When evaluating a musician we are not therefore trying to find out if there are any differences but identifying what the differences are and making qualitative judgements about them (relative to some abstract/subjective concept).
 
G 

 
By "want to do a blind test" I mean you "want" to know the truth, so you "want" to use whatever means is necessary to find that out.
 
It sounds like you are referring to the notion that we have a model of what differences the ear/brain can hear, correct? So in some cases you wouldn't need to do a test, but first measure A and B, and then refer to this model, correct?
 
Let's say "listening tests" doesn't include just blind ABX, but any kind of experiment conducted to determine the limits of audibility. This model of the ear/brain you are referring to in order to determine audibility---this model was developed with the assistance of listening tests, right? I am assuming the only two methods of knowledge to develop such a model are (1) investigating the biology of the ear and the nervous system, (2) conducting listening tests.
 
Let's say we have two amplifiers, and we want to determine if the difference between them is audible. What measurements would you run to determine if you need to do an actual ABX test? What results would convince you no ABX test is necessary?

 

[johncarm]

   
By "echoic memory" you mean the short term memory used in fast switching ABX tests ?
It seems normal that Short term memory > Long term memory, otherwise it would mean that memory doesn't decrease with time, or, more surprisingly, that a given memory becomes more faithful as time passes !
 
Here you are refering to the observation that subtle differences may be better assessed with more time. That's not exactly the same thing. I don't think that such a thing has been investigated in itself. It seems more logical just to proceed with enough time when there is need to.
If you setup a blind listening test aimed at detecting small differences, to arbitrary decide of a musical sample, its duration and to play it only once per trial should lead to very poor results in difference recognition. Let the listeners train, choose their samples, and let them listen to them as many times as they want, and the test should provide much more information, both in case of success (would have probably failed otherwise) or failure to recognize the difference (much less possible explanations).

 
Well let me put it this way. When you listen to signal A, a bunch of information is registered by your nervous system. When you then listen to signal B a very short time later, a bunch more information is registered. The nervous system has a way of making a comparison between A and B on the basis of information that has been registered. There are circuits in the nervous system that make that comparison.
 
It's like you have two pictures in front of you, which are almost the same but not quite, and you scan your eye back and forth, looking for differences. This is not the same thing, but similar.
 
It might seem that all the differences are "out in the open". I mean that you can easily feel confident that if there is any difference between A and B, you would hear it, because you feel like you are experiencing the totality of the signal. But actually that is something that needs to be proven about the nervous system. You would need to prove that all of the important information (*) in signal A is available in the nervous system to compare against all of the important information in signal B.
 
(*) I say "important information" because obviously not all information is even registered by the ear or brain. So we are assuming there are important features of the sound that affect how you experience it in other contexts, such as listening to music.
 
So has there been any work done to establish what features of the sound are "available" in the echoic memory for the "comparison function"?

 

[johncarm]

  it is relatively trivial to detect differences by measurement. anytime you detect something that could be of interest for some reason, you can isolate that passage and test it.
my bad. I thought that was answered with my poor example of seeing or not the interpret. no hope in my opinion, because anything introduced in a test has potential to alter said test. so a different context can certainly change a lot of things, thus our obsession with isolating the tested part from everything. but there is nothing wrong with trying different testing methods and see if one seems to give better results, and then question why. it's a very open process IMO where we never stop to look for variables, even those inadvertently introduced by the testing method itself.

Not quite able to parse your grammar. Are you saying that you don't believe C1 equals C2?

 

[gregorio]

  By "want to do a blind test" I mean you "want" to know the truth, so you "want" to use whatever means is necessary to find that out.

 
A blind test does not give you the truth! At best it gives you the truth as perceived by one individual or in the case of a sample size larger than one person, some non-perfect statistical probability of the truth.
 

  It sounds like you are referring to the notion that we have a model of what differences the ear/brain can hear, correct?

 
No, we have a well understood science (model) of sound and a quite well understood model of the ear but when we bring the brain into the equation and effectively talk about the perception of sound rather than what our ears actually hear, then there are no models or rather, there are lots of partial and/or different models, the vast majority of which are inaccurate or only accurate in certain circumstances.
 

  So in some cases you wouldn't need to do a test, but first measure A and B, and then refer to this model, correct?

 
Not necessarily, did you read my last post?
 

 
Let's say "listening tests" doesn't include just blind ABX, but any kind of experiment conducted to determine the limits of audibility.

 
If you are talking about sounds which are actually audible, as opposed to sounds which are perceivable, then I've already answered the rest of your questions in my previous post (the null test and what I do with the results of one).
 
G

 

[Pio2001]

   
So has there been any work done to establish what features of the sound are "available" in the echoic memory for the "comparison function"?

 
Yes, it seems so. Scholar.google.com returns a lot of results for the keywords "auditory discrimination threshold short term memory".

 

[johncarm]

   
A blind test does not give you the truth! At best it gives you the truth as perceived by one individual or in the case of a sample size larger than one person, some non-perfect statistical probability of the truth.
 
 
No, we have a well understood science (model) of sound and a quite well understood model of the ear but when we bring the brain into the equation and effectively talk about the perception of sound rather than what our ears actually hear, then there are no models or rather, there are lots of partial and/or different models, the vast majority of which are inaccurate or only accurate in certain circumstances.
 
 
Not necessarily, did you read my last post?
 
 
If you are talking about sounds which are actually audible, as opposed to sounds which are perceivable, then I've already answered the rest of your questions in my previous post (the null test and what I do with the results of one).
 
G

 
I read your post but it leaves a lot unsaid, so I'm trying to nail down specifics.
 
The general question is "Do A and B sound different?" and how we go about answering that. Let's assume they are definitely different signals. Now you said the difference might be -100 dB.

 
What specific measurement are you taking that results in the number -100 dB? I do know what dB is, by the way, but I'm thinking about how A and B are two continuous signals, possibly of some significant duration. What calculations do you use to reduce that continuous signal to a single number, -100 dB?
 
You said that the differences might be audible if the difference is greater than that, depending on the part of the spectrum. That sounds like you are referring to theories about thresholds of audibility, correct? And referring to both intensity levels and frequencies.
 
So what I was asking was whether listening tests were used in forming theories about the thresholds of audibility. For example, -100 dB is well below the threshold of audiblity, you say. So what do we know scientifically about thresholds of audibility? And how much did listening tests figure into making those determinations?

 

[johncarm]

   
Yes, it seems so. Scholar.google.com returns a lot of results for the keywords "auditory discrimination threshold short term memory".

I did look at a few of those, and a lot of them are dealing with intensity and frequency discrimination. But of course a musical signal is complex and full of many features. My question is, which of those features is available in echoic memory? I don't see how intensity and frequency discrimination studies have much to say about that. Presumably those tests are done with simple signals that don't approach the complexity of music. They only have one "feature" -- their intensity or frequency.

 

[Pio2001]

Originally Posted by johncarm /img/forum/go_quote.gif
 
What calculations do you use to reduce that continuous signal to a single number, -100 dB?
 
 
So what I was asking was whether listening tests were used in forming theories about the thresholds of audibility. For example, -100 dB is well below the threshold of audiblity, you say. So what do we know scientifically about thresholds of audibility? And how much did listening tests figure into making those determinations?ns?

 
For the first question, it would be better if you familiarize yourself with the basics of digital audio : what is a sound, what is a digital sound, how does an audio editor work, sample rate, quantization, then go on to more advanced topics, like spectrum analysis, RMS level etc.
 
For the second topic, maybe wikipedia can help you.
 
Answering your questions directly would fill entire pages.
 
Quote:

I did look at a few of those, and a lot of them are dealing with intensity and frequency discrimination. But of course a musical signal is complex and full of many features. My question is, which of those features is available in echoic memory? I don't see how intensity and frequency discrimination studies have much to say about that. Presumably those tests are done with simple signals that don't approach the complexity of music. They only have one "feature" -- their intensity or frequency.

 
I don't know. My advice is : experiment by yourself. This knowledge doesn't come from mathematical theorems. And the forum won't answer everything.