[cel4145]

I'm interested in any research that confirms that the switching between audio devices often done in ABX testing reflects real world listening. There's an underlying assumption behind how ABX testing is cited that the results of a study can be extrapolated to how we listen when not in an ABX situation.

I'm not disagreeing with the results of ABX testing for the ABX listening situation--in other words, the studies I have seen generally seem well-designed, and I trust that the results would typically hold true if someone listened to their equipment in a similar manner.

What I'm interested in is the idea that people may choose to listen to the equipment through their music or the music through their equipment. The latter is an aesthetic experience which evokes an emotional response, and doing the former, such as in ABX testing that I have seen, may represent a bias in ABX testing towards a particular type of listening that is not representative of how some/many people experience music through speakers or headphones. I think this could particularly be a problem with fast switching between equipment with short audio clips as is often done.

Anyone know of any research out there? I am just a hobbyist--not an audio professional--so any help would be appreciated. But I do have a good bit of expertise in the humanities, so I am aware that scientific efforts to quantify or qualify aesthetic experiences often are murkier/fuzzier/more difficult than scientists and engineers might like to believe.

UPDATE: xnor notes later in this thread that some of what is being discussed here in this thread is DBT, not ABX testing, and should be correctly labeled as such.

 

[Speedskater]

I think that you will find that ABX testing is so very sensitive to extremely small differences that it's a challenge to the testers.  Contrary to what you read in the popular press the test is just to sensitive.  It's a time consuming challenge to set-up the test so no differences are heard even with identical sources. 
 
If a skilled listener doing a well run ABX test with familiar sources cannot hear a JND (Just Noticeable Difference)  then there is zero chance of having a 'hearing' preference in real world listening.

 

[cel4145]

Thanks, but I'm interested in this because of the current research on headphone and speaker sound preference that ABX is being used for.

 

[Speedskater]

Tons of information are available at the site below.  But it might take days of reading to find what you want.
 
'The Pacific Northwest Section of The Audio Engineering Society'
http://www.aes.org/sections/pnw/ppt.htm

 

[cel4145]

Thanks. I was hoping someone could point me to a research article about that, rather than spending a lot of time researching something that might not even exist.

 

[nick_charles]

Given that there are numerous entire lectures series and books (going back over a century(Krehbiel,1908 for example)) about how to listen to music and that even professional musical performers do not always agree on how they want listeners to experience their music characterizing music listening as a binary (kit vs music) dilemma is overly simplistic. Music listening can incorporate a mix of emotional, physiological , intellectual elements to name but a few.
 
ABX testing can be used to determine preference or difference, as such it does not have to be short term, there is nothing stopping you living with X for an extended period such as in Tom Nousaine's "Flying Blind" studies, however we know with good certainty that memory for audible differences (apart from grotesque differences) is fleeting. In Tom Nousaine's study listeners had a box that may or may not have had a circuit adding 3% distortion to the signal. Long term listeners guessed its presence at chance levels, those allowed rapid switching A/B tests got it right much more frequently.
 
If I were cynical I would think you were looking for support for a subjective-apologist position 

 - well for that look to John Atkinson, Michael Fremer , Robert Harley and J Gordon Holt who know that DBTS are wrong because they know that all amplifiers sound different. Their premise is that in their own listening rooms they can easily tell stuff apart and that the fact that they know what it is in advance is irrelevant - of course they give us a wholly non testable and thus worthless hypothesis. That said you can trick audiophile fundamentalists by saying you have swapped components but not doing so and they still manage to perceive differences !

 

[bigshot]

I would imagine that ABX testing and real world listening are quite different. ABX is controlled listening with an attempt to hear specific things. Listening to music is casual and probably glosses over a lot of details that direct A/B comparisons would reveal. I think that information on "auditory memory" would apply here. You can google that. It deals with the ability of people to hear subtle differences in samples separated by time. Human auditory memory lasts no more than a couple of seconds.

 

[cel4145]

If I were cynical I would think you were looking for support for a subjective-apologist position   - well for that look to John Atkinson, Michael Fremer , Robert Harley and J Gordon Holt who know that DBTS are wrong because they know that all amplifiers sound different.

No. I think those people who refute all the scientific research around audio equipment as nonsense merely because they think they hear something different are just expressing desire, not good critical thinking and reasonable informed opinions. I'm more in the middle. I was reading Tyll's recent essay, and then Jon Lane of ARX's post about speaker voicing over on AVS today.

I also work in an area of the humanities that applies qualitative and quantitative research methods to subjective assessments, and so I'm pointing out an actual potential flaw in ABX testing if that has not been confirmed. This is nothing new to scientific research. Lab-based clinical tests and trials often don't come up with the same results as real world testing.

Now that I have defended myself, is it possible to avoid the veiled ad hominem attacks?

 

[bigshot]

The thing is, I don't know how much there is of practical value to take away from that comment on speaker voicing. My experience is that due to complexities of real world room acoustics and compromises that inevitably take place to keep a space livable, measurements of frequency response can only take you so far. Once you get close, you have to make little compromises to make it work with your particular system and room. But the compromises that I have to make with my system and my room don't apply at all to your system and your room. There really isn't much use talking about it, other than to say "listen to a lot of music and fine tune". I don't agree however that distortion adds "character" and imbalanced and colored sound can sound better.
 
Ideally, a flat response perfectly tuned to an acoustically optimized room is what you are shooting for. But in the real world of "where the wife wants the sofa to sit" and "where the window happens to fall on the wall" will always intrude and prevent rigid models of how a system should work. But until all wives and all windows are standardized, each person is on their own.
 
One thing I do know for sure... using words like "silky" or "smooth" don't help at all. There is no adjustment for stuff like that other than to find music that is silky and smooth. It's also pretty clear that human ears are MUCH more forgiving than spec sheets. If specs are in general ballparks, they are fine. For most modern electronic components, that means just about everything out there.

 

[stv014]

ABX does not reflect real world listening if you consider bias to be an important part of the listening experience. After all, regular listening is normally sighted, and is therefore affected by many factors not directly related to actual (objective) sound quality, like (visual) aesthetics, brand loyalty, etc. However, it is still important to be able to separate real audible differences (or lack of thereof) from these, since not everyone has the same biases; for example, someone might not be aware of the alleged flaws of an amplifier using negative feedback, and thus fail to "hear" the benefits of a no feedback one.

 

[cel4145]

I would imagine that ABX testing and real world listening are quite different. ABX is controlled listening with an attempt to hear specific things. Listening to music is casual and probably glosses over a lot of details that direct A/B comparisons would reveal. I think that information on "auditory memory" would apply here. You can google that. It deals with the ability of people to hear subtle differences in samples separated by time. Human auditory memory lasts no more than a couple of seconds.

Right. I've read about audio memory. There's a huge body of study in psychology on how unreliable all sense perception is.

And yeah. There's definitely a difference in the listening experience outside of ABX testing. The question I'm interested in is whether or not the differences matter for what is being tested. So for instance, with DACs where the ABX confirms there's no audible difference, I don't see how it could matter. But when making subjective evaluations of one thing compared to other, it might.

The thing is, I don't know how much there is of practical value to take away from that comment on speaker voicing. My experience is that due to complexities of real world room acoustics and compromises that inevitably take place to keep a space livable, measurements of frequency response can only take you so far . . .

True. But if you look at it from that perspective, that whole thread is moot

 

[jcx]

there is a listening test "Bible" - and a presentation from it: http://www.delta.dk/imported/senselab/AES125_Tutorial_T4_Perceptual_Audio_Evaluation_Tutorial.pdf
 
http://www.moultonlabs.com/main/ has a few interesting articles on listening and mastering
 
http://seanolive.blogspot.com/ is another source for commentary by pros experienced in perceptual audio evaluation
 
 
lots of research has been done on room and loudspeaker interaction, impact on listening experience

 

[Copperears]

I don't think the question is finally answered, either way. I trust the science, but I also respect the subjective evaluation.

My personal opinion is that we don't "sense" objectively. Our brains interpolate and transform from details. This makes it very difficult to measure response that is not just the consequence of frequency balance, but the final result of a particular brain ingesting that frequency spectrum over time (call it the "slime delta" ).

If we were to give in to that opinion, though, there would be no audio science applicable to more than a single individual at a time. Maybe a few hundred years from now; until then, ABX testing will have to do.

 

[xnor]

The shocking fact is how little "professional" reviewers even try to do blind testing.
If you professionally review components and don't even have a well-defined test setup then you will end up with rubbish reviews. The results will be influenced so strongly by biases that I can make more use of simple measurements than the usual flowery audiophile drivel.
 
Btw, take a look at ABC/HR.

 

[bigshot]

I don't think subjectivism means much to anyone except the individual. What some test subject hears because the salmon at lunch made him ill doesn't mean a heck of a lot to me. Objective testing is more valuable, because it applies to everyone.
 
As for the brain adapting to a particular frequency coloration, I don't think that's likely because every minute of our lives that we don't have cans on our noggin we are hearing flat response in the world around us. If we pattern on anything, it's that.