[Tilpo]

This internal EQ will also affect how we hear audiophile phenomina such as 'burn in'. It also suggests that blind testing should be done with quick chnages so that our internal EQ does not have a chance to kick in.

While that might be true, the mental burn-in is still very important. The mentally burned in sound is what you are going to hear when you own the gear, and this will always be different from what you initially hear when you audition new gear. This should be taken into consideration when comparing things to find out which you like the most.

If it's about whether or not a difference is perceivable, then ABX with short trials is the way to go. Although I doubt whether an ABX experiment can take long enough for mental burn-in to arise.
This raises the following question though: what if before burn-in everything sounds more similar? What if we need to take time before our brain gets used to subtle differences before it reports them? Does this mean that ABX might be fundamentally flawed?

 

[Prog Rock Man]

ABX is not flawed, it just produces an unexpected result which many cannot deal with as it is so contrary to what is expected.
 
Of the tests we have from long term such as Hifi Wigwam's power cord test to quick switches the results are the same. With ABX no one can reliably tell a difference.
 
Sighted testing does produce many reports of a change in sound, often called burn in by audiophiles.
 
IMO this all points towards sight having a big impact on sound quality.

 

[Tilpo]

ABX is not flawed, it just produces an unexpected result which many cannot deal with as it is so contrary to what is expected.
 
Of the tests we have from long term such as Hifi Wigwam's power cord test to quick switches the results are the same. With ABX no one can reliably tell a difference.
 
Sighted testing does produce many reports of a change in sound, often called burn in by audiophiles.
 
IMO this all points towards sight having a big impact on sound quality.

I know it's not flawed to any great proportion, I was just making a joke.
But it's still true that getting accustomed to a sound is real, and is should be taken into considerations.

I would also not argue that sight has a big impact on sound quality. Rather that bias created by sight has a big impact on perceived sound quality.
If only bias was something we could take out of the equation in this hobby. It'd save us a lot of money, alhough it would maybe also take away part of the fun.

Unfortunately nobody is immune to bias under any circumstance whatsoever. Even ABX is flawed; the one taking the test is aware that he is taking the test. That's actually quite a big bias if you think about it.

 

[Argo Duck]

Very nicely stated hypothesis Prog Rock Man. If true, it's a valuable, and also potentially money-saving finding. Take the Schiit Bifrost (not that I'm pushing it BTW!) - looks smart, not ridiculously expensive.
 
There's a paper from the 1970s - took me ages to recall the author as it's about 10 years since I read it - think it was Robert Cialdini, which might be relevant to ABX testing. I'll need to locate it and re-read it.
 
It may be relevant to a model of what is going on when people report a judgment. It's not a simple matter of "people report what they hear", as if we're a type of measuring instrument. Rather, we are affected by how our judgment will be received by others. So, sometimes we exaggerate what we're hearing - especially if we have other cues to give us a supportable position. Other times we down-play or say we don't know - especially if we have no supporting cues - as there can be major loss of face if we're wrong.
 
There are studies that manipulate identifiability of the participants and find things like these., e.g. people make bolder statements or choose bolder points on a scale if they're anonymous than if they're identifiable.
 
Anyway, just thinking out loud atm. More later, maybe.

 

[Prog Rock Man]

I have had a little search and found
 
http://www.ingentaconnect.com/content/dav/aaua/2004/00000090/00000006/art00005
 
of which the abstract reads
 
Product sound quality judgments are based on perception of both acoustical characteristics and various non-acoustical factors. This paper focuses on how non-acoustical, contextual, factors might impact sound quality evaluation. Three different experiments showed that a positive or negative attitude towards the product (induced by priming tasks, mood, or reading about others' opinions) systematically influenced how sounds were perceived. Moreover, temporary influences such as mood interacted with more stable individual differences such as noise sensitivity. These findings suggest that product sound quality evaluation is variant across people, and that both research and industry need to consider contextual factors to fully understand how the concept of quality is constructed.
 
 
http://intellagence.eu.com/acoustics2008/acoustics2008/cd1/data/fa2005-budapest/paper/572-0.pdf
 
which states that "the image of brand names or audio-visual interactions can significantly influence sound quality judgements" and includes various studies of sound quality such as snoring and the sound of a closing car door!
 
 
Continuing the car theme, car makers appear to have twigged to the relationship between sight and sound as this Danish study shows
 
http://www.madebydelta.com/imported/senselab/Visual_Bias_in_Subjective_Assessments_of_Automotive_Sounds.pdf
 
where pictures of cars influenced the subjects impression of sound quality of the engine.
 
Back to hifi and an Austrian study of the role of barnd and country of origin on product evaluation
 
http://www.wu.ac.at/werbung/download/publikationen/97cems.pdf
 
which found that identical CDPs were rated differently for sound quality depending on their supposed country of origin. (See page 43 of the report)
 
So I would argue the ear is greatly affected by sight.

 

[Argo Duck]

^ great. I'll take a look at these; this week if I can find the time, otherwise next week.
 
The first is clearly based within a social-cognitive approach, which is fine. The abstract is rather cautious in tone; it mentions neither effect-size nor significance values. Not sure whether the results were indeed only "suggest[ive]" or the author is just being properly conservative in what may be or may have been a pretty fresh area in 2003/4.
 
The second has a fair bit of related literature in psychology - I remember seeing some years ago - and possibly some crossover with ideas from semiotics. I'll take a look at the paper in detail later. There are lots of useful figures (graphs) but I don't see any straightforward analysis of effect at first glance: to be fair, this is only a forum-note though. I note with interest the use of a semantic-differential scale, as I suggested just such an instrument in the follow-up discussion about Tyll's headphone burn-in findings. 
 
Cialdini, who I mentioned yesterday, was coming from yet another perspective, also social/contextual but looking at the rewards and risks of giving judgments (opinions) in a social context.
 
Tentatively, I would suggest a modification of your final statement to "the ear is affected by sight". Quantification of the amount sight contributes would be established by a short series of studies - assuming something practicable can be done - if the available literature doesn't already answer this.
 
 

 

[Argo Duck]

A follow up: my evaluation is there is not much one can take from the second paper on a more thorough (though brief) review. It looked the most promising at first glance. It contains many results and covers a broad area. The approach was careful, logical and interesting. However, neither sample sizes nor statistical results are given. This is generally a tell-tale sign statistical power was inadequate (indeed, some of the confidence intervals suggest as much). I hasten to add this was a forum note, to be fair. This was likely to have been a report of work still in its early stages that will or did occupy some years, or evolved in other directions.
 
The other paper I looked at briefly was the fourth. There were some dubious statistics and language in the survey of previous studies, forgivable perhaps because these studies date to a time when negligible effects could be reported as "highly significant"*. Looking into the country-of-origin effect on perceived quality, it was reported "the country-of-origin factor proved to be highly significant" (p.17). I read on to find the partial eta-2 - full credit to the authors for reporting this - to be 0.08. That is, about 8% of the variation in consumers' perception could be 'explained' by this factor. Actually, slightly less as there was a brand-country interaction as well. OTOH, their structural equation model - the piece of work developed by the authors and right in their area of expertise - could well be pretty good.
 
The lesson from just these two of the four papers - this is not a big sample - seems two-fold. First, there is a relevant literature out there. This needs to be pulled together, plausible hypotheses developed and then tested in the audiophile setting before scientifically valid statements can be made. Science progresses by experimental test; deduction alone is not sufficient.
 
Second, if effect sizes are all about as small as suggested by these two papers, many factors are necessarily involved. A complex explanatory model is the almost certain outcome. This may not be a bad thing. In particular, it allows for non-linear effects, especially if there is feedback in the system. By non-linear, I mean large differences from small changes and vice versa.
 
As I am rather overdue on three large writing projects I had hoped to finish before Christmas - and I foolishly undertook a review for head-fi! - I won't look into the other links nor go any further into this for now. Interesting material for next year I think.
 
*note: the word pairing "highly significant" is unscientific and no statistician allows it. A result is either significant or not, just as a light switch is either on or not (I'm assuming a dimmer is not in circuit). We do not append, nor do we need, this kind of qualifier. A switch is not "highly on" for example.