[UltMusicSnob]

In listening tests, the instrument of measurement is the human ears belonging to the test subjects. Given a common analog/acoustic playback signal delivered over speakers or headphones, the received and experienced sound will vary among the listeners according to their own unique physiological capacities (and damage).
 
I've never seen a study done in which the individual subjects' hearing was tested and used in the data analysis following data collection. If not, the instrument of measurement is not being calibrated. The most I've seen is simple notations to the effect that "Experienced and novice listener groups were tested separately" or "Trained and Untrained listeners' results are charted in Figure 3").
 
It's hard enough getting the budget and equipment together for sound tests at all, this would add yet another layer of expense and resource needs. But if the point is to understand what the listener's judgment and possibly interpretation is, it's important to know the capacities of the [physiological] equipment they're measuring with.

 

[swspiers]

For me, this is a very interesting point.  I'm in the middle of my senior year as a psych major, and this is one of the nagging questions that I have.  On many, many levels I agree with the objectivist position.  But I have to admit, the critical assumptions that I see in studies leave me a bit baffled.
 
I hope some good discussion comes of this...

 

[nick_charles]

Interesting question...
 
Off the top of my head, Meyer and Moran tested listeners abilities and used that, made no difference as it happens, Blech and Yang categorized their listeners but again no difference was found between groups. Ashihara et al only used trained/expert listeners, for serious tests Harman train their listeners. Benjamin and Gannon trained their listeners first and any who were not capable of the discrimination task were dumped so making detection as easy as possible. Pras and Gustavino only used audio professionals, JVC's study of frequency detection only used industry professionals and musicians, the Swedish codec studies only used trained professionals, but many other researchers do test listeners and use the results in analysis.
 
 
The upshot being most researchers want to make it as easy as possible to get a non-null result when researching a specific issue such as detecting differences between stimuli, so Mr normal ears sometimes gets excluded, that said even for experts there can be wide variations between trials for the same subject and between expert subjects.

 

[swspiers]

Yeah, being informed is hard.  And expensive.  I'll have to make due with my university library for now.  I already have two professional memberships, and seriously need to hold off on joining another for my 'hobby'. In the meantime, accepting a statement like 'made no difference' at face value is on-par with accepting that silver cable enhances the midrange. 

 

[mikeaj]

"Makes no difference" shouldn't follow from a null result (maybe "made no statistically significant difference under the conditions tested"); it's mostly a problem of people overreaching on conclusions given the data.
 
But do note that for audio studies, the different treatments that are found to be perceptually not significantly different are often very easily measurably different with tools other than human hearing apparatus.
 
 
As for screening participants, that seems definitely of interest if you are looking for higher statistical power for results, which is why some more serious studies do it, as overviewed by nick_charles. But I think the performance of individuals is usually not of much interest in of itself. In fact, many studies may consider listener differences as nuisance variables and just attempt to average out their effect by getting enough different people. Trying to correct for each person individually is too much of an experimental hassle and might not even produce good results.
 
With respect to figuring what is possible for potentially someone (and not for the average person), I don't know if the research is as good. From my outsider's vantage point, audiology and related fields of study and even psychoacoustsics seem to tend to focus on treatment and diagnoses of people with substandard hearing or normal hearing—there is not as much research into people with significantly better or well-developed hearing for certain tasks, assuming those people do exist. And for good reason, generally.
 
That said, I believe that if you look at the physiology of hearing and most human functions, there shouldn't be too great of a range of performance outside of those with hearing damage of different kinds. That said, sometimes specific hearing loss aids in certain artifact or other treatment differentiation for listening tests. The differences in listener performance are more from training and in the brain rather than in the ear.

 

[swspiers]

  "Makes no difference" shouldn't follow from a null result (maybe "made no statistically significant difference under the conditions tested"); it's mostly a problem of people overreaching on conclusions given the data.
 
 
 
That said, I believe that if you look at the physiology of hearing and most human functions, there shouldn't be too great of a range of performance outside of those with hearing damage of different kinds. That said, sometimes specific hearing loss aids in certain artifact or other treatment differentiation for listening tests. The differences in listener performance are more from training and in the brain rather than in the ear.

thanks for taking the time to reply- I appreciate it.
 
I have to admit, as a student of psychology my interest lies in the differences in training and the brain, so I have to admit that I have a bit of an agenda.
 
For now, I am going to take some time and read the various threads in this forum, and try my best not to challenge methodology going forward.  Unless I see something really glaring, I don't have any valuable input aside from deep appreciation for all of the Head-fi members who participate in this part of the discussion.  That and the fact that paying attention to what is real can save money.  A lot of money.
 
-S

 

[nick_charles]

  "Makes no difference" shouldn't follow from a null result (maybe "made no statistically significant difference under the conditions tested"); it's mostly a problem of people overreaching on conclusions given the data.
 
Sure, that should be taken as read, made no difference is just shorthand 

 
 
But I think the performance of individuals is usually not of much interest in of itself. In fact,
 
yes, test enough people and by random you will get lucky coins
 
many studies may consider listener differences as nuisance variables and just attempt to average out their effect by getting enough different people.
 
some less ethical studies call extreme scores outliers and remove them , this is often referred to as cherry-picking. Similarly when you are trying to determine 
if a population can positively discriminate between two stimuli you would use a one-tailed t-test, sadly some use 2-tailed which mean that those who are
far worse than chance suddenly become golden eared (in reverse)
 
 

 

[bigshot]

Differences in hearing wouldn't make any difference because the people with different hearing are used to hearing the world that way. A recording with a flat response sounds the same to them as reality which has a flat response. Natural response is natural.

 

[UltMusicSnob]

The question of "trained" vs. "untrained", or pro/consumer, or expert/novice as a sample population issue is related to the specific research question of interest. If the research question is looking for possible differences in perceptions among different population clusters, then aside from making sure that self-identifications are true and accurate, it's probably less important to know individuals' specific hearing capabilities.
 
However, if the aim of the research is to evaluate a particular piece or pieces of equipment, then it is not acceptable under rigorous methodology to assume that all listeners are the same. Maybe the researcher will get lucky and all the listeners will have very similar hearing sensitivity. But it is not a warranted assumption--the only way to know for sure is to measure their hearing and find out.
 
An obvious example would be a device which employed some sort of new tweeter design. If we are interested in human evaluators, then we would prefer to have listeners whose ears were not insensitive to the frequency ranges of interest. At some point on a continuum of hearing deficits, it is definitely the case that a particular subject's ears will pass the point of acceptable sensitivity. Imagine someone who is completely deaf above 8kHz, for example. Such a person would be incapable of distinguishing between device A, which transparently reproduces 20 Hz-20kHz, and device B, which transparently reproduces 20 Hz-10 kHz, on the basis of high frequencies.
 
Short of deficits, differing sensitivities could likewise make a difference in the resulting evaluations--for one listener a soft passage might be at the minimum detectable passage, while another could be capable of hearing much softer program material still.
 
There are at least two possible choices available if hearing measurements are collected as data. One would be to construct an improved sampling population. Another would be to use the data in analysis afterward, to guide interpretations of their reports.

 

[Copperears]

What sounds good to me doesn't to you, and vice versa.

No amount of pseudo-science is going to prove one of us "right."

Equipment can, however, independently from listeners, be measured, and it has been, since 1960 or so.




Sent from my iPad using Tapatalk

 

[xnor]

What sounds good to me doesn't to you, and vice versa.

No amount of pseudo-science is going to prove one of us "right."

 
Must really suck if you go to a concert and you personally don't like the carefully acoustically designed concert hall which almost everyone else enjoys. Don't visit concert halls, then what about standardized cinemas?
 
There are well established standards and practices. If it doesn't sound right to you then you're - what statisticians call - an outlier.
 
 
Audiophiles seem to suppress the fact that the goal of sound reproduction is to accurately reproduce the music (= art) and not to turn the artists creation into something else.
 
 
Btw, there is some (albeit very little) science on cross cultural preference, but from what I can tell you don't seem to be interested.

 

[bigshot]

Today I heard a bird singing, but it was too strident in the upper mids and lacked body. I put my hands over my ears, but then it sounded like a veil was over the sound.

 

[Copperears]

What sounds good to me doesn't to you, and vice versa.

No amount of pseudo-science is going to prove one of us "right."

 
Must really suck if you go to a concert and you personally don't like the carefully acoustically designed concert hall which almost everyone else enjoys. Don't visit concert halls, then what about standardized cinemas?
 
There are well established standards and practices. If it doesn't sound right to you then you're - what statisticians call - an outlier.
 
 
Audiophiles seem to suppress the fact that the goal of sound reproduction is to accurately reproduce the music (= art) and not to turn the artists creation into something else.
 
 
Btw, there is some (albeit very little) science on cross cultural preference, but from what I can tell you don't seem to be interested.

I think you may be misunderstanding me.

But yes, on another note, I've rarely heard a really good concert hall; Symphony Hall for the BSO is pretty good. I appreciate all the science and hard work that goes into tuning such places.

What I was simply saying is that subjective listening tests aren't going to produce better accuracy, no matter what seeming test principles are applied to same.

Does that make more sense?


Sent from my iPad using Tapatalk

 

[Copperears]

In listening tests, the instrument of measurement is the human ears belonging to the test subjects. Given a common analog/acoustic playback signal delivered over speakers or headphones, the received and experienced sound will vary among the listeners according to their own unique physiological capacities (and damage).
 
I've never seen a study done in which the individual subjects' hearing was tested and used in the data analysis following data collection. If not, the instrument of measurement is not being calibrated. The most I've seen is simple notations to the effect that "Experienced and novice listener groups were tested separately" or "Trained and Untrained listeners' results are charted in Figure 3").
 
It's hard enough getting the budget and equipment together for sound tests at all, this would add yet another layer of expense and resource needs. But if the point is to understand what the listener's judgment and possibly interpretation is, it's important to know the capacities of the [physiological] equipment they're measuring with.

To make myself clear: the possibility of a using a standardized listening test as a measure of anything appears to be null, to me. There is no way of sufficiently calibrating any instrument for any such purpose, and the variables will never be not fuzzy (I hear better at 6 than I do at 22 only on Wednesdays, for example).



Sent from my iPad using Tapatalk

 

[castleofargh]

tastes are what they are you're right, like some would kill to get something like that

and others will say a 5years old can do better.
 
but what is kept blurred in audio for really bad reasons, is what defines the norm.
almost everybody can relate to a norm and learn to recognize it. it wouldn't change who we are and what we like to get some norms once and for all.
we cry our differences in hearing but it serves no purpose. our eyes are just the same, but strangely you never see photographers arguing about the "good" white balance. we go from a given camera with given settings, with a given lens, to a given computer with a given screen we calibrate ourself, then to some printer or whatever. and at the end we manage to all get pretty close to what we all know as a correct colorimetry. it would be completely impossible if image had been treated as an unfathomable cult like audio has.
it would be oh so easy to decide upon one frequency response (why not vaguely electrically flat?) and all say that it's neutral. pros I'm sure have always been ok with this on the pro-to-pro dialogue. it's when it comes to pro-to-consumer dialogues that the lines get blurry. and when it's about consumers tolkien to themselves then trolls pops out(sorry too tempting).
from this point on, all gears could be measured by how they differ from this universal neutral. and I can't think of a single negative consequence of this unification.
oh yeah there is one, 5billions people would not anymore be able to claim they like neutral when 99.9% of them actually don't. so what? it would just be one less fashion and people could start looking for what they really like and what is really crap, instead of trying to look pro.
 
back to my photo analogy, we know everything of any camera coming on the market, same for the lenses, the quantity of accurate information is just staggering. and knowing that this canon lens is better than that nikon one doesn't stop people from getting nikon, knowing that the last nikon camera crushes canon in high iso didn't make me sell my canon.
people still make choices based on what they like and what they need, we just all know a lot more and don't need to buy the last 10 cameras to find the right one.
as a result of always knowing what is what, I know how to balance the colors of a picture the right way because I learned from the norm so I can get it right when asked. and when it's only for myself, I abuse the reds because I love it ^_^. norms never restricted creativity and tastes, they just tell you how far you've wandered. 
that is in my eyes the full length of how different we are. audio should be something perfectly straight and normed from the pros to us. and then we would pic knowingly and adjust to our very own personal tastes and needs.
all this trial and error based on how we all are different is just a huge sodomy from the sellers to keep the market more active than it needs to be, and prevent the stagnant archaisms from getting bankrupted in favor of real quality.
quality that could and woul still have hundreds of unique sound signature, I wish for a norm, not a limitation.