glassmonkey said:

  I'm probably going to get blasted for this, but I don't believe in instant switching ABX testing. I think that the brain fills in acoustics that it expects to hear and I prefer to have a few seconds delay wash out period. This hypothesis is loosely supported by a recent meta-analysis of high definition audio discrimination studies in the AES journal, and is something that I think needs to be addressed in future ABX methodology tests. Instant switching increases the chance of false negatives.
 
The paper can be checked out here: http://www.aes.org/tmpFiles/elib/20160806/18296.pdf
 
I've put some clippings below:
 
<snip/>
 
What the forest plot shows is that studies with training appear to show a greater ability to discern high definition content, and overall there appears to be a a small ability to discern high definition content. The paper is interesting and not without limitations, so take with a grain of salt and read for yourself. Interesting, among the studies in the training subgroup the longer stimuli and longer intervals between stimuli appeared to correlate with greater ability to discern content.
 
The results should be taken in a somewhat guarded manner, as the subgroups have small sample sizes. The study also assesses most studies as having high risk of bias. It just isn't the case that audio studies involving people are generally well conducted. There is also lots of heterogeneity in the study methods of studies included in the meta-analysis, but I think the preparation work to make code the studies in a binary fashion, discern or not discern, was done well.
 
Anyway, go check out the study. It is well worth a read.

Click to expand...

torq said:

You won't get blasted by me!

The box I'm piddling around with is just a tool intended to take another person out of the loop and make sure that "random" is actually random ... I know that personally I find some effects only apparent after protracted listening (e.g. they cause fatigue, sooner or later than some other "thing"), and nothing specifically prevents the box being used in that manner.

Interesting material.

Click to expand...

torq said:

 

glassmonkey said:

  I'm probably going to get blasted for this, but I don't believe in instant switching ABX testing. I think that the brain fills in acoustics that it expects to hear and I prefer to have a few seconds delay wash out period. This hypothesis is loosely supported by a recent meta-analysis of high definition audio discrimination studies in the AES journal, and is something that I think needs to be addressed in future ABX methodology tests. Instant switching increases the chance of false negatives.
 
The paper can be checked out here: http://www.aes.org/tmpFiles/elib/20160806/18296.pdf
 
I've put some clippings below:
 
<snip/>
 
What the forest plot shows is that studies with training appear to show a greater ability to discern high definition content, and overall there appears to be a a small ability to discern high definition content. The paper is interesting and not without limitations, so take with a grain of salt and read for yourself. Interesting, among the studies in the training subgroup the longer stimuli and longer intervals between stimuli appeared to correlate with greater ability to discern content.
 
The results should be taken in a somewhat guarded manner, as the subgroups have small sample sizes. The study also assesses most studies as having high risk of bias. It just isn't the case that audio studies involving people are generally well conducted. There is also lots of heterogeneity in the study methods of studies included in the meta-analysis, but I think the preparation work to make code the studies in a binary fashion, discern or not discern, was done well.
 
Anyway, go check out the study. It is well worth a read.

Click to expand...

 
You won't get blasted by me!
 
The box I'm piddling around with is just a tool intended to take another person out of the loop and make sure that "random" is actually random ... I know that personally I find some effects only apparent after protracted listening (e.g. they cause fatigue, sooner or later than some other "thing"), and nothing specifically prevents the box being used in that manner.
 
And instant vs. delayed switching, as well as short-loops vs. entire albums is something it does already, so the choice is there for the user (assuming there's ever anyone using it but me ...).
 
Interesting material.

Click to expand...

dillan said:

 
I agree the religion bit was crossing the line - but many other times my posts have been deleted because I chose to bring up or challenge the reasons behind pricing of a certain item as well as asking the technical comparisons versus other devices and especially when I bring up double blind or A/B testing. That sort of speak is apparently banned outside of our dead sound science section which is actually extremely harmful to the community. Literally it drags us down as a whole to not only discourage it, but ban it.

Click to expand...

glassmonkey said:

  By the way, I am interested in your device (assuming affordable price) and I think that there are other reviewers that would be also. They might not work for any big magazines, though.

Click to expand...

ancipital said:

(I have nothing against a nice subjective writeup, but self-indulgent "James Joyce on acid" riffing makes me want to throw things. I want to know about the shiny, not read failed poetry.)

Click to expand...

ancipital said:

 

glassmonkey said:

  By the way, I am interested in your device (assuming affordable price) and I think that there are other reviewers that would be also. They might not work for any big magazines, though.

Click to expand...

 
Of course not, a device to facilitate ABX testing won't let the poor little flowers focus on what matters- matters like the fact that "the regular changes in rhythm want to sound paranoid, but not confused, lest the screams lose their purpose". Just the facts, ma'am.
 
No, I suspect that useful test gear, like sensible test methodology, might be like kryptonite to most big magazines.
 
(I have nothing against a nice subjective writeup, but self-indulgent "James Joyce on acid" riffing makes me want to throw things. I want to know about the shiny, not read failed poetry.)

Click to expand...

glassmonkey said:

  By the way, I am interested in your device (assuming affordable price) and I think that there are other reviewers that would be also. They might not work for any big magazines, though.

Click to expand...

ancipital said:

   
Of course not, a device to facilitate ABX testing won't let the poor little flowers focus on what matters- matters like the fact that "the regular changes in rhythm want to sound paranoid, but not confused, lest the screams lose their purpose". Just the facts, ma'am.
 
No, I suspect that useful test gear, like sensible test methodology, might be like kryptonite to most big magazines.
 
(I have nothing against a nice subjective writeup, but self-indulgent "James Joyce on acid" riffing makes me want to throw things. I want to know about the shiny, not read failed poetry.)

Click to expand...

orestes1984 said:

  Of course an objective test where a reviewer can't use such flowery language would be kryptonite to most audio magazines. Of course being a research graduate in my working life leads to one thing though. Intersubjectivity is life, this reminds me of the ongoing and never ending debates between relativists, that knowledge is truth, and subjectivity that everyone has their own opinion and everything is OK in between provided we are reflexive in our practices and willing to concede that others are allowed to have their opinion also.
 
The problem with intersubjectivity is that people on forums very rarely have the skills and nescessary knowledge to be able to concede in a civil manner that each of us are entitled to an opinion and that the "truth" is also always subjective.

Click to expand...

ancipital said:

 

orestes1984 said:

  Of course an objective test where a reviewer can't use such flowery language would be kryptonite to most audio magazines. Of course being a research graduate in my working life leads to one thing though. Intersubjectivity is life, this reminds me of the ongoing and never ending debates between relativists, that knowledge is truth, and subjectivity that everyone has their own opinion and everything is OK in between provided we are reflexive in our practices and willing to concede that others are allowed to have their opinion also.
 
The problem with intersubjectivity is that people on forums very rarely have the skills and nescessary knowledge to be able to concede in a civil manner that each of us are entitled to an opinion and that the "truth" is also always subjective.

Click to expand...

 
There's room for both, in an ideal world. If I'm considering dropping a non-trivial amount of money on a piece of gear, I'd like to know some useful data- maybe some measurements, or how it fared in blind testing- if the reviewer has the resources and expertise do it well. That doesn't preclude wanting to know how much the review enjoyed listening to it- how they felt it altered the sound and affected the amount of pleasure offered by the experience, as that's also interesting.
 
Even subjective observations can vary wildly in how useful they are, though. The sort of self-indulgent bollocks that you get in places like What Hifi can be a lot less informative than some of the more practical reviews posted around here. Sad but true.

Click to expand...

orestes1984 said:

Of course an objective test where a reviewer can't use such flowery language about things which don't technically exist would be kryptonite to most audio magazines. Of course being a research graduate in my working life leads to one thing though. Intersubjectivity is life, this reminds me of the ongoing and never ending debates between relativists, that knowledge is truth, and subjectivity that everyone has their own opinion and everything is OK in between provided we are reflexive in our practices and willing to concede that others are allowed to have their opinion also.

The problem with intersubjectivity is that people on forums very rarely have the skills and nescessary knowledge to be able to concede in a civil manner that each of us are entitled to an opinion and that the "truth" is also always subjective except in the limited cases where we can scientifically clarify through the shortest possible answer that there is no other way that the response could be anything other than what it is.

Click to expand...

glassmonkey said:

 
The results should be taken in a somewhat guarded manner, as the subgroups have small sample sizes. The study also assesses most studies as having high risk of bias. It just isn't the case that audio studies involving people are generally well conducted. There is also lots of heterogeneity in the study methods of studies included in the meta-analysis, but I think the preparation work to make code the studies in a binary fashion, discern or not discern, was done well.
 
Anyway, go check out the study. It is well worth a read.

Click to expand...