[Phronesis]

1. That is certainly a good step forward from many audiophiles, who not only openly state they "trust their ears" but often mock those who don't. However, that's still only a step forward, there are in fact further steps and without exception (as far as I'm aware) everyone who either mixes sound and/or blind tests sound quickly learns that their hearing perception is more fallible than they realised AND, this learning continues almost ad infinitum! By this I mean that even after significant testing experience and realising how fallible our hearing is, further testing experience reveals even more/deeper fallibility. However, I'm not entirely comfortable using the term "fallible", fallible in terms of an objective measuring device, in terms of representing the fidelity of the sound waves entering our ears, sure, highly "fallible". On the other hand, without that "fallibility" there would be no such thing as music and even our very survival as a species would have been impacted. So, it's not really a "fallibility", it's a strength, a desirable, even a necessary evolutionary trait, except if we want to use it to accurately measure the actual properties of a sound wave and, the same is true of our sight and other senses.
1a. It obviously depends on what we're testing for. For example, are we testing for a property of sound waves or a property of perception? And, "fallibility" doesn't mean that we are necessarily wrong, just that we are capable of being wrong. In other words, we don't "simply dismiss the results" of sighted listening tests! In fact, my job would be impossible if I didn't mainly rely on sighted listening tests. The only time we do "simply dismiss the results" is when we're talking about an actual property of sound waves AND, when the results conflict with the known facts/science of those properties. Going back to Rob Watts claim for example, -300dBFS represents an energy level that probably isn't enough to move even a single air molecule, let alone move the countless billions of air molecules necessary to propagate an actual sound wave. So it's not a question of how good/sensitive Rob Watts or anyone else's ears are, there's no sound wave there to hear!
1b. You mean results have gone against your conscious expectations. You state in one post that we have to remember that most of the brain's cognitive processing occurs at the sub-conscious level and then in almost the very next breath make an assertion which completely forgets the very fact which you've just informed us we have to remember!?
1c. That's a common fallacy which is often used by audiophiles. Effectively: It's not just me, numerous others have heard/reported a difference (with cables, DACs, etc.). Did you watch the link to the McGurk Effect I posted (McGurk Effect)? Probably more people heard the difference in this video than there are audiophiles on the planet! Should we not therefore consider all this evidence (that there is an audible difference) as more reliable/valuable and acceptable than any evidence the audiophile community could come up with? Do you therefore accept all this evidence (that there is an audible difference in the video) and if not, why not?

2. I can't see how this assertion is valid. "It's own pitfalls" (of blind testing) is NOT it's OWN pitfalls, ALL of them are also pitfalls of sighted testing! The fact that sighted testing simply ignores all those pitfalls doesn't mean they don't exist.
2a. If we're testing an actual property of sound waves, the "lack of access to subconscious effects" isn't a "pitfall", it's the whole point of a blind test to begin with! The ONLY time a "lack of access to subconscious effects" would be a pitfall is if we were actually testing for those subconscious effects!

3. From a scientific/factual point of view, it's not "hard to draw any conclusions", a high school level of understanding the decibel scale should be enough. What could make it hard is a lack of scepticism (a willingness to believe) in what Rob Watts is stating and therefore whether one bothers to recall one's high school education or of course, if one never had that high school education to start with.

4. That is certainly possible and may be the case. Although I can't rule out the possibility that he does in fact understand the technicalities but is deliberately misrepresenting them for marketing purposes, especially as we talking about such a fundamental technicality (the dB scale).

5. Agreed, the aspects of science and engineering of how the technology works is not at all simple, especially if we're talking about individual products, where even an expert might not know exactly how it works because some of that information could easily be a commercial secret. However, this typically isn't relevant! Because ...
5a. But, we don't need to go beyond electronics to "mechanical, acoustics", except in the case of transducers. And again, psychoacoustics/perception is not in the realm of audio reproduction equipment, that's the realm of those creating the audio to be reproduced.
5c. I agree but then, for the vast majority of audiophile myths and disputes, they don't need to know all the technicalities of how the equipment works, just a few basic principles. For example, there are numerous different ways and complex scientific and engineering technicalities of digital audio conversion. On the other hand, all digital audio converters must operate within the fundamental tenets/laws of digital data otherwise, by definition, they are not digital audio converters. So, we can dismiss any claim/assertion that breaks the fundamental tenets of digital data only by knowing those fundamental tenets and without having to know all the technicalities of how a particular converter works.



1. What you've described is effectively the definition of digital audio! A point by point measurement of complex signal amplitudes using an extremely small time increment.
2. However, to conclusively say that signal X is the same as signal Y, we do NOT have to do what you are suggesting. There is a far older, simpler and widely used test, the Null Test. We simply invert the phase of signal X (so it's -X) and sum/mix it with signal Y. So, we've got "-X + Y". If the result is zero (null), then X = Y, X must be the same as Y, no matter how simple or complex the signals are! Quick, easy and probably as old as audio recording itself!

G

Thank you for taking time to provide such detailed responses.

With Rob Watts, without reading the posts you mentioned and putting things in context, I'm reluctant to criticize him. His posts that I did read didn't touch on hearing super low dB, and instead focus on timing, noise floor modulation, etc.

I don't think there's a clean line dividing valid and invalid evidence, and much evidence falls in the gray zone. For example, if everyone who listens to a particular DAC describes its sound signature in a similar and distinctive way, I think that's stronger evidence than those people just saying it sounds "better" than something else. It's true that people could be wrong about what they're hearing and the similarity of their descriptions could be influenced by what other people have said, but it's also possible that there's really something there which people have picked up on.

In referring to "lack of access to subconscious effects," I meant mainly effects that are part of perception, but the listener isn't consciously aware of those effects and therefore generally unable to describe them. I think it's at least plausible that someone could do blind testing of A and B, and be unable to consciously tell them apart, but there's still a sound difference which matters over time. If such effects occur, I have no idea how significant they might be (could be insignificant).

Hopefully, it's apparent that I'm not cleanly in the objectivist or subjectivist camp. I think each perspective has its strengths and limitations, so we have to try to combine them, trying to find a balance between open-mindedness and credulity. This multi-perspectival approach is actually how I've looked at philosophy, religion, politics, etc. throughout my adult life.

 

[gregorio]

[1] It's true that people could be wrong about what they're hearing and the similarity of their descriptions could be influenced by what other people have said, but it's also possible that there's really something there which people have picked up on.
[2] I don't think there's a clean line dividing valid and invalid evidence, and much evidence falls in the gray zone.
[3] For example, if everyone who listens to a particular DAC describes its sound signature in a similar and distinctive way, ...
[4] I think it's at least plausible that someone could do blind testing of A and B, and be unable to consciously tell them apart, but there's still a sound difference which matters over time.
[5] Hopefully, it's apparent that I'm not cleanly in the objectivist or subjectivist camp.

1. Only if there is a difference in the first place and only if that difference is actually audible.

2. Sometimes there is a clean line. Returning again to the Rob Watts example, for his anecdotal evidence to be true it would need to break the laws of physics and the dividing line couldn't be any cleaner! But other times I agree, we give evidence a weighting, Even scientific double blind tests are fallible and most have at least one or two methodology flaws but still I would give that far more weight than a bunch of audiophile anecdotal evidence. What's more, in every case I'm aware of, when audiophiles anecdotal evidence about audible differences in cables or other myths are actually tested, those obvious "night and day" differences utterly vanish and without exception!

3. But in practise that rarely occurs. Many audiophiles might concur that there is a difference but then describe it's "sound signature" in very dissimilar ways!

4. First of all, again, there would have to be an audible difference in the first place, otherwise time is irrelevant anyway. And secondly, all the reliable evidence demonstrates an audio memory of just a few seconds, after which audio memory significantly degrades. There's not a single piece of reliable evidence I'm aware of which indicates the reverse. In fact, the only evidence I'm aware of which even attempts to contradict this demonstrated science is audiophile anecdotes.

5. I'm not sure where those terms came from but I don't like them. I'm pretty much a stereotypical "objectivist" as far as audiophiles would label me but in practise my job, my survival in my job and the main reason I love it so much is because it's so subjective! I'm probably more creative/subjective than most audiophiles who label themselves as subjectivists, the difference I believe, is that I have a better understanding of which aspects of audio are subjective and which are objective.

G

 

[Phronesis]

5. I'm not sure where those terms came from but I don't like them. I'm pretty much a stereotypical "objectivist" as far as audiophiles would label me but in practise my job, my survival in my job and the main reason I love it so much is because it's so subjective! I'm probably more creative/subjective than most audiophiles who label themselves as subjectivists, the difference I believe, is that I have a better understanding of which aspects of audio are subjective and which are objective.

Yes, I agree, I don't like the terms either. As long as we're talking about a mind and a physical world, there will be both interacting subjective and objective aspects.

And I'll reiterate the general argument that I think too much attention to sound quality can interfere with enjoyment of music. In fact, to truly enjoy music, I believe we need to get to a point where we don't pay attention to sound quality at all. With live music, sound quality never gets my attention unless it's especially bad.

 

[castleofargh]

Thank you for taking time to provide such detailed responses.

With Rob Watts, without reading the posts you mentioned and putting things in context, I'm reluctant to criticize him. His posts that I did read didn't touch on hearing super low dB, and instead focus on timing, noise floor modulation, etc.

I don't think there's a clean line dividing valid and invalid evidence, and much evidence falls in the gray zone. For example, if everyone who listens to a particular DAC describes its sound signature in a similar and distinctive way, I think that's stronger evidence than those people just saying it sounds "better" than something else. It's true that people could be wrong about what they're hearing and the similarity of their descriptions could be influenced by what other people have said, but it's also possible that there's really something there which people have picked up on.

In referring to "lack of access to subconscious effects," I meant mainly effects that are part of perception, but the listener isn't consciously aware of those effects and therefore generally unable to describe them.

Hopefully, it's apparent that I'm not cleanly in the objectivist or subjectivist camp. I think each perspective has its strengths and limitations, so we have to try to combine them, trying to find a balance between open-mindedness and credulity. This multi-perspectival approach is actually how I've looked at philosophy, religion, politics, etc. throughout my adult life.

https://www.head-fi.org/threads/watts-up.800264/page-18#post-13547900 he makes allusion to this often in his own section of the forum. took me a minute to find a few with advanced search. I even remember the same anecdote about noise shaping and the same idea of "do not assume what can be heard" in one of his videos. so for missing his mentions of this, I have to conclude that you're not a true fan ^_^.




for many issues in audio, there is a gray area only because people create/allow it. when you test something following a rigorous method, have removed or fixed all the variables you could think of, have tried to established a control test for those you might have missed. when you can control the one variable you wish to test, and when you can count on repeatability, you tend to get results that will directly relate to the variable you tested. and confidence in that will be pretty high. that is true both for measurements and listening tests.
but of course when instead we sit in a chair, do a sighted test and set up no control of any sort, the resulting impressions will have everything as a potential cause. and given how the human brain works, almost everything will probably have played a role in our impressions. so anybody making assumptions about the cause of some impression under those conditions is at best cherry picking his variable and deciding there is causality in the same fallacious move. when a test is BS of course the results are inconclusive. don't blame anything but the lack of proper testing.

deciding on what is valid evidence isn't always easy. we do the best we can attributing some level of confidence and count of more evidence, and no evidence of the contrary to raise our confidence in our idea. that's work in progress for a great many things. but I have absolutely no issue determining invalid evidence: if there is a reason to doubt and wonder if it's valid evidence, then it's invalid! welcome to the land of healthy skepticism.

 

[Phronesis]

https://www.head-fi.org/threads/watts-up.800264/page-18#post-13547900 he makes allusion to this often in his own section of the forum. took me a minute to find a few with advanced search. I even remember the same anecdote about noise shaping and the same idea of "do not assume what can be heard" in one of his videos. so for missing his mentions of this, I have to conclude that you're not a true fan ^_^.

Thanks for digging up that post. To me, what he's saying is much more nuanced than simply claiming he can hear things so quiet that they have to be inaudible. He himself is skeptical of such a claim, and seems to be saying that changing the product such that it results in a measurable change in an internal signal error parameter somehow results in an audible benefit for depth perception. He doesn't attempt to explain why that is, he just notes the observed correlation.

"I agree with you; talking of small signal errors at -350 dB upsetting depth performance is just plane crazy. Objectively, it makes no sense that the ear/brain can be so sensitive."

"Now when I listen to noise shapers, and change the performance of noise shapers, I can easily and reliably hear changes in depth perception. And when I measure the digital domain performance of the noise shaper, there is a correlation between the small signal performance of the noise shaper and depth perception; so a 200 dB noise shaper (this means it can't resolve small signals below -200 dB accurately) has poorer depth than a 220 dB noise shaper - and continuing this process on I ended up with 350 dB performance (and I could still hear improvements from 330 to 350)."

"Now these numbers really do not make any kind of rational sense."

"... these performance numbers still make me uncomfortable; but I have repeated it with different pulse array noise shapers, and truncator noise shapers about half a dozen times with different designs - and I get exactly the same result. And I can hear you saying it is confirmation bias, because the listening tests are not double blind, and I know which noise shaper I am listening too - and I can't rule that out as a possibility. But I do plenty of listening tests where I get zero difference when I am actually expecting a sound quality change as I conduct my listening tests to be as accurate as possible. Because of these multiple tests, I am convinced that these results are real."​

I don't know what a noise shaper is but, again, my impression is that what he's talking about has nothing to do with being able to directly hear a sound at -350 dB.

Rob's approach seems to blend empirical experimentation with theoretical considerations and an open mind. I think that's a good thing.

 

[pibroch]

Thanks for digging up that post. To me, what he's saying is much more nuanced than simply claiming he can hear things so quiet that they have to be inaudible. He himself is skeptical of such a claim, and seems to be saying that changing the product such that it results in a measurable change in an internal signal error parameter somehow results in an audible benefit for depth perception. He doesn't attempt to explain why that is, he just notes the observed correlation.

"I agree with you; talking of small signal errors at -350 dB upsetting depth performance is just plane crazy. Objectively, it makes no sense that the ear/brain can be so sensitive."

"Now when I listen to noise shapers, and change the performance of noise shapers, I can easily and reliably hear changes in depth perception. And when I measure the digital domain performance of the noise shaper, there is a correlation between the small signal performance of the noise shaper and depth perception; so a 200 dB noise shaper (this means it can't resolve small signals below -200 dB accurately) has poorer depth than a 220 dB noise shaper - and continuing this process on I ended up with 350 dB performance (and I could still hear improvements from 330 to 350)."

"Now these numbers really do not make any kind of rational sense."

"... these performance numbers still make me uncomfortable; but I have repeated it with different pulse array noise shapers, and truncator noise shapers about half a dozen times with different designs - and I get exactly the same result. And I can hear you saying it is confirmation bias, because the listening tests are not double blind, and I know which noise shaper I am listening too - and I can't rule that out as a possibility. But I do plenty of listening tests where I get zero difference when I am actually expecting a sound quality change as I conduct my listening tests to be as accurate as possible. Because of these multiple tests, I am convinced that these results are real."​

I don't know what a noise shaper is but, again, my impression is that what he's talking about has nothing to do with being able to directly hear a sound at -350 dB.

Rob's approach seems to blend empirical experimentation with theoretical considerations and an open mind. I think that's a good thing.

Phronesis, as someone with knowledge of how science works, I’m puzzled that you could place so much credence in what is not only an unverified set of results and conclusion, but is actually part of a product promotion for the author’s employer.

 

[Phronesis]

Phronesis, as someone with knowledge of how science works, I’m puzzled that you could place so much credence in what is not only an unverified set of results and conclusion, but is actually part of a product promotion for the author’s employer.

I'm not saying that I agree with what Rob is saying, or that I even understand the details. My point is that Rob isn't claiming what he's reported to be claiming, and I find the methodological approach Rob describes as reasonable, even if it's subject to error (which Rob acknowledges).

 

[bigshot]

Well, I suppose you could research the details and find out whether he knows his stuff or is full of hot air. I have to do that all the time with Facebook "news" stories. I also consider the source. If there is a commercial interest involved I'm even more critical.

 

[pibroch]

I'm not saying that I agree with what Rob is saying, or that I even understand the details. My point is that Rob isn't claiming what he's reported to be claiming, and I find the methodological approach Rob describes as reasonable, even if it's subject to error (which Rob acknowledges).

Phronesis, Rob clearly gave credence to an alleged effect without properly testing it. A sighted impression of depth! (You don't have to understand the science or technology in the rest of the experiment to get the import of that do you?) Do you really think that is reasonable? I don't think most reasonable people would think so.

Later in the same post Rob goes on to discuss something he read in the entertainment magazine New Scientist concerning the possibility of individual neurons each having their own quantum processor. He states how if true this may be involved in depth perception, implying that this may in turn possibly account for his experimental (unverified sighted impressions of depth differences) result. I mean really!

 

[bigshot]

The crux of the problem is that he comes up with the cause of problems and solutions for them before he's even proven that the problem actually exists. That is an age old ploy used by equipment salesmen to prove that their product is better than the competition, even if it is audibly identical.

 

[pibroch]

The crux of the problem is that he comes up with the cause of problems and solutions for them before he's even proven that the problem actually exists. That is an age old ploy used by equipment salesmen to prove that their product is better than the competition, even if it is audibly identical.

I think you nailed it.

 

[Phronesis]

Phronesis, Rob clearly gave credence to an alleged effect without properly testing it. A sighted impression of depth! (You don't have to understand the science or technology in the rest of the experiment to get the import of that do you?) Do you really think that is reasonable? I don't think most reasonable people would think so.

Later in the same post Rob goes on to discuss something he read in the entertainment magazine New Scientist concerning the possibility of individual neurons each having their own quantum processor. He states how if true this may be involved in depth perception, implying that this may in turn possibly account for his experimental (unverified sighted impressions of depth differences) result. I mean really!

Rob describes changing a parameter in his device and consistently hearing a difference for some cases but not other cases. I think this is reasonable, even if fallible, since listening has to be part of the process. Subsequent blind testing would be better, and I don't know if he did that.

This specific issue isn't important to me personally, since I've done my own listening tests, and was satisfied that I consistently heard differences in the sound quality of the Mojo and Hugo 2 which I preferred, so I bought them. If my listening tests mislead me, that's ok, since I needed DAC/amps anyway, the costs won't break the bank, and the products at least look cool and seem to have plenty of power to drive my headgear. My level of scrutiny and testing will be more rigorous if I ever consider "upgrading" to the Dave, which costs about $10K. In the end, I'm a pragmatist about such purchases, and my search for evidence is scaled to what's at stake for me.

Regarding quantum effects and the mind, many reputable scientists have speculated about that: https://en.wikipedia.org/wiki/Quantum_mind. I haven't explored this area enough to have any opinions on any of the hypotheses, but it occurs to me that, in this forum, there's often reference to ears/brains, but reference to minds seems to be avoided. At some point, we have to deal with the problem of consciousness and minds if we're going to be talking about subjective musical experiences, not just observable biological brain processes. Just as I'm not convinced that any computer, no matter how sophisticated, can "think," I'm not convinced that such a computer could experience music the way humans can, via our minds and consciousness. This question takes us to the limits of science, and perhaps beyond, but IMO it's a real question nonetheless.

 

[castleofargh]

I understand why you would think that way. Rob does post a lot of true things, and as I said, he knows what should and shouldn't be and doesn't go on to make too many blatant claims. that's why I said in the beginning that he was dangerous in that respect, because he's not a random fool at all. it's easy to start trusting all he says. but the thought of sounds at levels so low making a difference does not make any sense, yet he lets us all entertain the idea that extra fidelity even at those levels can still have an impact on soundstage. it's a very conscious and calculated decision IMO. he knows what he's doing and it serves his marketing of over the top noise shaping and gazillion taps that will keep increasing with new models years after years. what it does not serve is truth, as we now have people genuinely concerned with stupidly small stuff no human would ever notice and that we have no mean to reproduce into a headphone.

arguing about audible stuff at -200dB, that's not a small extension due to caution. that's like taking a random guy, and say we shouldn't assume to know how high he can jump because once I was on a plane and saw someone jumping on the wing from the ground while at cruising altitude. all the "I know it shouldn't happen" in the world won't be enough to stop you from thinking it's a ludicrous statement that shouldn't have been made. I feel that way about his anecdotes on crazy low amplitude changes making an audible difference. maybe you don't in part because the logarithmic aspect of decibels makes it all look small and reasonable?
oh and also, there are no playback devices resolving 200dB at their output. but I guess at this point it's just one more detail.

you like the guy, we get it, I like him too aside from those marketing stunts. and his gears are pretty cool based on those I have tried. but all that is irrelevant. some things are true, others are not. who said them doesn't matter and you're making a mistake constantly bringing who he is and what else he says and does in the mix. we're not doing Rob's trial, we're saying that this specific anecdote is nonsense.

 

[Phronesis]

I understand why you would think that way. Rob does post a lot of true things, and as I said, he knows what should and shouldn't be and doesn't go on to make too many blatant claims. that's why I said in the beginning that he was dangerous in that respect, because he's not a random fool at all. it's easy to start trusting all he says. but the thought of sounds at levels so low making a difference does not make any sense, yet he lets us all entertain the idea that extra fidelity even at those levels can still have an impact on soundstage. it's a very conscious and calculated decision IMO. he knows what he's doing and it serves his marketing of over the top noise shaping and gazillion taps that will keep increasing with new models years after years. what it does not serve is truth, as we now have people genuinely concerned with stupidly small stuff no human would ever notice and that we have no mean to reproduce into a headphone.

arguing about audible stuff at -200dB, that's not a small extension due to caution. that's like taking a random guy, and say we shouldn't assume to know how high he can jump because once I was on a plane and saw someone jumping on the wing from the ground while at cruising altitude. all the "I know it shouldn't happen" in the world won't be enough to stop you from thinking it's a ludicrous statement that shouldn't have been made. I feel that way about his anecdotes on crazy low amplitude changes making an audible difference. maybe you don't in part because the logarithmic aspect of decibels makes it all look small and reasonable?
oh and also, there are no playback devices resolving 200dB at their output. but I guess at this point it's just one more detail.

you like the guy, we get it, I like him too aside from those marketing stunts. and his gears are pretty cool based on those I have tried. but all that is irrelevant. some things are true, others are not. who said them doesn't matter and you're making a mistake constantly bringing who he is and what else he says and does in the mix. we're not doing Rob's trial, we're saying that this specific anecdote is nonsense.

I don't want to go in circles but, again, Rob himself is saying that the claim doesn't make theoretical or rational sense. He only says that he personally hears a difference, and admits that he could be wrong in what he's hearing. And he doesn't claim that anyone else besides him heard the same difference. So I think he's being plenty transparent, and we the readers can give however much or little weight we want to what he reports from his hearing tests (I personally don't give it much weight, because if someone's ears are going to be relied on, I'd rather rely on my own). It's not fair play to criticize him for claiming things he never claimed.

 

[castleofargh]

I care about what is true and what isn't, I care about limiting or avoiding the spread of misinformation. you care about being cool to the nice guy. we have different concerns.