[sm60]

If I understand the discussion, and I admittedly may not, I think Rob is not suggesting that we can hear -301db, but that changes made at -301 do effect things in our hearing range and we can hear those changes. This is akin to the way that changes in tweeters at well beyond human hearing do affect the range we hear. There are many examples of things we cannot perceive having an influence on that which we can.

It seems that Rob and some other audio designers are taking an approach of creating products that they themselves want to hear and using their own preferences and hearing to tweak the product, hoping that if they hear it and enjoy it others will too. I agree with this approach. Music is way too subjective for measurements, tests and etc to be anything more than a part of the discussion. You don’t order from the menu based on some chemical analysis machine that objectively dictates which ingredients are needed for cellular maintenance. You order based on what you like. That doesn’t mean food tasting is entirely wish-wash subjectivity either. There are expert tasters and they have sensitivity and vocabulary to taste. The best of these would also be chefs, who can not only taste but prepare better food.

IMO the last say, the final evaluator—the last instrument worth measuring audio gear is the human ear and brain. If the human in question is trained how to hear and what to listen to, they will be a better advisor. If they also understand the underlying engineering, they may be able to make a great product. I dont want an analyzer machine to tell me what I like, or should like. If I did, I would be using a Topping… I am after joy, happiness, fun and music, music, music. Musical engagement, soul, connection to the MUSIC I love. I use gear to listen to music, not music to hear gear. If i spend a decade happily, blissfully fully engaged in loving my music for hours a day and am super happy, who cares if what caused it isnt fully objective or understood? Who cares if it is placebo even? My perspective is very limited, but as I mentioned in an earlier post, the measurement, bench test type in the audio community rarely talk about music or the enjoyment of music, and seem rather unhappy in general….

Rob has brought me lots of joy. Lots. For that I am grateful.

Here’s one interesting factoid about our hearing that you may not know (from Stevens book on Audiology). At the most sensitive frequency for human hearing (around 1 kHz), if you look at the faintest sound we can reliably hear (what audiologists call the JND -- just noticeable difference), the eardrum moves less than the width of a hydrogen atom! Just ponder that for a minute….how in the blazes does the ear/brain figure out such a low level signal that moves the eardrum less than the width of a hydrogen atom, while remaining impervious to all the random molecules bouncing around in our ear.

This makes me want to believe in Rob’s thesis about -300dB differences, although I want a deeper understanding of what’s going on here……

 

[Reactcore]

the last instrument worth measuring audio gear is the human ear and brain. If the human in question is trained how to hear and what to listen to, they will be a better advisor.

It keeps me wondering..
@Rob Watts did you solely conducted all hearing tests alone while tweaking filters or were there also other persons involved?

Lol i can imagine a debate about a magnitude of a change one hears that the other dont

I dont think all human brains are exactly equal in creating their 'illusion of sound'

 

[Rob Watts]

While what you say about DACs is verifiable through measurement or simulation, some of the claims you make about psychoacoustics seem to me to be sometimes rather more speculative. Could you cite some references in the literature about the influence of noise floor modulation on “the brains ability to create the illusion of sound“?

I haven't seen anything about noise floor modulation in the psychoacoustic literature. It's just my listening tests that suggests that this is a serious problem.

Psychoacoustics seem stuck on determining the performance of the ear, and not on the problems of the brain creating the audio illusion.

I normally don't wade into these types of discussions on this forum, but (perhaps against my better judgment) I'd like to ask a few questions:

1) Are you actually claiming that your digital modules "accurately reproduce" -301 dB? You must be speaking theoretically instead of real world performance, correct?
1a) If not, how do you actually measure/verify such stated performance given the limitations of real world test equipment (on the order of 170 dB range, I believe)?

2) Have you published your (controlled) listening test methodology and results?
2a) If not, how would "doing one's own listening tests" have any bearing on whether one dismisses your tests as "not objective, rational, or scientific" when they have no way to duplicate what you've done? Empirical science is about the verification of controlled, repeatable tests.

3) If a listener "does [their own] tests" and decides they like the sound of DAVE better than DAC X, how does that necessarily hinge on noise floor modulation, etc., versus any of dozens of other variables between the DACs?

No problem in asking questions, and actually question 1 is pertinent and something that nobody has asked and should ask.

1. So the tests are set up with a 80 bit sine wave source, that is running at 768 kHz - the OP is noise shaped down to the input of the module under test. That would be at least 24 bit input, but 768k/24b aggressively noise shaped will give noise well below -400 dB within the audio bandwidth. Of course, on a 384kHz bandwidth it will be limited to 24 bit levels, so worse than 144 dB. Then this signal is passed through the module under test, simulated, and results captured. I then do FFTs on the OP data, and can check the accuracy of the -301 db signal (which is usually set to 6kHz). Just to clarify it's all done with Verilog simulation which tells you exactly how a correctly performing module will behave with that data input.

So the test is not theoretical, but real, in that it will test how the module actually performs with an aggressively noise shaped -301dB 6 kHz signal.

2. I have not published my results in say the AES. I have done papers before, and it takes a huge amount of time to do - time I don't have, given how far behind I am on my design projects. Agreed, empirical science should be about verification; the difficulty here is the subjective element. My listening tests would be considered anecdotal and not scientific evidence; but my job is to close the gap from live to recorded sound, and the tests I do are extremely carefully done and objective - and when I get a results that defies explanation I will do a single blind test with disinterested parties. That is more than enough evidence for me to do my job - that is closing the gap from live to recorded.

If you look at my blog Watts up? on Head-Fi you can read about recent recordings I have done so I can compare directly live against recorded - and the thing that struck me was the huge difference in depth perception when listening live against reproduced - this is the primary challenge that audio faces. So if the problem with depth reproduction is not down to small signal accuracy, then why does reproduced audio fail so badly with depth perception?

3. It doesn't solely depend on noise floor modulation, there are many other problems too. And you have to counter too with someone's taste - many listeners are not very objective in their listening tests.

Just to clarify - I am not saying that a -301dB signal is in any way audible - I am just saying that the distortion performance at that level is audible. How the distortion becomes audible I do not understand; perhaps it's the distortion affecting signals that are audible, although the levels are minute. Perhaps it's something to do with correlation functions that the brain employs.

Rob did not hypothesise about the relative effects of 2nd harmonic distortion and noise floor modulation on the behaviour or the brain. He stated as a matter of fact that “ the brain does have mechanisms to deal with simple distortion; but it can't deal with small signal distortions, nor noise floor modulation .. - these are unnatural distortions, and severely interfere with the brain's ability to create the illusion of sound.”

This wasn’t stated as a hypothesis, or speculation, it was stated as a matter of fact about how the brain works. Can we have a reference to an accepted text or published paper where these mechanisms are described? Who has studied the effect of noise floor modulation on the brain, and where have they published the results?.

The distortion of the ear being tolerated or processed by the brain is something I read about in the 1980's when spending time in the psychology library reading about psychoacoustics. I can't give you a reference.

If I understand the discussion, and I admittedly may not, I think Rob is not suggesting that we can hear -301db, but that changes made at -301 do effect things in our hearing range and we can hear those changes. This is akin to the way that changes in tweeters at well beyond human hearing do affect the range we hear. There are many examples of things we cannot perceive having an influence on that which we can.

It seems that Rob and some other audio designers are taking an approach of creating products that they themselves want to hear and using their own preferences and hearing to tweak the product, hoping that if they hear it and enjoy it others will too. I agree with this approach. Music is way too subjective for measurements, tests and etc to be anything more than a part of the discussion. You don’t order from the menu based on some chemical analysis machine that objectively dictates which ingredients are needed for cellular maintenance. You order based on what you like. That doesn’t mean food tasting is entirely wish-wash subjectivity either. There are expert tasters and they have sensitivity and vocabulary to taste. The best of these would also be chefs, who can not only taste but prepare better food.

IMO the last say, the final evaluator—the last instrument worth measuring audio gear is the human ear and brain. If the human in question is trained how to hear and what to listen to, they will be a better advisor. If they also understand the underlying engineering, they may be able to make a great product. I dont want an analyzer machine to tell me what I like, or should like. If I did, I would be using a Topping… I am after joy, happiness, fun and music, music, music. Musical engagement, soul, connection to the MUSIC I love. I use gear to listen to music, not music to hear gear. If i spend a decade happily, blissfully fully engaged in loving my music for hours a day and am super happy, who cares if what caused it isnt fully objective or understood? Who cares if it is placebo even? My perspective is very limited, but as I mentioned in an earlier post, the measurement, bench test type in the audio community rarely talk about music or the enjoyment of music, and seem rather unhappy in general….

Rob has brought me lots of joy. Lots. For that I am grateful.

Thank-you. Agreed on the enjoyment of music front - that's all I am trying to do.

 

[801evan]

On the flip side, many have tried to explain how another product is better than Chord (via selective objectivism), but just listening to Chord products I know Chord is better without needing to know why.

 

[JohnA1]

I can highly recommend you THIS seminar from Martin Mallison (CTO ESS).
They present their own findings that were used when designing their dacs.
And indeed about 32:00 minute Martin is talking about modulating noise which is very "annoying" to human ear.
The whole presentation is very interesting to any audiophile that cares/believe in technical side of our equipment (not to some esoteric crap that many companies are pushing)

This is the very first time I've come across someone else describing concepts similar to those of Rob and the unbelievably high db figures that the human ear/brain may be able to discriminate

Nice find.

 

[watermad]

Ground Devices?

I am a vey happy long-term owner of a Chord DAVE/M-scaler pairing and was wondering if anyone has tried either of these devices on the DAVE; since there was reports of RFI pickup between the M-Scaler and DAVE, I added ferrites to my double BNC connectors?

https://chord.co.uk/product/groundaray-advanced-high-frequency-noise-reduction/
https://ansuz-acoustics.com/products/ansuz-sortz

I will be getting a demo of the Ansuz in the coming months so will report back on if it has any effect on the DAVE.

 

[BassicScience]

I haven't seen anything about noise floor modulation in the psychoacoustic literature. It's just my listening tests that suggests that this is a serious problem.

Psychoacoustics seem stuck on determining the performance of the ear, and not on the problems of the brain creating the audio illusion.



No problem in asking questions, and actually question 1 is pertinent and something that nobody has asked and should ask.

1. So the tests are set up with a 80 bit sine wave source, that is running at 768 kHz - the OP is noise shaped down to the input of the module under test. That would be at least 24 bit input, but 768k/24b aggressively noise shaped will give noise well below -400 dB within the audio bandwidth. Of course, on a 384kHz bandwidth it will be limited to 24 bit levels, so worse than 144 dB. Then this signal is passed through the module under test, simulated, and results captured. I then do FFTs on the OP data, and can check the accuracy of the -301 db signal (which is usually set to 6kHz). Just to clarify it's all done with Verilog simulation which tells you exactly how a correctly performing module will behave with that data input.

So the test is not theoretical, but real, in that it will test how the module actually performs with an aggressively noise shaped -301dB 6 kHz signal.

2. I have not published my results in say the AES. I have done papers before, and it takes a huge amount of time to do - time I don't have, given how far behind I am on my design projects. Agreed, empirical science should be about verification; the difficulty here is the subjective element. My listening tests would be considered anecdotal and not scientific evidence; but my job is to close the gap from live to recorded sound, and the tests I do are extremely carefully done and objective - and when I get a results that defies explanation I will do a single blind test with disinterested parties. That is more than enough evidence for me to do my job - that is closing the gap from live to recorded.

If you look at my blog Watts up? on Head-Fi you can read about recent recordings I have done so I can compare directly live against recorded - and the thing that struck me was the huge difference in depth perception when listening live against reproduced - this is the primary challenge that audio faces. So if the problem with depth reproduction is not down to small signal accuracy, then why does reproduced audio fail so badly with depth perception?

3. It doesn't solely depend on noise floor modulation, there are many other problems too. And you have to counter too with someone's taste - many listeners are not very objective in their listening tests.

Just to clarify - I am not saying that a -301dB signal is in any way audible - I am just saying that the distortion performance at that level is audible. How the distortion becomes audible I do not understand; perhaps it's the distortion affecting signals that are audible, although the levels are minute. Perhaps it's something to do with correlation functions that the brain employs.

Mr. Watts,

Firstly, thank you for taking the time to answer my questions. Secondly, as a general statement to various other responders to my post, I'm not suggesting that measurements alone will tell you too much about how you will perceive soundstage or any other attribute of a DAC's performance. In fact, it seems to me that Mr. Watts is the one advancing that argument when he says "any distortion, however small, has audible consequences on depth perception". Based on a lot of personal experience with 2-channel systems over the years, rooms and speakers generally have more to do with soundstage dimensions and layering than DACs do. Rooms aren't a factor in headphone listening obviously (although individual ear topology is), but clearly soundstaging varies rather dramatically between different headphones, again more so than between different DACs I've owned or auditioned over the years.

Regarding the specific answers to my questions:

1) You state "Then this signal is passed through the module under test, simulated, and results captured". I'm slightly confused here. What is the "module under test" exactly, and why is simulation necessary?

2) I wasn't suggesting that you should necessarily publish your results in AES or elsewhere, given intellectual property concerns, the time involved, etc. I was pushing back on the notion that people weren't being "rational" in questioning your findings when they have no actual documentation on exactly what those findings are and how you arrived at them. You also state "the tests I do are extremely carefully done and objective", but we don't really have insight into your exact methods or testing setup. It's pretty easy to convince oneself that you're being completely objective (I've done it myself), but you're developing commercial products, not working for a research institution, so without external verification of results, people might reasonably have doubts about that, IMO.

Thanks for clarifying the -301 dB audibility claim. What you're saying seems at least plausible now, although it's still not clear to me how much it may actually affect the perceived performance of a DAC. The DAVE is clearly a very fine and unique product, but there are a lot of high end DACs now on the market which have addressed noise floor modulation and reduced distortion artifacts well below audibility. The differences in sound between them may or may not hinge on the performance metrics you are choosing to highlight. It's an interesting academic question, but it may remain academic given that you're unlikely to publish your findings anytime soon, and almost everyone selects equipment based on personal auditions coupled with ergonomic and pricing factors anyway.

 

[SteveHulk]

Over the Summer I'm going to be in the market for an upgrade to the power supply of my DAVE.

At the moment I'm looking at the Sean Jacobs one-box solution outboard PSU and the Farad 3 three-box setup with a control board which goes inside the DAVE.

My own experience, particularly with power amps, has shown me that when the costs of a design have to be reduced to bring the device in at a price point it is very often the PSU that gets the most savage haircut.

Does anybody here have any views or experiences relating to the Dave PSU upgrades that they would care to share?

 

[Ciggavelli]

Over the Summer I'm going to be in the market for an upgrade to the power supply of my DAVE.

At the moment I'm looking at the Sean Jacobs one-box solution outboard PSU and the Farad 3 three-box setup with a control board which goes inside the DAVE.

My own experience, particularly with power amps, has shown me that when the costs of a design have to be reduced to bring the device in at a price point it is very often the PSU that gets the most savage haircut.

Does anybody here have any views or experiences relating to the Dave PSU upgrades that they would care to share?

25% improvement with the SJ DC4 on my DAVE. 10% improvement with the other DC4 on my M-Scaler is my rough estimate. Improved noise floor, which helps with dynamics, soundstage and bass.

If you are running hard to drive headphones out of your DAVE directly (like TCs or Susvaras), it’s a 200% improvement, literally. You can’t adequately drive TCs and Susvaras off the standard power supply, in my opinion. You definitely can with the SJ DC4. It’s literally night and day. I have a WA33, so I don’t really care, but that solid state option is nice. Dare I say bass impact is actually greater on the DC4 DAVE than my WA33. Details are perhaps better, but it’s also more fatiguing

 

[jlbrach]

the power supply definitely can make a huge difference because when I added the powerman to my formula s some time ago I found it able to drive the susvara pretty well despite modest wattage...I havent heard my dave with any upgrade but it is interesting

 

[Sampajanna]

2X DC4 makes the Dave into Dave 2.0… It takes everything you love about HMS+Dave and improves it… Well worth it if you can afford it!

 

[GryphonGuy]

I don't know what the base DC4 does to DAVE but I have the ARC6 version with Mundorf wiring upgrade and it is as good as would expect a reference level Chord DAC to sound and not in the Choral range. Huge change in DAVE. I'm so satisfied with the upgrade that I will be saving some money this year as there is no desire to upgrade anything further in my system.

 

[sm60]

I haven't seen anything about noise floor modulation in the psychoacoustic literature. It's just my listening tests that suggests that this is a serious problem.

Psychoacoustics seem stuck on determining the performance of the ear, and not on the problems of the brain creating the audio illusion.



No problem in asking questions, and actually question 1 is pertinent and something that nobody has asked and should ask.

1. So the tests are set up with a 80 bit sine wave source, that is running at 768 kHz - the OP is noise shaped down to the input of the module under test. That would be at least 24 bit input, but 768k/24b aggressively noise shaped will give noise well below -400 dB within the audio bandwidth. Of course, on a 384kHz bandwidth it will be limited to 24 bit levels, so worse than 144 dB. Then this signal is passed through the module under test, simulated, and results captured. I then do FFTs on the OP data, and can check the accuracy of the -301 db signal (which is usually set to 6kHz). Just to clarify it's all done with Verilog simulation which tells you exactly how a correctly performing module will behave with that data input.

So the test is not theoretical, but real, in that it will test how the module actually performs with an aggressively noise shaped -301dB 6 kHz signal.

2. I have not published my results in say the AES. I have done papers before, and it takes a huge amount of time to do - time I don't have, given how far behind I am on my design projects. Agreed, empirical science should be about verification; the difficulty here is the subjective element. My listening tests would be considered anecdotal and not scientific evidence; but my job is to close the gap from live to recorded sound, and the tests I do are extremely carefully done and objective - and when I get a results that defies explanation I will do a single blind test with disinterested parties. That is more than enough evidence for me to do my job - that is closing the gap from live to recorded.

If you look at my blog Watts up? on Head-Fi you can read about recent recordings I have done so I can compare directly live against recorded - and the thing that struck me was the huge difference in depth perception when listening live against reproduced - this is the primary challenge that audio faces. So if the problem with depth reproduction is not down to small signal accuracy, then why does reproduced audio fail so badly with depth perception?

3. It doesn't solely depend on noise floor modulation, there are many other problems too. And you have to counter too with someone's taste - many listeners are not very objective in their listening tests.

Just to clarify - I am not saying that a -301dB signal is in any way audible - I am just saying that the distortion performance at that level is audible. How the distortion becomes audible I do not understand; perhaps it's the distortion affecting signals that are audible, although the levels are minute. Perhaps it's something to do with correlation functions that the brain employs.



The distortion of the ear being tolerated or processed by the brain is something I read about in the 1980's when spending time in the psychology library reading about psychoacoustics. I can't give you a reference.



Thank-you. Agreed on the enjoyment of music front - that's all I am trying to do.

Following up on Rob’s really cool observations on the inferiority of USB vs. optical, which I had previously cast doubt on, I realize that instinctively I was always preferring to the sound of red book CDs (either on the Blu or on my ultra Uber CEC TL0 Mk2 transport) to what I hear from streaming via Qobuz/Roon. I wonder if there’s something to his astute observation now. I can’t explain it rationally. USB should sound the same as SPDIF. Except it doesn’t. I still enjoy playing back my CDs through a high quality CD transport. Why? Is it because subliminally my brain is telling me something is awry with respect to USB streaming.

 

[801evan]

Following up on Rob’s really cool observations on the inferiority of USB vs. optical, which I had previously cast doubt on, I realize that instinctively I was always preferring to the sound of red book CDs (either on the Blu or on my ultra Uber CEC TL0 Mk2 transport) to what I hear from streaming via Qobuz/Roon. I wonder if there’s something to his astute observation now. I can’t explain it rationally. USB should sound the same as SPDIF. Except it doesn’t. I still enjoy playing back my CDs through a high quality CD transport. Why? Is it because subliminally my brain is telling me something is awry with respect to USB streaming.

CDT is simpler and have components that are lower in noise in general over digital file streaming, which needs an operating system and such. CDT always has a wider soundstage, better tonality that digital streaming has troubles catching up with and making redbook always superior over Hi-Res

 

[Rob Watts]

Mr. Watts,

Firstly, thank you for taking the time to answer my questions. Secondly, as a general statement to various other responders to my post, I'm not suggesting that measurements alone will tell you too much about how you will perceive soundstage or any other attribute of a DAC's performance. In fact, it seems to me that Mr. Watts is the one advancing that argument when he says "any distortion, however small, has audible consequences on depth perception". Based on a lot of personal experience with 2-channel systems over the years, rooms and speakers generally have more to do with soundstage dimensions and layering than DACs do. Rooms aren't a factor in headphone listening obviously (although individual ear topology is), but clearly soundstaging varies rather dramatically between different headphones, again more so than between different DACs I've owned or auditioned over the years.

Regarding the specific answers to my questions:

1) You state "Then this signal is passed through the module under test, simulated, and results captured". I'm slightly confused here. What is the "module under test" exactly, and why is simulation necessary?

2) I wasn't suggesting that you should necessarily publish your results in AES or elsewhere, given intellectual property concerns, the time involved, etc. I was pushing back on the notion that people weren't being "rational" in questioning your findings when they have no actual documentation on exactly what those findings are and how you arrived at them. You also state "the tests I do are extremely carefully done and objective", but we don't really have insight into your exact methods or testing setup. It's pretty easy to convince oneself that you're being completely objective (I've done it myself), but you're developing commercial products, not working for a research institution, so without external verification of results, people might reasonably have doubts about that, IMO.

Thanks for clarifying the -301 dB audibility claim. What you're saying seems at least plausible now, although it's still not clear to me how much it may actually affect the perceived performance of a DAC. The DAVE is clearly a very fine and unique product, but there are a lot of high end DACs now on the market which have addressed noise floor modulation and reduced distortion artifacts well below audibility. The differences in sound between them may or may not hinge on the performance metrics you are choosing to highlight. It's an interesting academic question, but it may remain academic given that you're unlikely to publish your findings anytime soon, and almost everyone selects equipment based on personal auditions coupled with ergonomic and pricing factors anyway.

Your first question was:

"1) You state "Then this signal is passed through the module under test, simulated, and results captured". I'm slightly confused here. What is the "module under test" exactly, and why is simulation necessary?"

Digital design starts off by defining an overall functional spec, then breaking the design down into various modules. Each module will have it's own spec; you then write the HDL (hardware description language - I use Verilog) code (in the good old days it would be done with macros and gates schematically) and then you have to test that code. You could jump straight to a FPGA and test it in real life, but that's a dangerous and reckless way to proceed. So you test it by creating a simulation, where the module under test is fed some input data, and the output of the module is captured with both a visual display and with data outputted into a file. The tests are to explore that it works as intended, and you run some what if scenarios to catch unexpected behaviour. The more you test, the more likely you won't have bugs.

But the really interesting thing about simulation is that you can do really powerful measurements - and these measurements are 100% accurate given the input data - and you can explore the audible performance using these real measurements. That's how I can observe -301dB performance (and stuff below that as well). Once your suite of tests are complete, you can listen to the various modules and then see if a particular distortion makes a difference to the sound. And this is the really strange thing - ultra small effects are audible. When designing Dave I started with my usual 200dB noise shapers, and progressively improved the performance of the noise shaper and noticed that every improvement gave an improvement in depth perception - I ended up with 350 dB noise shaping (that's the best I can do currently), and to test these noise shapers I used the -301dB test. Now every module that is in the digital audio path has to pass the -301 dB test - that is perfect amplitude accuracy and perfect phase accuracy too - if you want it to be transparent. And I do this now as a matter of course - it's just my standard test. And I re-evaluate this with listening tests - that's why I am confident that this is a good metric to define transparency in the digital domain, as I have repeated these listening tests on many different occasions - and still hear the same thing.

Your second question was:

"2) I wasn't suggesting that you should necessarily publish your results in AES or elsewhere, given intellectual property concerns, the time involved, etc. I was pushing back on the notion that people weren't being "rational" in questioning your findings when they have no actual documentation on exactly what those findings are and how you arrived at them. You also state "the tests I do are extremely carefully done and objective", but we don't really have insight into your exact methods or testing setup. It's pretty easy to convince oneself that you're being completely objective (I've done it myself), but you're developing commercial products, not working for a research institution, so without external verification of results, people might reasonably have doubts about that, IMO."

Apologies if I gave the impression that people were not being rational in questioning my findings. It is entirely rational to be sceptical - and even rational to be sceptical about ones scepticism. My observations are just that - things I have evaluated and concluded to be important, and should be treated as just my opinion. I always treat my listening tests as being tentative and subject to re-evaluation, no matter how careful you approach things. In particular, one has to be very careful about whether a SQ change is actually good or bad - it's extremely easy to hear an increase in brightness as better transparency when in fact it's actually worse due to more noise floor modulation for example. That said, when you do a listening test and it clearly sounds better in a defined way (like better depth - there is no question about interpretation here) and do that test many times over on many different occasions and it still does the same, then it's sensible to conclude that something real is happening - even if it is due to something that appears ultra small or insignificant.

My annoyance comes when people instantly dismiss listening tests and state that it's impossible that something can make a difference without doing any kind of listening tests themselves. Science is about discovering and understanding new things - that means being both highly sceptical and very open minded at the same time as reality is much more complex than our very limited understanding.