... and ignored the fact that the measurements we have are of course not covering all possible situations and environments with all possible switches.
This is unnecessary, and you'll have to admit, ridiculous.
I lived in Los Angeles the first few decades of my life. If while there, I decided to do a comparison of 2 umbrellas, I'd have a problem because although it rained every year I was there, it didn't rain often or much. So if I go out and test both every day for a month, and find both "work" equally well (no rain to test though), I'm not saying much.
The answer to a failed measurement (no problem means no testing a solution) is not test everything everywhere at all times... but to look for a potential predictable problem (e.g. ask those who have experience)(see below).
And again: the manufacturer or seller or other person that claims the unlikely is the one who should prove it. So why don't they?
Yes, good question. My guesses are: they don't know how or have the equipment (very unlikely), they don't want to name the equipment that exhibits the problem (to avoid a war; also very unlikely), or they don't feel the evidence would be convincing.
@sander99 Is this your normal tone of conversation or am I reading too much into it?
Don't worry about
@sander99. He's not a native speaker, but I think he does really well. He's a good guy. (whispering... and he's Dutch. Most of the many Dutch researchers I've worked with are taller than you and me combined... shhh!)
You may need to adapt the measurement method or find a new parameter to measure.. OR make do with subjective and/or surrogate parameters.
There is only so much you can measure...
There are 2 types of measures that are pretty easy, but don't give much insight into mechanisms... but they let you know if the phenomenon exists and warrants further investigation. As others have mentioned, a blind listening discrimination test (what you described wasn't convincing - let me or others know if you need tips) is one. The other, a type of measurement, doesn't require you know what to measure or what you're looking for: a null test. You simply record (once, or several times is better) setup A and setup B, and use software designed for a null test: I can highly recommend
DeltaWave (link). It's free. Let me know if you want to try it... I can help, or better yet, Paul Kane, the developer, is very nice and helpful.