[manbear]

   ABX tests are tools for the people making the claims to support them. If they can't support them, that's points for us but they cannot be used to prove a negative.
 

If someone can't support their claims with ABX, nobody gets points. Check your basic logic--how can you get any points if you can't prove a negative?

 

[RRod]

The continued inability of people to hear differences in formats via ABX testing is evidence for those who say they cannot.  Say we naysayers are full of crap, and it's actually the case that the human organism is perfectly capable of hearing difference between formats.  Well, then ABX testing would be showing, over time, that people can indeed hear the difference.  You cannot shift the burden of proof until you can show that humans can past the test.

 

[manbear]

I am not shifting the burden of proof for the hypothesis that "A and B are different." I am only pointing out that absence of evidence is not evidence of absence. If one cannot distinguish A from B in a blind test, the failed results of such test mean absolutely nothing more. Failure to prove this hypothesis is not evidence in support of the hypothesis that A and B are substantially identical--there is no dichotemy because youre neglecting the true null hypothesis, which is simply that nothing has been proven at all.

Everyone making a specific claim has their own burden of proof. You have no burden of proof if you just want to say that nothing has been proven at all. But you want to go farther than that. You want to say that A and B actually sound the same. Thats something you have to prove with blind testing.

 

[Head Injury]

 
If someone can't support their claims with ABX, nobody gets points. Check your basic logic--how can you get any points if you can't prove a negative?

I suggest you read my other posts in the thread, as I already explained why ABX tests cannot be used to prove a negative. This is not a problem exclusive to audio, this is not our inability or unwillingness, and it's not a flaw with the testing. It's simply not something the test is designed to do.
 
Besides, I don't need to prove I can't hear a difference. A simple null test of two lossless files will reveal no difference in the waveform. This is all the evidence anyone should need. If you are not convinced, prove you can hear the difference with an ABX test. The burden of proof is yours, and no amount of misunderstanding of how reality works will change that.

 

[money4me247]

Yes, this is a good point. Just as "A is different from B" can be tested at various levels of confidence, "A is identical to B" can also be tested at various levels of confidence. Failure to establish that "A is different from B" at x level of confidence is not necessarily confirmation that "A is identical to B" at any reasonable level of confidence. 

I would honestly challenge any of the so called objectivists to show ABCX results showing that two supposedly equivalent files A and B really are indistinguishable from C with, say, 99% confidence. 

 
... lol. considering this thread is about possible sonic differences between LOSSLESS formats... which by definition should have no discernible difference from the source....... not quite sure why there is even any debate on this matter.
 
You are right. A series of failed ABx test (with the resulting percentage of correct identification not being statistically-significantly higher than random chance) does not prove that the 'A' and 'B' are identical. The conclusion is simply that you cannot accurately distinguish between them based on sound. This implies that even if there is any differences between them, the differences are below the threshold that our hearing can detect. Therefore, it is reasonable to state that there is no audible differences.
 
For a single individual doing an ABx test, your findings will be binary: either (+ for correct identification) or (- for incorrect identification). Therefore, your results will be your percentage of correct identifications. The point of statistical analysis to determine if that percentage is statistically significant beyond random chance. For the most accurate results, you will need to increase your sample size. For small sample sizes, the power of the study is lower, so it is more likely that you will make a type I error (seeing see a correlation where none exists) or type 2 error (stating there is no difference when there is a difference).
 
You will frame this statistical test as a chi-squared test with the null hypothesis of no difference and that both correct identification & incorrect identification are equally likely. There is one degree of freedom. After calculating the chi-squared value of your experimental data, you consult the chi-squared distribution for 1 degree of freedom to see if the probability of observing your specific results. If the probability is higher than 0.01 (which would be the criteria for statistical significance at a 99% confidence interval), then the null hypothesis will be accepted.
 
If you are talking about the statistical confidence interval, 99% confidence does not mean that you are 99% sure that your conclusion is correct. The 99% confidence interval is preset by the researcher and specifies the tolerable range where the means of repeated samples can occur. A higher confidence interval means that it is more likely that you will discard the correct null hypothesis as you are less tolerant of deviations in the mean that may be due to random chance, resulting in a type I error. A lower confidence interval means you are more likely to accept an incorrect null hypothesis, which would be a type II error.
 
So, actually your challenge is pretty easy to accomplish from a statistical standpoint.

 

[Claritas]

You are right. A series of failed ABx test (with the resulting percentage of correct identification not being statistically-significantly higher than random chance) does not prove that the 'A' and 'B' are identical. The conclusion is simply that you cannot accurately distinguish between them based on sound. This implies that even if there is any differences between them, the differences are below the threshold that our hearing can detect. Therefore, it is reasonable to state that there is no audible differences.

Do you think that if someone hears a difference sighted, they should be able to hear it blind?
Do you think that if someone doesn't hear a difference sighted, they shouldn't be able to hear it blind?
Do you think that a blind test can be run in each case?

I'm aware that these questions sound elementary, so please don't think I'm trying to insult you by asking them. Thanks.

 

[castleofargh]

 

Originally Posted by Claritas /img/forum/go_quote.gif
 
I would suggest that the burden of proof is equally distributed on everyone. It's worth considering whether the claim to hear no difference is actually a positive claim about the world that looks like a negative claim because it uses the word "no." Maybe if someone claims there's no difference, he should test for that too. That test likewise doesn't determine anything beyond the results without making further assumptions.

Yes, this is a good point. Just as "A is different from B" can be tested at various levels of confidence, "A is identical to B" can also be tested at various levels of confidence. Failure to establish that "A is different from B" at x level of confidence is not necessarily confirmation that "A is identical to B" at any reasonable level of confidence. 

I would honestly challenge any of the so called objectivists to show ABCX results showing that two supposedly equivalent files A and B really are indistinguishable from C with, say, 99% confidence. 

 

you realize that you're asking for a statistical tool to be 99% accurate right?
also nobody's shifting the burden of proof. lossless files are by definition identical to the last bit, they have the same impact as using winzip or winrar to compress a folder, and will be extracted back into the same pcm file before even reaching the DAC(because a DAC doesn't speak flac, alac, etc). so they have to sound the same because they are the same. when they aren't the same, it's because of a malfunction from the computer or a software, and in no circumstance could it be blamed on the lossless file.
and some matter of proof have been shown already if you look back, not that it really matters.
  one guy is making a claim he's hearing differences, the burden of proof is on him from the instant he made that claim. we're just trying to help by suggesting to remove the bias that is knowing in advance what he's listening to. it's not a lot, but probably enough to fail the test if the difference in sound doesn't come from his computer and software being really messed up. so ABX should be a source of information for him. given that he's willing to know what really happens. something people making the weirdest claims aren't usually famous for.

 

[limpidglitch]

  I suggest you read my other posts in the thread, as I already explained why ABX tests cannot be used to prove a negative. This is not a problem exclusive to audio, this is not our inability or unwillingness, and it's not a flaw with the testing. It's simply not something the test is designed to do.
 
Besides, I don't need to prove I can't hear a difference. A simple null test of two lossless files will reveal no difference in the waveform. This is all the evidence anyone should need. If you are not convinced, prove you can hear the difference with an ABX test. The burden of proof is yours, and no amount of misunderstanding of how reality works will change that.

 
Don't anyone see that you just can't prove no difference?
I would have no problem showing that two files were identical, wether or not they actually were different. I'd just guess.
How would you set such a test while taking account for the inevitable false negatives that would occur? And manbear, please explain how you'd place those confidence intervals.

There is a fundamental, and very very good reason why hypothesis testing always is done on the pattern of:
H₀: No difference
H_a: Difference
Discard H₀ if difference exceeds a certain predefined confidence level.

This is done everywhere, from pharmaceutical and other scientific research, all the way to myth-busting.

 

[RRod]

I am not shifting the burden of proof for the hypothesis that "A and B are different." I am only pointing out that absence of evidence is not evidence of absence. If one cannot distinguish A from B in a blind test, the failed results of such test mean absolutely nothing more. Failure to prove this hypothesis is not evidence in support of the hypothesis that A and B are substantially identical--there is no dichotemy because youre neglecting the true null hypothesis, which is simply that nothing has been proven at all.

Everyone making a specific claim has their own burden of proof. You have no burden of proof if you just want to say that nothing has been proven at all. But you want to go farther than that. You want to say that A and B actually sound the same. Thats something you have to prove with blind testing.

 
If you fail to reject the null hypothesis, then you mean that your particular sample cannot provide you with the evidence to reject it and accept the alternative. If we start piling up such samples independently, this should indeed provide evidence that we should "accept the null," as we can pool the outcomes and continually improve our confidence of the result.  It is true that we can never say we fail to reject the null "almost surely / with probability 1", but we can get arbitrarily close with enough samples.
 
The original claim that started this thread was that difference could be heard.  People jumping on saying "what about the other side!" are indeed shifting the burden.

 

[manbear]

Don't anyone see that you just can't prove no difference?

I would have no problem showing that two files were identical, wether or not they actually were different. I'd just guess.

How would you set such a test while taking account for the inevitable false negatives that would occur? And manbear, please explain how you'd place those confidence intervals.


There is a fundamental, and very very good reason why hypothesis testing always is done on the pattern of:

H₀: No difference

H_a: Difference

Discard H₀ if difference exceeds a certain predefined confidence level.


This is done everywhere, from pharmaceutical and other scientific research, all the way to myth-busting.

There are a number of discrimination testing methodologies that would work.

More importantly, the null hypothesis is "no discernable relationship." Discernable or detectable is the key word--failure to reject the null is evidence that a relationship has not been observed. NOT evidence that a relationship does not exist.

Also, replacing the word relationship with difference is incorrect. Youre erroneously representing "no relationship" as "no difference = same." Sameness is a relationship, and is a separate alternative hypothesis you would need to test.

 

[Makiah S]

  @KlarkKentThe3rd and others who claim to hear differences between lossless formats (WAV, AIFF, FLAC, ALAC, etc.)...
 
I created this thread for you to present your case and test it scientifically.
 
If those among you who know how to conduct proper tests would be so kind as to assist in any manner you see fit, it would be appreciated.
 
I ask that everyone approach this experiment with respect and avoid taking anything personally.

I think I've said this before, but different Dac Chip sets decode the different formats differently. Those minor changes in the DACS output based on format input result in their being a different, are the differances HUGE not really, but they are some what noticeable. So then, does WAV sound better than FLAC? Depends on your Dac, keep in mind software is what dictates hardware [or firmware] so changes in software will result in slight differances in hard ware performance. 
 
or at least that's what I tell my self :/ if that's 100% bull schiit logic your more than welcome to let me know :3 

 

[manbear]

you realize that you're asking for a statistical tool to be 99% accurate right?
also nobody's shifting the burden of proof. lossless files are by definition identical to the last bit, they have the same impact as using winzip or winrar to compress a folder, and will be extracted back into the same pcm file before even reaching the DAC(because a DAC doesn't speak flac, alac, etc). so they have to sound the same because they are the same. when they aren't the same, it's because of a malfunction from the computer or a software, and in no circumstance could it be blamed on the lossless file.
and some matter of proof have been shown already if you look back, not that it really matters.
  one guy is making a claim he's hearing differences, the burden of proof is on him from the instant he made that claim. we're just trying to help by suggesting to remove the bias that is knowing in advance what he's listening to. it's not a lot, but probably enough to fail the test if the difference in sound doesn't come from his computer and software being really messed up. so ABX should be a source of information for him. given that he's willing to know what really happens. something people making the weirdest claims aren't usually famous for.

Not sure why people are trying to tell me that i dont know how confidence intervals work. I've said nothing thats specific enough to even be incorrect. 99% confidence intervals exist. But that is not interesting.

Yes, youre right that the burden of proof is on the OP in this specific case. And i agree that he probably wont be able to demonstrate a difference between lossless formats, though some kind of computer error could be causing a difference on his system.

My comments are more broadly directed at the "because science" folks who dont understand how to properly interpret test results and who distort the meaning of failure to reject the null hypothesis. In this specific case as well as generally.

 

[Makiah S]

Not sure why people are trying to tell me that i dont know how confidence intervals work. I've said nothing thats specific enough to even be incorrect. 99% confidence intervals exist. But that is not interesting.

Yes, youre right that the burden of proof is on the OP in this specific case. And i agree that he probably wont be able to demonstrate a difference between lossless formats, though some kind of computer error could be causing a difference on his system.

My comments are more broadly directed at the "because science" folks who dont understand how to properly interpret test results and who distort the meaning of failure to reject the null hypothesis. In this specific case as well as generally.

DING DING DING
 
I got Validation for my thoughts LATER [thank you logic <3] 

 

[manbear]

I suggest you read my other posts in the thread, as I already explained why ABX tests cannot be used to prove a negative. This is not a problem exclusive to audio, this is not our inability or unwillingness, and it's not a flaw with the testing. It's simply not something the test is designed to do.

Besides, I don't need to prove I can't hear a difference. A simple null test of two lossless files will reveal no difference in the waveform. This is all the evidence anyone should need. If you are not convinced, prove you can hear the difference with an ABX test. The burden of proof is yours, and no amount of misunderstanding of how reality works will change that.

I havent made any testable claims. You have.

 

[limpidglitch]

There are a number of discrimination testing methodologies that would work.

More importantly, the null hypothesis is "no discernable relationship." Discernable or detectable is the key word--failure to reject the null is evidence that a relationship has not been observed. NOT evidence that a relationship does not exist.

Also, replacing the word relationship with difference is incorrect. Youre erroneously representing "no relationship" as "no difference = same." Sameness is a relationship, and is a separate alternative hypothesis you would need to test.

 
Pattern. Of course you'd adjust the particulars to the specific task at hand.
 
Please tell me about those testing methodologies, they'd be new me.