Thanks. Thanks really interesting. That's in accord with my intuition but with much better reasoning. I am somewhat surprised that how to choose the null hypothesis not more cut-and-dried and is not in some senses more "objective" or formulaic. This seems to leave room for argument in areas where I was convinced there was none. I suppose there are situations where you could flip a coin as to which of two views should be the null hypothesis--and this affects profoundly the conclusions reached and the types of testing done.
Even interesting more is that I got four substantially different answers from four very intelligent people (don't get too smug, I might change my mind tomorrow).
Well, I imagine that most of us would agree that the hypothesis that all properly designed DACs used as intended under the conditions intended would sound the same. And I imagine there are some readers who would find this choice of null hypothesis undesirable. And based on what I have read, it is more a matter of judgment and purpose rather than "hey buddy, you made the claim, the burden of proof is on you." I would also perceive what
@bigshot is doing is testing what most of us would consider to be the null hypothesis. It's fair game in a sense to inquire of his methods, but we should also keep in mind that in a sense he is going above and beyond in doing so. If someone wants to question his methods in testing the null hypothesis rather than an alternative hypothesis, it would seem to me to be fair game to suggest that someone criticizing him either do a better job themselves of testing the null hypothesis, or come up with an alternative hypothesis and assemble the tests and data necessary in order to do so. Otherwise it comes down to
@bigshot or others like him did at least something (since we have ruled out in the course of this conversation the idea that he is not telling the truth) and that's more than anyone else in this conversation did (including yours truly).
So here is what I propose is a good null hypothesis of this thread, by the consensus of
@castleofargh,
@SoundAndMotion,
@gregorio, and
@Phronesis: all properly designed DACs used as intended under the conditions intended would sound the same.
Does anyone disagree that this should be the null hypothesis? Remember, technically speaking, as I understand things now, it is a
guess--having it be the null hypothesis does not make it a truism or an undebatable point. All is not lost for anyone who has another view. So. . .
Does anyone have a proposed alternative hypothesis that we can test against this null hypothesis in some kind of decent way?