[Jaywalk3r]

Not at all... I'm not asking anyone to accept or reject anything.

That's exactly what you're doing. You are asking us to assume, without evidence, that there exist at least two current, in-spec DACs that sound audibly different. You've made the claim repeatedly, each and every time without anything that could be reasonably considered evidence.

In science, a hypothesis generally isn't "accepted " or "rejected" without evidence and experimentation.

Correct, yet you expect us to do just that, reject the null hypothesis without evidence.

 

[Jaywalk3r]

I've simply pointed out that "the default hypothesis" is based on incomplete, out of date, and largely only tangentially related information.

No doubt you believe that, but all you've actually pointed out is that you don't understand how the scientific method works.

 

[Phronesis]

IMO, reference to "null hypothesis" is being misused in this context. Sometimes we just don't know something, and in those cases there's no default null hypothesis from a practical standpoint. In a test, an investigator may label A and B "not sounding different" as their formal null hypothesis, but that doesn't necessarily mean that the investigator expects them to not sound different.

 

[Jaywalk3r]

IMO, reference to "null hypothesis" is being misused in this context. Sometimes we just don't know something, and in those cases there's no default null hypothesis from a practical standpoint. In a test, an investigator may label A and B "not sounding different" as their formal null hypothesis, but that doesn't necessarily mean that the investigator expects them to not sound different.

Right. The research hypothesis is typically suspected of being true, otherwise there would be no reason to do any research. But the investigator understands that concluding that they have audible differences is not justified without seeing data which would be unlikely if the null hypothesis is, in fact, true.

 

[Phronesis]

Right. The research hypothesis is typically suspected of being true, otherwise there would be no reason to do any research. But the investigator understands that concluding that they have audible differences is not justified without seeing data which would be unlikely if the null hypothesis is, in fact, true.

No disagreement. We just need to be clear that we're talking about a bunch of different things in this thread, and sometimes jumbling things together which are best kept separate. The question of whether two DACs can sound significantly different needs proper scientific investigations, and like Keith, I'm not aware of such investigations having been done to date.

 

[Jaywalk3r]

No disagreement. We just need to be clear that we're talking about a bunch of different things in this thread, and sometimes jumbling things together which are best kept separate. The question of whether two DACs can sound significantly different needs proper scientific investigations, and like Keith, I'm not aware of such investigations having been done to date.

I think the disagreements arise because some posters don't understand that in the event there is not sufficient evidence to reject the null hypothesis, the only alternative is to not reject the null hypothesis. That doesn't mean that the researcher no longer suspects the research hypothesis to be true, it just acknowledges that data observed would not be unlikely if the null hypothesis is true, and that further investigation will be required before the null hypothesis can be rejected justifiably.

Failure to reject the null hypothesis is not the same as embracing the null hypothesis.

 

[KeithEmo]

Thank you... that's what I've been trying to get across for some time.

No disagreement. We just need to be clear that we're talking about a bunch of different things in this thread, and sometimes jumbling things together which are best kept separate. The question of whether two DACs can sound significantly different needs proper scientific investigations, and like Keith, I'm not aware of such investigations having been done to date.

 

[bigshot]

BigShot is claiming that there is NO audible difference, between ANY two well designed DACs.

I haven't said that. What I've said is... Of the dozens of DAPs and DACs I've owned in the past couple of decades, I have tested every one of them and determined them to be audibly transparent- meaning there is no difference between them. I asked for someone to point me to one that wasn't audibly transparent, so I can know that colored DACs exist.

You pointed me to a DAC, but it was a lousy recommendation because you made no effort to determine if it really was colored. If I wasted my time chasing down every biased "impression" in Head-Fi, I would be spinning my wheels until the cows come home, and I'd get no closer to finding a colored DAC. The reason I ask here in Sound Science for examples of colored DACs is because I assume that people here know how to filter bias and conduct a halfway decent comparison test. I was wrong to assume that you would do that.

I've only had one other DAC suggested to me, and that was by the Geek Squad guy. I asked him how he determined that it was not audibly transparent, and he pointed to measurements showing noise at -60 or -70dB or something like that. He claimed that was "clearly audible". When I started asking him about his listening test procedures, he ignored me and started in with the yellow highlighted quotes from books saying a noise floor of -120dB was required for audible transparency. He also hadn't done a halfway decent listening test. He was cherry picking quotes to fit his pre conceived theories.

Some people think of reasons for something to exist before they establish that it even exists. You kept doing that by jumping ahead to talking about filters and Sabre chips and all that, when you hadn't even established that it sounded different. When I pointed out that I own a DAC with a high end Sabre chip and I have tested it and determined it to be audibly transparent, you blew right past it and continued talking about filters and "digital glare" and how lots of people say Sabre chips sound harsh. None of that is based on anything except words and paragraphs and posts in internet forums repeating "common knowledge" without any personal knowledge.

I read people's posts here and I try to figure out if they know what they're talking about. If they do, that is great because I can learn from them. I consider the source. There are people here whose posts I don't read past the first line or two because it just isn't worth my time. That's fine. We should all be filtering what we are presented with on the internet. It isn't all useful.

 

[Jaywalk3r]

Thank you... that's what I've been trying to get across for some time.

Keith, you have, by far, committed the most egregious lapses of logic in this thread.

 

[Phronesis]

I think the disagreements arise because some posters don't understand that in the event there is not sufficient evidence to reject the null hypothesis, the only alternative is to not reject the null hypothesis. That doesn't mean that the researcher no longer suspects the research hypothesis to be true, it just acknowledges that data observed would not be unlikely if the null hypothesis is true, and that further investigation will be required before the null hypothesis can be rejected justifiably.

Failure to reject the null hypothesis is not the same as embracing the null hypothesis.

I don't think you understood my point. We're not talking about a formal study and formal language like "null hypothesis," we're talking about what we believe or speculate may or may not be true, and the basis for that, in the absence of properly conducted studies.

And even when you talk about formal studies, the level of evidence or statistical significance needed to reject the null hypothesis is still a matter of judgment (e.g., cutoff p-value of 0.05 is arbitrary), and sometimes studies give misleading findings because they weren't properly designed or conducted or interpreted.

A lot of people who talk about "science" in this forum don't seem to really understand how actual science works.

 

[Phronesis]

I haven't said that. What I've said is... Of the dozens of DAPs and DACs I've owned in the past couple of decades, I have tested every one of them and determined them to be audibly transparent- meaning there is no difference between them. I asked for someone to point me to one that wasn't audibly transparent, so I can know that colored DACs exist.

You pointed me to a DAC, but it was a lousy recommendation because you made no effort to determine if it really was colored. If I wasted my time chasing down every biased "impression" in Head-Fi, I would be spinning my wheels until the cows come home, and I'd get no closer to finding a colored DAC. The reason I ask here in Sound Science for examples of colored DACs is because I assume that people here know how to filter bias and conduct a halfway decent comparison test. I was wrong to think that you would do that. Point taken. Source considered from now on.

Have you considered that other people may be able to hear differences which you can't? It's not scientific to assume that your particular ears/brain serve as a reference benchmark for audible "transparency."

 

[KeithEmo]

Interesting....

I performed a rather informal test...
And you did no testing on either of those DACs whatsoever...

And, not to nit pick, but did you actually test all those DACs against each other?
Or did you compare two or three to each other at a time?
And how many different speakers, headphones, and amplifiers did you use?
(It seems to me that you tested a very tiny percentage of the overall number of products available.)
And, by the way, how many test subjects were involved in each run?
And what was their age spread?
And what was the provenance of your test content?
(Please note that I'm not belittling your efforts.... and I would even consider them to be "somewhat suggestive".... but they hardly constitute a conclusive test.)

I did mention my reason for making that specific suggestion - which was based on my subjective impression.
My subjective impression was that the difference was very obvious, which would seem to make those two DACs excellent candidates for testing.
(It would hardly make sense to test two DACs which I thought sounded about the same.)
However,I didn't actually ask anyone to accept my claim as fact.
I simply made a suggestion about a good unit to test... which was what you asked me to do.
You seem determined to reject any recommendation about running any sort of experiment unless someone else has already run the experiment.
To me, that doesn't seem like a good way to learn anything new...

I haven't said that. What I've said is... Of the dozens of DAPs and DACs I've owned in the past couple of decades, I have tested every one of them and determined them to be audibly transparent- meaning there is no difference between them. I asked for someone to point me to one that wasn't audibly transparent, so I can know that colored DACs exist. You pointed me to a DAC, but it was a lousy recommendation because you made no effort to determine if it really was colored.

 

[Jaywalk3r]

And even when you talk about formal studies, the level of evidence or statistical significance needed to reject the null hypothesis is still a matter of judgment (e.g., cutoff p-value of 0.05 is arbitrary), …

Alpha is semi-arbitrarily chosen, but it's fixed once the data are observed. That's the important bit. It can't be second-guessed after the analysis. There's plenty of reason to use a large alpha value, such as 0.1, or a small value, like 0.000001. Sometimes the purpose of a study is simply to see if further research is justified. A small sample and large alpha would be warranted, usually to keep costs low. At the other extreme, if your research will modify our understanding of the universe by modifying physical laws, then a large sample and tiny alpha is warranted.

… and sometimes studies give misleading findings because they weren't properly designed or conducted or interpreted.

That's not the fault of the process. It's the fault of the researchers. All research conclusions are conditioned on the research assumptions being, in fact, true.

 

[bigshot]

I performed a rather informal test..
And you did no testing on either of those DACs whatsoever...

I'm beginning to lose faith in what you say, I'm afraid. If I had more confidence, I'd actually be interested in discussing this kind of stuff with you.

I do level matched, direct A/B switched blind comparisons. I only do sighted to verify what I already know. I always use my iPod classic as my reference. Everything is compared against that. As I already told you, I use my Oppo PM-1s because speakers introduce too many variables of their own. I don't intend to publish in the AES. The tests I do are for my own purpose, which is to determine if something is transparent in normal use. So ! didn't do a full bells and whistle scientific test, but at least I make an effort to eliminate bias.

My advice is to demand accuracy from your own tests and your on conclusions first. That way, you'll recognize when someone else has done a decent job or not. I think at this point your ego is bruised and you're grabbing at straws. I don't need to go any further. I know what I need to know now. I'll move on to other posters.

 

[Jaywalk3r]

Have you considered that other people may be able to hear differences which you can't? It's not scientific to assume that your particular ears/brain serve as a reference benchmark for audible "transparency."

Um … You just diverted from a scientific approach.

Bigshot's research responsibilities extend only to being open to the possibility that if an audible difference exists, he's open to the possibility of being among the humans able to hear it.

You, like Kieth, are confusing "I require evidence before believing humans can actually hear sounds which physiological knowledge suggests are below the threshold of human audibility" with "It's impossible for anyone to hear audible differences between DACs." They are not equivalent statements. Nor would one investigate the claims in the same manner.