[KeithEmo]

   
In the absence of proper evidence actually supporting a claim of difference it is most logical to not believe that claim rather than to believe it. Thus the burden of proof is on those claiming (hypothesizing) a positive difference between two conditions to provide compelling evidence for it. If such bodies choose not to, for whatever reason, we do not have to take their claims at all seriously, if they cannot provide strong evidence for their claims. Anecdotes do not count as evidence.

 
So, in other words, if I were to suggest that dog feces and filet mignon probably taste differently, you would maintain that, since we haven't seen any actual taste comparisons between the two, they probably taste the same? Really? You believe that, logically, a 16/44k file and a 24/96k file "must" sound the same because there's nothing missing from the 16/44k file to begin with - and so nothing that the 24/96k file can possibly do better. Yet someone else, equally logically, believes that a file that contains more than twice as much information "must" logically sound different. (While we're at it, I guess we can safely assume that I can paint as well as Rembrandt - until someone actually stages some sort of comparison to make sure (and, after all, I can use the same paint colors he did).
 
Are you beginning to see the problem? What you DIDN'T consider is that the way we humans do things is that we make assumptions based on our own personal prior experience and the base-line information we've decided to accept as true, then we "require" "strong evidence" to support any claims that disagree with those base-line assumptions. This works fine most of the time - as long as we always remember which pieces of "knowledge" really are, and which are just assumptions.
 
Five hundred years ago we'd be having this argument - about whether I could prove the Earth wasn't flat. Fifty years ago, if someone tried to sell you a pill that, when mixed with tap water, could run your car for a year, you knew it was a scam - because a pill couldn't possibly contain enough power to run a car for a year. But I wonder if, a hundred years from now, there will be cars that run on atomic power. (And I wonder if the fuel pellet they run on will be any bigger than "the pill that couldn't possibly exist". I'm guessing that a chunk of plutonium the size of a quarter just might be able to run a car for a year.) Quite a while ago it was "accepted fact" that vinyl was "virtually indistinguishable from the original. Then a certain tape company was certain you couldn't tell the difference between something recorded on their cassettes and reality. Then, when CDs came out, a bunch of people were just as totally convinced that they were "audibly perfect". Now, oddly, a lot of people seem to be claiming that CDs fall a bit short, and we "really need 24/96k". (Also, as for credibility, and "ulterior motives", it seems like many of the people pushing for high-def files today are the very same companies who were pushing for accepting that CDs were "good enough" when they came out.)
 
It happens to be my personal philosophy that, when I buy a copy of something I care about,  I always get the best quality version I can afford (within reason) - because it's been my experience that my needs change. (Most of the GIFs that "looked pretty good" on my old monitor fall a bit short on my current one, I never succeeded in making a cassette copy that I couldn't tell from the original, and I definitely hear things with my new electrostatic headphones that I didn't notice at all with my dynamics. I truly pity the folks who own several thousand songs in MP3 or AAc format, and now find themselves having to buy them all over again after finally noticing the difference between what they have and CD quality audio. Therefore, given the choice, I'm always happy to pay an extra $5 for a 24/96 copy of something for "insurance"). Since storage space is cheap, and everything I own can play either version, I don't agonize over whether I can hear a difference at the moment or not - I just assume that, IF there is an audible difference, the 24/96k version will be the better one. And I'm also willing to concede that, when I do hear an obvious difference, it's just possible that "the evil music company" deliberately made the 16/44k version sound a little worse so I'd buy the "better" one. However, even if that's the case, I'd still rather have the better one. (We must always keep in mind that the discussion here is not whether it makes sense for the music industry to sell high-res downloads or not; the question is whether it makes sense for us as consumers to buy them.)

 

[KeithEmo]

   
 
The most probable reason ASIO does not sound better than WSASPI is that there is no technical difference sufficient to cause them to even sound different.
 
For something to sound better or worse, it must first sound different.

 
I agree - but apparently many people do not

 

[goodyfresh]

   
I agree - but apparently many people do not

I can't hear a difference at all between ASIO, Kernel Streaming, or WASAPI. . .all bit-perfect output methods sound the exact same to my ears

So the real question is. . .which one gives the best performance in terms of computer processor load, latency, etc.?  ASIO vs. Kernel Streaming vs. WASAPI?

 

[KeithEmo]

   
There have been many such tests that give reliable evidence supporting the "no differences" hypothesis. How many tens of thousands of such tests will you demand?

 
I'll settle for one test, or an aggregate of several, that:
 
1) include several thousand (or at least several hundred) participants
2) include data about the hearing acuity of the participants (showing that at least some of them have hearing "at the top end" of "human hearing ability")
3) include playback speakers or headphones that are good enough to not obscure any potential difference
(I can provide a list of electrostatic headphones that retail for below $1k that are good enough; otherwise we need to at least use a wide variety of types)
4) include other playback equipment that, similarly, won't obscure any potential difference
(again, we need to use one amplifier that everybody agrees is "good enough", or a wide variety to cover the major possibilities)
5) include a variety of source material (perhaps we should let the folks claiming to hear a difference choose their own)
(remember that, if even one example exists where a difference is audible, then we have proven the case that such differences exist)
 
In general, all of the tests I've seen published were conducted using too few participants, an insufficient variety of source material, test equipment that was either not what I would consider "good enough" or wasn't specified at all, and experimental protocols that were dubious (or, again, not specified). There is also the "little issue" about whether the test protocols themselves are appropriate, complete, and correct.
 
For example, the hearing tests that "prove" that we can't hear above 20 kHz were almost certainly conducted as a standard "audiologist test" - where a tone is played through a headphone and the subject reports whether they hear it - usually by raising a hand or pressing a key. However, that test doesn't show whether, even though not audible as sound, a continuous 22 kHz tone will give listeners a headache after an hour, or whether a 22 kHz tone that is switched on and off, or occurs as a periodic type of distortion, will cause an audible effect because it alters the sensitivity of our hearing at other frequencies that are audible when it's playing, or even whether bandwidth limiting a sample of music to 20 kHz, even though it doesn't produce what we perceive as a difference in sound, may or may not shift the apparent position of instruments in the sound field. The fact that we don't perceive 22 khz as "sound" in no way rules out those other possibilities. (I mentioned a few of those in particular because there have been tests which claim to prove that some of them may in fact be true.)
 
Most human experience is a continuum. For example, "you can't see light at a frequency of 850 nm" (that's the frequency used by many infrared remote controls). However, that statement is in fact not true. In reality, many people can see light at 850 nm IF IT'S BRIGHT ENOUGH. (You won't see the light on your remote control's LED, but the same frequency of light from an IR LASER source, which is thousands of times more intense, is visible as a faint pink dot.) Likewise, there is a ring-tone some people call "a bumblebee tone" - it is at a very high frequency that most teenagers can hear, but most adults cannot (which makes it popular amongst students who wish to avoid having a teacher hear when their phone rings in class). Now, clearly, whether a "test" would "prove" that bumblebee tone is or is not audible would depend on the age of the participants chosen (and, if you were to conduct that test at an AES meeting, unless at least some of the members happened to be teenagers, the results got got would probably be completely inaccurate).
 
Since nobody seems to have the inclination to spend the time and expense to run what I would consider to be "thorough tests", I guess we'll all have to decide for ourselves what level of testing is sufficient to convince each of use - one way or the other. Personally, I haven't seen enough evidence either way that I consider it to be conclusive.

 

[ktm]

No, It was MS security stuff causing the issue. I was hoping using Mcafee would stop it.
Leaving the network disconnected and using the add on to set process priority more or
less took care of it. I had to play around in XP to make that work properly.
I also had a Vista laptop I used for meets. All of them required tweaking.
Please keep in mind none of these were big bucks high performance machines.
It shouldn't require that to run a simple app like foobar.
 
Runeaudio on Raspberry Pi replaced it. It frees up the pc for other uses.
The footprint is the size of a deck of cards, and you can buy a snap on DAC.
No moving parts, no noise, low cost. Supports NAS storage as well.
 
As far as using WSASPI or ASIO, 6 of one, half dozen of the other.
They are just different ways of doing the same thing.
Kind of like Apple lossless vs. FLAC.

 

[KeithEmo]

  I can't hear a difference at all between ASIO, Kernel Streaming, or WASAPI. . .all bit-perfect output methods sound the exact same to my ears

So the real question is. . .which one gives the best performance in terms of computer processor load, latency, etc.?  ASIO vs. Kernel Streaming vs. WASAPI?

 
I agree - under most circumstances. Most USB DACs these days use asynchronous USB mode, which means that the DAC provides the clock used by the incoming data, which means that the DAC is largely "immune" to jitter on the incoming signal. However, a lot of very low cost DACs still use the older isochronous mode, and even a few high-end DACs use it for various reasons. The point there is that, if you happen to have a DAC that DOESN'T use asynchronous mode, then the differences between different amounts of jitter may in fact be audible on that DAC, and the various types of drivers may in fact produce very different amounts of jitter (so there may be an audible difference between them on certain DACs).
 
The other thing is that your question is a slight oversimplification. Every method of getting sound out of Windows requires drivers. The drivers to support UAC1 are built into both Windows and Apple computers; Windows includes UAC1 drivers, but requires separate drivers for UAC2 (Apple computers have UAC2 support built in); and ASIO always uses separate drivers. Therefore, the question isn't simply about which method gives the best performance, but about whose drivers offer the best performance with which method. (Drivers written by different companies may offer very different performance, and one company's ASIO drivers may be better, while the WASAPI support in another company's UAC2 drivers may be better, and they may both perform entirely differently if you use a different version of Windows (or Apple OS/X).
 
Therefore, when you make direct comparisons, you need to specify the operating system, the device itself, and the drivers you're using (even different versions of the same driver, from the same company, may perform differently).

 

[goodyfresh]

   
I agree - under most circumstances. Most USB DACs these days use asynchronous USB mode, which means that the DAC provides the clock used by the incoming data, which means that the DAC is largely "immune" to jitter on the incoming signal. However, a lot of very low cost DACs still use the older isochronous mode, and even a few high-end DACs use it for various reasons. The point there is that, if you happen to have a DAC that DOESN'T use asynchronous mode, then the differences between different amounts of jitter may in fact be audible on that DAC, and the various types of drivers may in fact produce very different amounts of jitter (so there may be an audible difference between them on certain DACs).
 
The other thing is that your question is a slight oversimplification. Every method of getting sound out of Windows requires drivers. The drivers to support UAC1 are built into both Windows and Apple computers; Windows includes UAC1 drivers, but requires separate drivers for UAC2 (Apple computers have UAC2 support built in); and ASIO always uses separate drivers. Therefore, the question isn't simply about which method gives the best performance, but about whose drivers offer the best performance with which method. (Drivers written by different companies may offer very different performance, and one company's ASIO drivers may be better, while the WASAPI support in another company's UAC2 drivers may be better, and they may both perform entirely differently if you use a different version of Windows (or Apple OS/X).
 
Therefore, when you make direct comparisons, you need to specify the operating system, the device itself, and the drivers you're using (even different versions of the same driver, from the same company, may perform differently).

See I'm trying to figure out which output mode is best to use in Foobar2000 to play music with my Fiio X3ii as a USB DAC.  The Fiio X3ii is asynchronous as a DAC, and uses proprietary ASIO drivers in its default mode for getting input from the PC. I can't seem to find info on whether Kernel Streaming or WASAPI will give better performance with the Fiio X3ii as a DAC than the default ASIO drivers.
 
Then, I am also planning to get a HiFiMeDIY UAE23+ asynchronous, power-isolated DAC sometime soon.  It is plug-and-play, doesn't necessarily require any drivers to work with Windows, but there are optional ASIO drivers available for it (specifically for the Sabre ES9023 chip in it) that supposedly allow for better tweaking of performance, SUPPOSEDLY.  Once I get it, what should I do with it. . .just use it Plug-and-Play, and if so, with Kernel Streaming or with WASAPI. . .or, should I use it with the optional ASIO drivers?  Am I correct that if simply using it Plug-and-Play with no extra downloaded drivers, then Kernel Streaming will give the lowest latency, while the actual sound will be the same as WASAPI?

 

[arnyk]

 
See I'm trying to figure out which output mode is best to use in Foobar2000 to play music with my Fiio X3ii as a USB DAC.  The Fiio X3ii is asynchronous as a DAC, and uses proprietary ASIO drivers in its default mode for getting input from the PC. I can't seem to find info on whether Kernel Streaming or WASAPI will give better performance with the Fiio X3ii as a DAC than the default ASIO drivers.
 
Then, I am also planning to get a HiFiMeDIY UAE23+ asynchronous, power-isolated DAC sometime soon.  It is plug-and-play, doesn't necessarily require any drivers to work with Windows, but there are optional ASIO drivers available for it (specifically for the Sabre ES9023 chip in it) that supposedly allow for better tweaking of performance, SUPPOSEDLY.  Once I get it, what should I do with it. . .just use it Plug-and-Play, and if so, with Kernel Streaming or with WASAPI. . .or, should I use it with the optional ASIO drivers?  Am I correct that if simply using it Plug-and-Play with no extra downloaded drivers, then Kernel Streaming will give the lowest latency, while the actual sound will be the same as WASAPI?

 
Don't worry about it unless you hear severe problems. Based on the many measurements and listening tests that I've made with USB DACs both asynch, adaptive and isosynchronous, there are many options that net out to measure the same and sound the same.  

 

[Music Alchemist]

  So, in other words, if I were to suggest that dog feces and filet mignon probably taste differently, you would maintain that, since we haven't seen any actual taste comparisons between the two, they probably taste the same? Really? You believe that, logically, a 16/44k file and a 24/96k file "must" sound the same because there's nothing missing from the 16/44k file to begin with - and so nothing that the 24/96k file can possibly do better. Yet someone else, equally logically, believes that a file that contains more than twice as much information "must" logically sound different. (While we're at it, I guess we can safely assume that I can paint as well as Rembrandt - until someone actually stages some sort of comparison to make sure (and, after all, I can use the same paint colors he did).
 
Are you beginning to see the problem? What you DIDN'T consider is that the way we humans do things is that we make assumptions based on our own personal prior experience and the base-line information we've decided to accept as true, then we "require" "strong evidence" to support any claims that disagree with those base-line assumptions. This works fine most of the time - as long as we always remember which pieces of "knowledge" really are, and which are just assumptions.
 
Five hundred years ago we'd be having this argument - about whether I could prove the Earth wasn't flat. Fifty years ago, if someone tried to sell you a pill that, when mixed with tap water, could run your car for a year, you knew it was a scam - because a pill couldn't possibly contain enough power to run a car for a year. But I wonder if, a hundred years from now, there will be cars that run on atomic power. (And I wonder if the fuel pellet they run on will be any bigger than "the pill that couldn't possibly exist". I'm guessing that a chunk of plutonium the size of a quarter just might be able to run a car for a year.) Quite a while ago it was "accepted fact" that vinyl was "virtually indistinguishable from the original. Then a certain tape company was certain you couldn't tell the difference between something recorded on their cassettes and reality. Then, when CDs came out, a bunch of people were just as totally convinced that they were "audibly perfect". Now, oddly, a lot of people seem to be claiming that CDs fall a bit short, and we "really need 24/96k". (Also, as for credibility, and "ulterior motives", it seems like many of the people pushing for high-def files today are the very same companies who were pushing for accepting that CDs were "good enough" when they came out.)
 
It happens to be my personal philosophy that, when I buy a copy of something I care about,  I always get the best quality version I can afford (within reason) - because it's been my experience that my needs change. (Most of the GIFs that "looked pretty good" on my old monitor fall a bit short on my current one, I never succeeded in making a cassette copy that I couldn't tell from the original, and I definitely hear things with my new electrostatic headphones that I didn't notice at all with my dynamics. I truly pity the folks who own several thousand songs in MP3 or AAc format, and now find themselves having to buy them all over again after finally noticing the difference between what they have and CD quality audio. Therefore, given the choice, I'm always happy to pay an extra $5 for a 24/96 copy of something for "insurance"). Since storage space is cheap, and everything I own can play either version, I don't agonize over whether I can hear a difference at the moment or not - I just assume that, IF there is an audible difference, the 24/96k version will be the better one. And I'm also willing to concede that, when I do hear an obvious difference, it's just possible that "the evil music company" deliberately made the 16/44k version sound a little worse so I'd buy the "better" one. However, even if that's the case, I'd still rather have the better one. (We must always keep in mind that the discussion here is not whether it makes sense for the music industry to sell high-res downloads or not; the question is whether it makes sense for us as consumers to buy them.)

 
The only thing that matters is the master. If it is in a higher resolution but is an inferior master, it won't sound as good. Some hi-res downloads definitely sound better than the CD version, but only because it's a better master. When you convert the hi-res files to Red Book yourself, there is no audible difference. It is physically impossible to distinguish between Red Book and hi-res unless there is a problem in your system, in which case it would produce distortion, not music. The reason for this is simple: 16-bit already has more than enough dynamic range for all recordings, so 24-bit adds nothing audible. 44.1 kHz files are designed to play all the frequencies humans can hear, so higher sample rates add nothing audible. If you haven't already, I strongly recommend reading this article that I link to repeatedly. It goes into great detail on the technical stuff. Also, as far as I know, no one has ever been able to tell 256 kbps AAC apart from lossless. The main reason many lossy downloads sound inferior is because, again, they use an inferior master. I would be extremely impressed if you could distinguish between 256 kbps AAC and 16-bit lossless under controlled conditions.

 

[goodyfresh]

   
Don't worry about it unless you hear severe problems. Based on the many measurements and listening tests that I've made with USB DACs both asynch, adaptive and isosynchronous, there are many options that net out to measure the same and sound the same.  

Thanks for the tip!

   
The only thing that matters is the master. If it is in a higher resolution but is an inferior master, it won't sound as good. Some hi-res downloads definitely sound better than the CD version, but only because it's a better master. When you convert the hi-res files to Red Book yourself, there is no audible difference. It is physically impossible to distinguish between Red Book and hi-res unless there is a problem in your system, in which case it would produce distortion, not music. The reason for this is simple: 16-bit already has more than enough dynamic range for all recordings, so 24-bit adds nothing audible. 44.1 kHz files are designed to play all the frequencies humans can hear, so higher sample rates add nothing audible. If you haven't already, I strongly recommend reading this article that I link to repeatedly. It goes into great detail on the technical stuff. Also, as far as I know, no one has ever been able to tell 256 kbps AAC apart from lossless. The main reason many lossy downloads sound inferior is because, again, they use an inferior master. I would be extremely impressed if you could distinguish between 256 kbps AAC and 16-bit lossless under controlled conditions.

The main reason I like to keep my music library as FLAC files rather than 320kbs MP3's or something, is so that it will be future-proof for further file-conversions no matter what new technology arises in the coming years or decades.

 

[Music Alchemist]

  The main reason I like to keep my music library as FLAC files rather than 320kbs MP3's or something, is so that it will be future-proof for further file-conversions no matter what new technology arises in the coming years or decades.

 
Well, it would be reckless to delete the original lossless files. When you convert to lossy, it creates new files and does not delete the originals unless you do that yourself or program the software to do so. I just leave all my files as-is, since I have practically unlimited hard drive space. The main reason to convert to lossy is to be able to fit more files on portable devices. By the way, AAC is superior to MP3 in that it can be compressed more without losing quality.

 

[KeithEmo]

 
See I'm trying to figure out which output mode is best to use in Foobar2000 to play music with my Fiio X3ii as a USB DAC.  The Fiio X3ii is asynchronous as a DAC, and uses proprietary ASIO drivers in its default mode for getting input from the PC. I can't seem to find info on whether Kernel Streaming or WASAPI will give better performance with the Fiio X3ii as a DAC than the default ASIO drivers.
 
Then, I am also planning to get a HiFiMeDIY UAE23+ asynchronous, power-isolated DAC sometime soon.  It is plug-and-play, doesn't necessarily require any drivers to work with Windows, but there are optional ASIO drivers available for it (specifically for the Sabre ES9023 chip in it) that supposedly allow for better tweaking of performance, SUPPOSEDLY.  Once I get it, what should I do with it. . .just use it Plug-and-Play, and if so, with Kernel Streaming or with WASAPI. . .or, should I use it with the optional ASIO drivers?  Am I correct that if simply using it Plug-and-Play with no extra downloaded drivers, then Kernel Streaming will give the lowest latency, while the actual sound will be the same as WASAPI?

 
Kernel Streaming may or may not be the lowest latency, but, when you use Kernel Streaming (DS), Windows WILL resample everything you play to the default sample rate you set in Control Panel. (In that mode, the only way to get Windows to play high-res files at their native sample rate is to go into Control Panel and change the default rate to match the file you're playing.) Therefore, it is usually considered to be the lest desirable mode by most audiophiles. Between ASIO and WASAPI I would probably go with whatever the DAC manufacturer recommends. You also need to make sure that the player program you want to use supports it (not all support both). I personally find WASAPI to be easier to configure, and more widely supported, but I've seen a few DACs whose drivers support ASIO and not WASAPI.
 
For a DAC to work plug-and-play in Windows (without external drivers) it must be running in UAB Audio Class 1 (UAC1) - which means that it can only support up to 24/96k. In order to support 24/192k or higher in Windows, the DAC must use UAC2, which means you need to install drivers. (Some DACs are ONLY UAC1, and so only support up to 24/96k, but never need drivers; some DACs are ONLY UAC2, and so support 24/192k and above, but need external drivers for Windows; and some DACs can run in UAC1 without drivers, or in UAC2 with drivers. (Note that this all pertains to Windows PC's; current Apple computers have the drivers built in for both UAC1 and UAC2.
 
For Raspberry Pi fans, note that most versions of Linux support UAC1, and many also support UAC2. Also note that there is a free program called ASIO4ALL - which is what they call "an ASIO wrapper" - and which allows some programs that only support ASIO to connect to many devices that are not themselves ASIO (think of it as "an ASIO adapter").
 
 

 

[arnyk]

   
Kernel Streaming may or may not be the lowest latency, but, when you use Kernel Streaming (DS), Windows WILL resample everything you play to the default sample rate you set in Control Panel. (In that mode, the only way to get Windows to play high-res files at their native sample rate is to go into Control Panel and change the default rate to match the file you're playing.) Therefore, it is usually considered to be the lest desirable mode by most audiophiles.

A lot of smart people find out what audiophiles believe and do, and then run the other way. This phobia about resampling is one reason why. Not all resampling is done well. At one time I was on Creative Lab's "Bad boys" list because I found that the resampling in some of their earliest audio interfaces was substandard, proved it with relevant technical tests, and wasted no time telling the world. However, resampling is not rocket science and doing it right is often taught to fledgling DSP programmmers and the like. This is a classic example of how a problem of over 10 years ago resonates through the audiophile community long after it is solved. One might wonder why anybody would spread such irrelevant old news so long after it is relevant?

 

[KeithEmo]

A lot of smart people find out what audiophiles believe and do, and then run the other way. This phobia about resampling is one reason why.Not all resampling is done well. At one time I was on Creative Lab's "Bad boys" list because I found that the resampling in some of their earliest audio interfaces was substandard, proved it with relevant technical tests, and wasted no time telling the world. However, resampling is not rocket science and doing it right is often taught to fledgling DSP programmmers and the like.This is a classic example of how a problem of over 10 years ago resonates through the audiophile community long after it is solved. One might wonder why anybody would spread such irrelevant old news so long after it is relevant?

 
As you say, not all resampling is done well. Many folks also seem to agree that the resampling done by Windows itself is not amongst the best. Beyond that, resampling is a form of interpolation, which means that there are various different ways of doing it, and they do yield slightly different results. (Let's avoid another lengthy dispute about whether the differences are or are not audible; suffice it to say that there are different ways of going about it, and different opinions about which does a better job...)
 
However, since most DACs can handle all the standard sample rates perfectly well, isn't it simpler to avoid the question the easy way - and NOT resample the audio?
(Assuming that your DAC can play the file at its native sample rate, why go out of the way to alter it, when doing so might affect the quality negatively, and serves no positive purpose at all.)
 
 
 

 

[nick_charles]

   
I'll settle for one test, or an aggregate of several, that:
 
1) include several thousand (or at least several hundred) participants
2) include data about the hearing acuity of the participants (showing that at least some of them have hearing "at the top end" of "human hearing ability")
3) include playback speakers or headphones that are good enough to not obscure any potential difference
 
What parameters will be needed to judge the adequacy and what are the required values for those parameters. Speakers and headphones as you know are generally far worse in terns of distortion, noise and uneven frequency response than competent solid state kit kept within their limits, even good headphones can have up to or more than 1% distortion which would be regarded as unacceptable on lots of items.
 
(I can provide a list of electrostatic headphones that retail for below $1k that are good enough; otherwise we need to at least use a wide variety of types)
 
4) include other playback equipment that, similarly, won't obscure any potential difference
(again, we need to use one amplifier that everybody agrees is "good enough", or a wide variety to cover the major possibilities)
 
Given the massive drag factor of the headphones the rest of the kit unless it is pretty terrible will have a relatively minimal effect, certainly it is trivial to find sources with noise/distortion levels that would not affect CD playback but are you insisting that the kit be capable of the theoretical 144db dynamic range that 24 bits gives you , this is as far as I am aware is not presently possible and certainly no amp I am aware of has noise levels that would not effectively degrade this down to well under 144db and then to take advantage of this extraordinary dynamic range would require conditions so extreme as to be impractical. So if 24 bits is out of the question where do you draw the line 23 bits, 19 bits ...?
 
 
 
5) include a variety of source material (perhaps we should let the folks claiming to hear a difference choose their own)
(remember that, if even one example exists where a difference is audible, then we have proven the case that such differences exist)
 
As above finding source material that actually exploits the dynamic range of 24 bits or even 19 bits is going to be nigh on impossible unless you create your own test material that you would need a silent room to listen to and would be rather loud at full scale?
 
 
In general, all of the tests I've seen published were conducted using too few participants, an insufficient variety of source material, test equipment that was either not what I would consider "good enough"
 
Again, what is a good working definition of good enough and how is it evaluated. The disc spinners in M&M's study hit about 110db in terms of SNR so well in excess of red book needs but well below 24 bits.
 
 
or wasn't specified at all, and experimental protocols that were dubious
 
Can you give examples, I agree protocols must be explicitly described otherwise nobody can attempt to replicate. I assume you have some formal training in experiment design that informs your criticisms?
 
(or, again, not specified). There is also the "little issue" about whether the test protocols themselves are appropriate, complete, and correct.
 
However, that test doesn't show whether, even though not audible as sound, a continuous 22 kHz tone will give listeners a headache after an hour
 
Yes inaudible sounds can cause headaches and nausea but It has to be at a very high level of intensity and over a long duration, certainly the UK and Europe have health and safety laws about such things, however unless you are deliberately adding high intensity inaudible high frequency content the point for music is moot as the level of intensity of very high frequencies is normally down on the fundamental except in a very few cases, unless you make a recording of a constant loop of keys jangling and/or cymbals crashing and amplify it to absurd levels, but if you do that the audible component will itself evoke pain anyway. Sure you could artificially lift the supersonic part but other than to prove a point , and it would constitute torture , why...