Better use foobar's ABX component or any other proper ABX software, if you want to conduct a proper test.
Like chewy4 said, in order to ABX 16 bit vs. 24 bit, just take a 24 bit file, downconvert it to 16 bit and upconvert it back to 24 bit.
You want to set a number of trials beforehand (12 is good, 16 if you like) and stick to it. Hide the results until you're done. Save the log, look at the score: a statistically significant result (i.e. successful) is inferior to a 5% probality that you're just guessing.
But you cannot playback 16 bit and 24 bit in Foobar using WASAPI without changing the settings between tracks.. So you are suggesting truncating the 24 bit files to 16 bit to do the test?