For the reconstruction filter (the interpolation filter in the DAC) there is only one way of doing it, and that is using an ideal infinite response sinc function - that is if your intention is to perfectly recover the original bandwidth limited analogue signal. Any other type of filter will change the sound by adding uncertainty to the reconstruction of transient timing; and this uncertainty severely damages sound quality. Timing uncertainty due to small tap length filters add a softness, warmth and a bloated bass; some actually like that sound (I do not understand why as it sounds nothing like real un-amplified music) but for sure it is a distortion, or change, from the original.
Now theory has nothing to say about how the signal is bandwidth limited; it simply states there must be zero output above FS/2. Actually, I have designed two decimation filters; one that is linear phase but rings before the impulse; one that is non-linear phase (IIR) but has no pre-ringing. Both filters have attenuation at and above FS/2 of 300 dB, so both meets the requirements of bandwidth limiting in terms of aliasing and sampling theory. I will post blind recordings with the different filters so we can all hear the difference.
I am not sure about your last question, but the decimation filter will actually work at 768 kHz. So I will have three files; 768 kHz, 768 kHz BW limited to 24 kHz using FIR, 768 kHz BW limited to 24 kHz using IIR. All files will be randomly labelled. So what we can hear is only the decimation bandwidth limiting, without the decimation. The only worry I have is that bandwidth limiting itself has a SQ benefit, due to better noise floor modulation because of the absence of HF noise, so I must ensure that Davina itself does not create extra HF noise. This is part of the project's requirements anyway.
