Differences in bitdepth is something that only occasionally, or at the extremely low bitdepths you have just found, makes a hearable difference.
A higher bitdepth is important for small differences in level or waveform when there is a lot of sound in the music, like, say, the decay of a triangle in the midst of an outburst from an orchestra. That is not a common thing in music, nor is it easy to hear. I'm not so sure that could have been heard through any hifi system I've ever listened to, but I know I have heard it live.
The more important thing is headroom. For example just as already discussed, for digital volume control, but also for when something goes wrong in the digital domain. Take a look at Defiant00's excellent explanation. One lost bit of 24 is a lot less likely to make a difference you can hear than one lost bit of 16, remember that the 8 extra are a lot less significant. This means the DAC has a slightly easier job. Upping the sample rate will do the same, this is the only justification for upsampling as far as I can see, and you might have heard upsampling can sometimes be a good thing even when with the signal already on the same chip as the DAC...
Enough with the derailing. Back on topic: Applause to TMRaven for the revolutionarily short and concise review! Refreshing, to say the least. :)