In theory it might under certain circumstances. Without dither we’re left with quantisation error, in virtually all cases that would be inaudible at reasonable listening levels. However, it would be possible to create a signal and create a type of quantisation error called “truncation error” that would be borderline audible given a very loud (just about “reasonable”) listening level and an excellent listening environment. The chances of that type of signal existing in a music recording and those other conditions all aligning are probably somewhere between none at all and extremely remote.So without dither 24bit would sound superior to 16bit?
However, your question is hypothetical. In almost all cases where quantisation error would occur, dither is applied automatically and cannot be disabled (in ADCs and DACs for example). The only place it could occur is when manually reducing bit depth but then of course you should manually apply dither when doing so.
G