Only my 24/192 files. Everything else is in the appropriate format. I always make sure to put it on the proper file playback manually as I don't want it to upsample anything if it doesn't have to. It does downsample on the 24/192 though but still sounds real good. I find upsampling makes things more distant. Makes the sound stage bigger which I don't like. I can see it working on an in your head IEM or can with a small sound stage. But having IEMs that present large sound stages it just doesn't sound right to me.
That's the answer then, you have resampling enabled for when you need it. Originally, I didn't, which is why I had trouble, and once I found a solution, I wanted to share it in case others have the same issue :)