You mentioned high res file first, and then 24/96 vinyl rips, make me wonder, when you say high res, are you referring to loseless 16/44.1? or 24/96? or something in between (e.g., 24/48)??
I have a friend who own a huge vinyl library, and I have heard some of his vinyl rips, in both high bit PCM and DSD format. One thing I notice that the rips are not necessarily dead quiet to start with, so maybe that is not a good starting point to show how low the noise floor of the player is.
24-96. I have them downsampled to 16-44 on my J3 and I can't hear the hiss nearly as well - it's there, but on the X5, it's very apparent and in your face. I think it is a very revealing dap based partially on this. I've been loading more stuff onto it and am about to give it another whirl.