If I'm not mistaken, by going into Windows Advanced Sound Settings and setting it to 24/192, that means that all sounds that come through normally (through the Windows sound mixer) will be padded to 24 bit resolution and resampled to 192kHz, so that you can play multiple sounds at the same time (which may originally be of different sample rates and bit depths). To be honest, I've listened to music using different sample rates selected in the advanced sound settings, and I'm not really sure if I can hear a difference.
As for re-setting the output each time you play files of different sample rates, normally to get around this you would try to output the digital bits through something like ASIO or WASAPI Exclusive mode, in which the bits are unaltered (and don't go through the Windows sound mixer) as they are passed through to the USB DAC (in this case, the X3ii). Thus, each time you switch between files of different sample rates, your computer through either of these modes will automatically just send the bit depth and sample rate the music file was originally encoded at. However, you will need to find a music player that lets you output the digital bits through these modes (something like Foobar2000 with its add-ons or JRiver Media Center). As far as I know, something like Windows Media Player is only able to output music through DirectSound mode, which is basically outputting the digital bits of the music through the Windows sound mixer and thus is being padded and resampled to whatever setting you set in the advanced sound settings.
Also, keep in mind that at least while playing music in WASAPI exclusive mode, the player that is playing music in that mode has "exclusive" control over the sound card or USB DAC or whatever, so no other sounds will be able to come through (such as notifications from chats, etc.).
As for how safe it is to set the latency to minimum latency, well I've only tried the different latency settings with the default ASIO buffer size of 2048 bytes, and minimum latency is mostly ok except for rare instances. So personally, I've just set it to low latency, which I find doesn't really increase latency all that much but pretty much has no problem with everything that I've thrown at it.