Doing upsampling on the software end during playback can have benefits, but it depends on the DAC.
Chord DACs do very advanced digital processing and are better without it, but most other DACs aren't like that. I heard that all DACs—including NOS (non-oversampling)—do at least some oversampling, but most approach it in a simplistic manner, so stacking (so to speak) could help.
HQPlayer is the best player I know of for software upsampling (and noise shaping, etc.), but it's expensive and the interface is atrocious. Bug Head Emperor / Infinity Blade is free and also has advanced DSP capability...but again, its interface isn't so good. All the relevant plugins I know of for foobar2000 are nowhere near as advanced, but may be worth trying just to see how you like the sound.