Sometimes it is the whole of the audio system...
I don't generally believe that about DACs. I will most often tell a person to get the best DAC they can, and play with the rest of the system. But even regarding DACs, there is the transport or digital audio source, digital cable, quality of the interface on both ends. And as you mentioned, how you are sending the bits from your computer source. There are so many rules I follow, I should probably write myself a cookbook.
Is the DAC powered by USB? Or is the USB interface to the DAC powered by USB? Or is nothing in the DAC dependent on USB, other than the digital bits? Anyways...glad you're liking it. And though I know it to be a competing product, you may want to pop into the REGEN thread. They truly do some mystical experimental things there!