Is it just the Hugo 2 that does not have the problem? I think one thing you're not taking into account is the Hugo 2 runs off it's own internal battery, and does not take power from the pc. (It's supposed to have galvanic isolation.) I think you can rule out RFI so I don't think you're going to need ferrite clamps. If it was it would affect the Hugo as well.
Another thing to check, do you have any powered usb hubs connected to your pc? I've found that on most budget friendly motherboards, the power from a cheap (or sometimes even expensive) usb hub can feed back into the usb controller. (They have no isolation at all). If you do, try unplugging this from your pc and see if the problem persists. (Personally I would unplug each of your USB cables from the pc one at a time while the sound is going, as a bad USB cable can also feedback into the USB bus. With a resolving enough system you'll hear the noise.)
Lastly, you can try to get a decent pci add-in card. There are some that have their own controllers (some even have their own controller per port. I used these for passthrough to VM's, very handy.) This will take you off the motherboards usb controller, which can also be introducing noise into your signal. The one you purchased might not have had it's own controllers. The one I use is
https://www.amazon.com/gp/product/B00XPUHO10/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 . (I would not normally recommend this as this card is rediculous overkill for just a usb dac. )
I highly doubt the type of usb connector has anything to do with it.
What model is your motherboard? What kind of power supply is in your pc?