Well I did an experiment. Due to the sheer size of the yggy, I needed to buy longer XLR cables so I could locate it a couple feet away from my amp (simply not enough room to stack both on my desk), so I bought some, what I thought, were high quality 6 ft XLR cables on amazon. But they're advertised to be more for use for microphones & not really specifically geared toward hifi audio. Anyway, as a test, I took my old, shorter XLR cables that are designed specifically for audio amplifier use, and I immediately noticed a cleaner sound. It's not mind-blowing mind you, but it absolutely improved the sound.
I had a sneaking suspicion it could have been those XLR cables, but again they weren't cheap, about $35 each.
So based on this, you guys have any XLR cable brands you recommend? I'm firmly in the camp that cable quality can absolutely alter sound. So I'd like to use the best quality XLR cables that this dac deserves (within reason of course). Also explains why RCA sounded better than XLR.