Well it seems to be related to what's around it. Source is just USB, nothing analog, so the sound quality should all be on the EF2 at this point since it's its DAC.
But, for example, if I turn off my LCD monitor, the buzzing gets more quiet, but not completely. Sometimes even with the LCD on, the buzzing will go away by itself..
I can stimulate it when it buzzes. If I give it a kick drum sound, the buzzing gets loud then quiet nearly right after the sound.. Almost like a DC filtering capacitor has failed.