From the OP - which states that the problem is fixed by a cheap ground loop isolator transformer, and nothing else - it is already quite clear that the noise is indeed caused by a ground loop. Another simple test is to connect the headphones directly to the sound card (which could possibly drive the headphones fine in the first place).
Thanks for clarifying. Good to know in case I have to troubleshoot, too. have a Little Dot I+ getting here this week. I knew that the Little Dots are prone to ground loop problems, but am hoping I don't have to deal with it.