Well I have these kind of problems in the past, I can't guarantee my ideas will fix the problem, but they might help:
If it happens in Windows OS which probably means its the drivers for one of the hardware in the computer. Therefore try to find out if a specific or combination of thing is causing it. Try uninstalling every single drivers and slowly install the drivers one by one. Not using the setup.exe, but using Device Manager and install individual components while checking the sound after each install.
Use DPC Latency Checker while playing the music and see when the latency is bad. But this software is not reliable because it shows high latency even if your computer simply busy, but it help a little.
DPC Latency Checker
Also go into your bios and disable all extra hardwares controlled by the mobo's chipset, such as onboard ethernet, onboard ide controller, raid. Unplug all your usb devices including usb wifi card. Disable all background running softwares. Don't overclock. See if the problem go away when nothing is plugged or features turned off, then put them back in and see if any of them trigger the problem.
You can also try the driver directly from
Realtek, instead of Gigabyte. If all this don't work, lash out in anger and get a hammer and smash your motherboard into pieces.