You'll definitely lose out on gaming features. Only the X-Fi products with the EMU20k1 (PCI) or EMU20k2 (PCIe) DSPs tend to have proper EAX 5 and CMSS-3D Headphone support, and they're all internal cards. (I have no idea how the X-Fi Go! Pro can claim the former; it's probably not hardware-accelerated, but driven in software.)
For music listening, though, the X-Fi HD USB is probably still competent.
If you're concerned about the sound card blocking airflow to the GPUs, then maybe you could try and find an Auzentech X-Fi Forte, where the low-profile form factor can help a bit with not blocking off airflow...but if even that still unnerves you, it seems like you'd have no choice but to go external.
Creative needs to have a proper external audio device with great sound quality (which the X-Fi Go! Pro does NOT deliver from what I've read), EAX 5, and CMSS-3D Headphone all in one by now...