Just looking at your rig now, I'm wondering whether the Yamaha Receiver actually performs better as a DAC than your X-Fi. What I'd do is to do an extensive comparison using your Headfive/headphones. Price is not a terrific indication of sound quality so use your ears as a judge and not the size of the price tag.
Having done the comparison, it should be fairly easy to decide whether or not to keep the receiver based on sound quality. If money is not too much of an issue, why don't you get a modded X-Fi and test it against the unmodded X-Fi and Yamaha reciever?
I have no idea how the X-Fi (modded or otherwise) performs but USB DAC/Amps do exist. To get you started, here are three:
$137 -
Fubar II
$159 -
DIYEDEN SVDAC04
$225 -
Fubar III
All of them have RCA output so you can connect it with any active/powered speakers. The prices are reasonably affordable so you could pick one up and then compare to what you're currently listening to. It'll certainly be a much more compact and streamlined solution.