I use an Omni, which is the same as the Z but USB instead of PCIe. The volume is not an issue at all and the sound isn't bad, but once I put a real amp in there, the sound improved noticeably.
It's one of those things where you will not think it sounds bad until you hear it with the amp and then you'll go, "Ooooh, so that's what they meant!"
BTW, the DAC in the Omni was comparable to the Schiit Modi to my ears, so I returned my Modi. SBZ/SB Omni with an amp is a pretty solid budget setup.
Which amp did you get and is there a drawback to the Omni compared to the SBZ? I see its about $20 cheaper.