What's your CPU and video card? Certain models are easier to cool quietly than others.
I found that there wasn't one single "loud" fan in my box. Every time I knocked off the loudest, I just started getting annoyed by the second loudest...
I got a mondo passive heatsink for my prescott, and one of these for my overclocked nvidia 6800gt:
http://www.arctic-cooling.com/vga1.php
... it ducts out the back of a PCI slot, instead of recirculating hot air back into the case.
They also sell passively cooled power supplies now. And before buying new case fans, trying getting some of those cheap inline resistors that lower the voltage first. A lot of fans are almost inaudible at 6-7V. Motherboard fans are usually louder than snot, because they're cheap, and passive heatsinks work real well on those chips. Any by all means, don't do this without a temperature probe handy. Those chips can overheat *fast* if you go overboard.
Once your fans are quiet, if you have a loud hard drive (that bearing ringing), those noise dampening boxes for disks work pretty well if the drive isn't overly hot. Forget it if you've got 10k+ drives.
Water cooling is also potentially very silent. Especially if you locate the pump in a different room. Haven't quite gotten up the energy to play with it...