Well, the primary reason you can't just put a large capacitor on there is that large capacitors (electrolytics in particular) generally have poor high-frequency response. So you'd have to go with ceramics or whatnot, which have small capacitance.
There are two primary concerns in building a power supply for audio: Noise and ripple.
Ripple is the 50/60Hz variation in the DC voltage caused by the original 50/60Hz VAC. You can never get rid of it entirely, but you can work towards it. A good linear regulator can provide around 60-80 dB of ripple rejection; more if you add more filter capacitors in various places. You will definately notice a 60Hz tone in your audio; this is the cause of the charateristic hum of AC transformers.
The other concern is noise; the AC line will likely contain noise from other audio equipment, people's telephone conversations, your neightbor's washing machine, computer equipment, etc. Most of this shouldn't be at audible frequencies, but you can bet some of it will be (look at a spectrum analyzer if you care to see how much is in your power). As I pointed out before, you can't really get rid of it with just a big capacitor, because it's too high-frequency.
A switching power supply will introduce noise of its own as it switches on and off, so it's right out.
An unregulated power supply assumes the input voltage is at some particular level -- definately not a valid assumption, even if you're running off batteries. And a switching power supply will introduce noise of its own as it switches on and off.
So that basically leaves you with a regulated linear supply. From there you can get better ripple and noise rejection by putting capacitors all over the place. If you're building with discrete transistors, you can probably improve ripply rejection by ensuring all nodes of each transistor are buffered. Even if you're using e.g. an LM317, you should still ensure all pins are buffered.
And don't forget the big cap on the output anyway; it helps protect you from large current swings on the load. Generally you should have a large electrolytic (preferably tantalum) capacitor and one or more ceramic capacitors to pick up high-frequency stuff.
