You've got two things to attack with the caps - supply impedance and droop. Really they are sort of the same thing, just different frequencies and slightly different causes and effects.
For high-frequencies the little wires you hook up your circuit with look like inductors and thus start to have a significant impedance (i.e. much higher than your battery or regulator). This can cause cross-talk, or worse, instability in the amplifiers. This is why you should put small (1uF, 0.1uF something like that) at every power pin of every amp in the circuit. You essentially hide the wires from the amp by putting tiny capacitors right next to it. They don't need to be big because all they are doing is supplying current while the inductance of the wires is charging - which happens pretty fast.
At low frequencies the wires don't look very inductive, so you tend to see your regulator or battery's impedance. This can cause the supply rail to droop. Again, not a great thing. Since the low frequency signals are around for a long time and can be sucking lots of current you need a lot of capacitance to supply it. But, you can pretty much throw it wherever (most people put it right where the power enters the board) and you call parallel lots moderate sized ones instead of just one big one. Sometimes that is even prefereable. The main issue is having enough total capacitance there.
