One method is that if you have DPDT or SPDT switch then you no longer wire the PSU direct to the amp, you break that positive PSU rail connection then connect the amp positive power rail to the common, center switch pin(s). Connect the PSU positive power rail to one end/throw of the switch pins (the "on" throw) such that when the switch is in on position it completes the circuit, amp gets power.
On the other throw of the switch, the "off" switch position, the other end switch pin(s) are connected to the amp negative rail (not power or virtual ground) through a low Ohm resistor. Since the resistor is not a continual load, current only flowing for a moment when power is turned off, it doesn't need to be a very large value, but if you had a pile of different wattages lying around you might as well use a a higher value (within reason, no need for more than 2W) rather than some little 1/4W. You might try around 10 Ohm and if the problem remains choose an incrementally lower value.
If you were going to short the caps to virtual ground instead of what is described above, with a DPDT, you'd wire the amp positive rail lead to one of the switch's center/common contacts, the amp negative rail lead to the other center contact. To the switch's "on" position end contact, wire the PSU positive on the same contact row as the amp positive. Make sure it is on the same contact row so you aren't shorting out amp positive rail with PSU negative! Use a multimeter to confirm proper wiring before trying it with power on. To the other switch "on" position contact wire the PSU negative. On the switch "off" position contacts, each contact has a small ohm resistor as mentioned above, connected to a lead going to the power ground on the amp board. You can connect both resistor ends together leading away from the switch (but not those two switch pins directly connected) to take only one wire from the resistors to the amp power ground if you like. I think one possible place to connect that wire is to a power ground hole labeled "PG". So in summary when the switch is on it then connects both of the PSU leads to the amp V+ & V- leads. When the switch is off, the amp rails each separately drain through a resistor to power ground. The key is that these resistors only connect the power rails to ground when the switch is in the off position so they aren't always using up a lot of current and getting hot, only conducting momentarily when the amp is turned off. Because of this, you can also get away with using a very small wire from the resistors to the virtual ground position as it won't carry high current for long enough to heat up significantly.
When you get it wired up, confirm with a meter resistance readings tests that at no time, on no switch throw are the PSU positive and negative ever shorted (practically 0 ohm), at no time is the PSU positive shorted to amp negative nor PSU negative shorted to amp positive, AND at no time are the amp positive and negative rails ever shorted together or to virtual ground, that they only conduct to virtual ground through the expected resistor values you added. All these things should be confirmed before you plug the PSU into the amp or turn it on. If your pimeta had a 2nd switch (like in the little alps pot) that switch should be left on for amp use and for these tests but the amp itself would not be allowed to run until you are sure the wiring is right.
There are fancier ways you could do it instead of blindly picking a resistor value if one rail is draining faster than the other. By monitoring the rails you could use a slightly lower value of resistor on the rail that doesn't drain as quickly (if they aren't close enough) or use same resistor values and add a 2nd parallel resistor to the slower draining rail so the paralleled resistor values result in nearer an equal drain. Either way you want to drain the caps quick enough that there is minimal time in an unstable voltage region but not directly shorting them so the current isn't very high.
If neither of these configurations work well enough, you might look at how much capacitance you are using and reduce that some by swapping in different caps. In theory someone could have upwards of 10,000 uF per rail (maybe more) if they just fit the biggest caps they could find but it doesn't need caps so large. IMO, 3000uF per rail might be the highest value you might need and many people find it acceptible to use nearer 1000uF per rail.
Some people might use connector shells for the "Barrel", or just a piece of round or oval plastic of high enough diameter. I think somebody makes split boots too but I don't recall who or where to buy 'em. IMO, adhesive lined, double walled heatshrink tubing is better in that it's a more durable and secure, permanent solution. I don't worry about that much though, it's not like it'll just fall apart otherwise and sometimes you may not want the glue squirting out the ends as it will inevitably do at least a little bit.