Generally, alkalines are much less prone to leakage than cheapo carbon zinc cells (which are junk anyway). Their leakage probability seems to increase with discharging - I've had to clean up some leakage from empty and long overdue (date code wise) Energizer cells in my first camera that sat unused for close to 10 years, while with an ICF-7600A (that looked like it had spent a number of years on a dusty attic, face up) I got some age old Sony branded alkalines (Made in W. Germany, these can't be much younger than 1989/1990 and were presumably bought together with the rx which was discontinued in 1988) that still had around 1.4 V left and did not show any leakage (they're now being kept in my "battery museum"), while the foam on the battery compartment door didn't fare quite as well... Recharging attempts also tend to cause leakage.
I'd say if the alkalines used are close to full and there's no battery drain in off mode (e.g. with devices that have a "hard" power on/off switch), one could leave the batteries in. Otherwise better remove them and possibly store them in the fridge.