Assuming that your amp works like mine does, I see no reason to remove the headphones before switching off.
Mine makes absolutely no noise ("thud" or similar) like some amps do when turning off. None whatsoever.
And when turning on, the auto-mute feature silences the output for 10 seconds when the power is turned on. Unless you have the volume setting at something other than full off, when it keeps the output silent until you turn down the volume, and then up again.
I've only had the amp for a month, and only used it with one set of headphones (Sennheiser HD-540 Reference, 300 Ohm model of the first series), but so far I'm pretty impressed. Very quiet (no noise - very high S/N ratio), mechanically well built, does not seem to remove or add anything to the music - kind of "dissapears" from the chain. Effortless, if you will. Good range of inputs, and pre-out - I use mine also as a control amp for my active monitors (small Genelecs, the 6010A). It is not inexpensive, but for me it is worth the price, and I say that being a pretty frugal person....