Imo, if you you are happy with the current sound signature the best thing you can do would be to upgrade your source/amp and perhaps a recable; changing the grill is a nice superficial mod, but doesn't affect the sound quality too much imho.
I found the best way to loosen the glue off was to cover the drivers with cling wrap and dip them in about 1/2 inch of boiling water in a shallow pan and leave them for ~10mins.
If you are going to make the holes, start with 1 hole on one cup and A/B compare them to the other cup, then if you like the change maybe make 2 and A/B again, etc, until you are happy with the sound. But again, I found it made the bass slightly bloated, if you are after a tigher bass I would not bother with making the holes.
Thanks for the tip! So I'm guessing even just 1 to 2 holes would ruin the tight bass. How about your opinion on damping the back of the driver?