Actually get one of those spray cans with air in them and use that to blow away dust gently. If you blow with your breath, you will also blow moisture onto the diaphragm, since your breath always contains moisture. You wouldn't want to inadvertently damage the headphone, right? :)
Compressed air is a bad idea, too strong for sensitive diaphragm, if moisture or coffee breath :) is a concern, use a rubber syringe.