This is true, but you don't need excess amounts of it. I'm no doctor, but I believe you only need a thin film of the wax to do its job of protecting the ear canal. Removing the excess will not cause harm as long as you don't go too deep and are careful about it.
I've been cleaning my ears (not on a daily basis, but fairly regularly) for long enough to know that it has caused me no problems. I've never had an ear infection and my ears don't ever get clogged. All I'm saying is that it isn't unsafe if done properly. Using some kind of chemical to completely strip the wax from your ears would be when I would assume that problems could present themselves, but not from just a regular light swab.
Anyway, that is all personal preference. Clean your ears or don't. I don't care! They aren't my ears, or my IEMs. :) Lets get back to the topic!