"Chronic Q-tip use creates dryness in the sensitive ear canal. Dry skin itches. People with itchy ears use Q-tips (and other found objects) to scratch them. The scratching causes itching. Do you see the cycle here? Eventually, the damaged skin will break down and crack, allowing opportunistic bacteria or yucky fungus to invade. Congratulations. You have caused otitis externa, also known as Swimmer's Ear.
Speaking of swimmers, chronic water exposure can wash out all of the protective wax, too, causing the water to remain in the ear canal. In order to remove the water, people will use Q-tips, or put drops of drying alcohol in the ear canal, often making the matters worse." -- from webmd
I experienced this myself. Not using q-tips after every shower actually stopped the itching in my right ear.
Also I don't have large chunks or pieces of earwax so q-tips work perfectly fine for me (there's no pushing back of earwax into the ear canal).
Been at the doc a few weeks ago and he said that everything's fine - and I have never used anything else but hot water and q-tips to clean my ears.