I know you all are dismissing what the doctor said as complete rubbish, and while I do believe that the way he worded it seems slightly awkward (i.e. that headphones are hurting your brain), I feel that there has to be some truth to what he said.
Despite what some people may think, most doctors do not BS when they don't know anything on the subject. If they litterally know nothing about the subject, they'll refer you to someone who does. I don't know what kind of a doctor he is, but if he's at all good at what he does, then chances are he has heard/read it somewhere in one way or another.
I don't want to scare you, but the funny thing is, ever since I started listening to headphones, the same issues that you're right now encountering happened to me, hence why I've stopped listening for now. Pressure in both ears/ringing is possibly a symptom of a cold, but honestly, before listening to headphones, this had never been an issue for me. Since I have stopped listening, i've had two colds, neither of which have triggered this response; and I listened at VERY low volumes (we're talking 65 db peaks here). Sure we all would like to say that this is a cold issue and nothing else, but lets be honest here; I think your doctor is right when he tells you to lay off the headphones for a while.