How does the brain adapt? Do you feel like you hear normally but don't?
As far as it can, the brain recalibrates just about everything all the time without us noticing. Some things are done in minutes, some may take a month or more depending on the magnitude of change or how unusual it is (for the brain, normal only means it often happens, nothing more). And of course, there is a point where some issues are so big that the brain just can't hide them from us. It's not an audio thing, we have constant recalibration for all our senses. If you wear colored glasses for a while, you'll slowly start "seeing" most of the colors as if not tainted by the glasses. And when you later take off the glasses, for a short time you see weird colors.
When you get in a room, initially the amount of reverb can be massive and very noticeable, but if you remain in the room long enough, the brain will understand what portion of the sound is room reverb and lower the amount we consciously perceive (it's guessed that we do that to help stay vigilant and better focus on the might-be-relevant sounds signaling a danger). Subjectively, it just feels like we get used to it, and we usually don't even think about the reverb of a given room (maybe if you sing in the shower you do^_^, because you introduce a very clear marker that you know all too well, your own voice).
At a very slow level, our ears "grow" over the years. And if you use IEMs that go deeps, they will stretch your ear canals. But you wouldn't notice a change of sound like that because it's progressive, and the brain has plenty of time and references to adjust to the new normal.
When you get a new headphone, it's the same thing. You've been using a headphone for a long time, there has been some habituation. When you get a new one, it takes a while to get used to it, and a little longer because of how different it is from the previous one we were used to already. This phenomenon is very clear, but off course it's not reaching the point where all headphones end up sounding the same. That's too much for the brain. But finding a little bump in the frequency response less annoying after a few days of use, getting more used to the different amount of isolation from the headphone, ending up feeling the bass even if it's 10dB below what the last one had relative to midrange, those are things the brain usually handles well with time.
A more peculiar case, if you spend a really long time using a headphone that has a really unmistakable channel imbalance, that will never completely go away. But if you almost exclusively use that headphone to watch TV, over time, the sound will subjectively align with the TV (this one might take months!). Some work is fully automated, and some work kind of follows a narrative that the brain believes. When it comes to sound and vision, basically every brain will trust what it sees over what it hears if the 2 disagree. So in the long term at least, a visual anchor like the TV (or for me a pair of speakers, even though they're not ON), that can be enough to let the brain recalibrate based on that visual anchor. It must be consistent, so the brain just switch into "headphone listening mode", the same way it switches into "driving mode" when you get in your car.
It's the beauty and also what's so annoying about the human brain, it's never as simple as the models we make to explain what it's doing. And it's also true this time, but hopefully you get the general idea without me misleading you too much.
Anyway, we do constantly change how we interpret senses, from not feeling as cold after a while in water, to the most subtle things we're never even aware of. What creates or solves a problem is the magnitude of the issue, how obsessed we are about it, and how easy it is to know what the "right" solution should be based on whatever reference available. It starts with a concept as universal as perceived neutral. It's an objective certainty that we all receive different frequency responses at our eardrums, because we have different body sizes, head and ear shapes, different ear canal lengths. We tend to look and be different enough for the frequency response and inter aural delays to be pretty specific to your own body. Yet we all subjectively feel like we're hearing the real sounds around us, unprocessed, unEQed. It sounds normal, natural, neutral, etc. And growing up, while we all lose so much sensitivity in the high frequencies, we still pretty much keep feeling like we're hearing sound normally with a proper balance. It's really when the loss is huge and usually to the point where it hinders our ability for speech recognition or some other tasks, that we really stop feeling like all is normal.
I'm still not sure if OP feels that imbalance only with headphones? I think that can make a world of difference, because for example, Maybe some asymmetry in the face+ears causes a clear imbalance without headphones when he's looking straight at a sound source at some distance (let's say someone talking to him). Or maybe he just has a habit of turning his head slightly to one side when looking at something. Those would be incredibly common things and the brain would have used vision over time to determine something along the lines of "never mind the audio localization error, just set the new center over there". In such a situation, you put on a headphone, it bypasses the head pivot if there was a trend to have one, and most of the head's and ears' asymmetry by having the sound source so close to the ear. There isn't even a visual cue to say where the center/mono should be because the headphone turns with the head. Chances would be in such a case that the brain sticks to what it's used to and offsets the perceived sound when there is no more reason to. Now the headphone feel imbalanced, but turning it around doesn't help because it's the brain's DSP that's adding something we don't need for that particular use called headphone listening.
IDK if it's OP's situation, but objectively speaking, nobody is perfectly symmetrical so instead of wondering if someone is, it's more of a question of how much it ends up altering sound, at which frequency and if that changes a lot when using headphones.
As a counter example, the ear canal creates a pretty strong resonance near 3kHz, making us most sensitive in the region. One ear canal being significantly longer or with a very hard bend that almost closes the canal in 2, that would be enough to shift the frequency of the resonance and maybe the amount of it, causing a very clear interaural imbalance in the midrange. Most people have that to some degree. But because that imbalance applies to 100% of what we're hearing, it's all we know. It's highly unlikely for anybody to ever feel like there is an imbalance. The brain will be completely used to it and have defined it as normal, balanced, flat sound for the real world. Headphone listening or sounds from a distance, same imbalance, same compensation, same experience. In this case, only inserting IEMs deep in the ear canals might reveal the imbalance, removing most or all of the ear canal resonances and creating again a case of brain compensation that is no longer necessary, itself causing an impression of imbalance that doesn't exist.
This wall of words to say, things would be so much easier if it was all just basic acoustic. But our brain often goes "nah ah!" without asking for our opinion.
Edit for Angrish.
