There are definitely a lot of factors that can create hearing imbalance. Just off the top of my head I can think of:
-Outer ear shape.
-Ear canal shape/diameter.
-Ear wax. Some people produce too much wax and end up with plugs, and use of IEMs can increase wax production. NEVER try to remove it yourself if you think you have wax plugs, you can severely damage your eardrums and at best probably just push the wax back in. This is definitely a job for a doctor with the correct equipment.
-Differences between eardrum structure (probably from damage).
-Eustachian tube differences. Probably a lot of different causes here, including fluid blockage, inflamation, or just structural differences. Abnormal adenoid development can block one or both sides.
-Cochlear damage (often from unilateral exposure to extremely loud sounds, for example using firearms without hearing protection).
-Nerve damage or inflamation.
-Neurological processing differences? Like with hands and eyes you should have a dominant ear, and I wouldn't be surprised to find that this can impact positional audio perception a little.
-Ménière's disease. I like mentioning this one because it's one of those great vague disorders you can enjoy being paranoid about.
I find that I have about a 1-2 db balance issue favoring the left. I would imagine basically everyone has some amount of offset considering the number of "moving parts" involved in sound perception, but anything less than 1dB is fairly hard to detect. People who don't use headphones very often may never notice it; I find that I automatically adjust my head slightly when I'm watching TV, which fixes the center channel position.
As far as dealing with it on the source end of things here's what I've come up with so far:
-Remember that some amps have channel balance errors themselves. It's good to know what your amp is doing so you can factor that in when you're making adjustments. While many decent amps have a balance that would be considered inaudible, if it's in the same direction as your head's balance problem it can just make the whole thing worse.
-The Fiio E17 has a digital balance control that will let you make small adjustments to balance. I'm not sure exactly how small the increments are but I'll be finding out tomorrow when mine arrives. If you're okay with it doing pre-amping you can use this balance setting and still plug it into a heftier amp, but I know a lot of people consider double amping to be extra-forbidden.
-If you use an iPhone/iPod Touch you'll find a balance option under settings->general->accessibility. Unfortunately it's not very precise, being a standard ios slider with no numbers. It also snaps to center, and for me this means that the range I actually want for adjustment isn't available (the lowest right position you can get it to stick to is more than I want). If your hearing offset is more significant then this might be an okay option.
-There's at least one equalizer app for iOS that can do per-channel EQ, which you can use to adjust as needed. The disadvantage is that you'll have to play your music through the app as it's not allowed to have any effect on what the default player does.
-OSX has a slider a lot like the iOS one, which is sort of a slide and listen situation where you can't input specific dB offsets. It also has snap to center but it's less severe because the slider itself is so much larger. Not very accurate but I can find the right balance with it.
-Android doesn't seem to have any left/right balance options available, which seems bizarre to me considering how much you can configure in Android. It's unfortunate because if any portable device was going to have numerical balance controls like Windows, I'd expect it to be an Android device. Someone correct me if I'm wrong here.
-Easy in Windows. Balance options in volume controls per device, you can even right click and go to dB mode instead of the far less useful percentage. If all your listening is through a Windows computer you can ignore everything else I just wrote.
That's all I can think of at the moment. Seeing your doctor, or even an ENT specialist about it is probably a good idea, especially if it turns out to be something easily correctable. I've actually never done this for some reason.