Personally, and it is just my opinion based on my reading of what's out there, I would stockpile different headphones rather than different amps, if I wanted something new (though some amps make no attempt at transparency and so on, definitely sound different, etc.).
Also, a very important point to make is that many many amps and DACs, not just the O2 and ODAC—and for many situations and headphones, plenty of options cheaper than the two as well—have pretty excellent measurements in the areas that people test and ostensibly care about, that matter. O2 and ODAC aren't particularly special, just two more specimens that do a pretty decent job. Take that as you will.
Based on the output power, the noise, distortion, and so on... Almost everything should be okay. O2 should handle most headphones in a similar manner. Electrostatics: no. K1000: no. HE-6 and maybe some vintage K240 and some others: no if you listen loudly.
Many people will tell you otherwise, or say they don't like certain pairings (as for the reasons, they vary). That's fine. They're just talking about what they perceive when listening for themselves, for what that's worth. Is there some phenomenon that people haven't discovered yet, that the O2 lacks or does differently than something else? Or a different way to test things? I think not; other people think it's out there.
Rather than have channel imbalance, you can turn down the volume in software if you're looking at really sensitive IEMs. For fullsize headphones, O2 channel balance should mostly be in the "okay" or better range (yes, you can do better). Typical 1 dB off at -55 dB, increasingly better at the higher rotations. Many headphones are matched worse than that. Anyway, it doesn't look like you're looking at sensitive IEMs.
This is more a question about headphones than amps or anything else, so maybe a different forum is more appropriate.
