Personally I think it's a matter of matching the drivers properly. As I mentioned, think monitor speakers - they generally use two drivers. The larger midbass unit is a dynamic driver and can have a paper cone, or a metal such as aluminum, or some type of plastic like polypropylene. The tweeter can be a planar magnetic driver, or silk dome, or metallic dome, or ribbon, or some variant of the Heil AMT driver, etc. Ribbons and AMT drivers are very fast like a BA driver, yet they can usually be made to blend quite well with the large dynamic driver.
So in summary I don't think it's necessarily the driver type that causes this disparity. We tend to focus on it around here when it comes to hybrid designs, and in some cases it is a valid complaint, but there are also designs using solely multi-BA drivers that still don't sound coherent.