Maybe you should add some sealant or gasket in between the clearance. If you insist to use the original housing. But I've actually been switching 40mm driver to 50mm. Rather than mod the original housing, I made new one with 3D printer