Do you have thin sheets of rectangular foam on each side of the drivers? I find removing them makes a big difference in sound.
i only have speaker cloth on both sides of the driver, no foam.
i did seal the drivers to the "openings" on both ends by replacing the foam supports with "cardboard bitumen" and bluetack gaskets. i think it worked quite well.