It's all dependent upon the competency of the sub. Once you get above the low end subs put out by some of the commodity brands, most current subs have no issue with speed, quickness, or any other adjective applied to the ability of the device to correctly produce the required notes.
There's been massive development in motor and magnet structures over the last decade or so, leading to the ability to control the cone and eliminate the overhang that used to create "sloppy" bass. Take a look at the size of the motor/magnet in the JL subs, or the use of a servo in Rythmic's offerings - either can start/stop a large cone as quickly as any small sub. All of this is fairly easy to measure and confirm - what used to be an issue with larger subs really isn't a problem with the current generation.
If you're looking for physicality/impact, a larger driver is what's required. The ability to create that tactile sensation is linked to cone displacement, so unless you have an 8" sub with an XMECH above 3", it really won't be able to compete with a 12" or larger driver's output/physicality.