I'm not. if only because there's a tight relation between FR and an impulse response(or any transient event). So even if the only relevant change(anything audible) was the FR change, I couldn't claim that the transient stuff are the exact same. And at least objectively(audibility is another story), FR change isn't the only thing going on when you change the volume of the internal cavity and very likely a good deal of acoustic damping with a different amount of seal.
But how do you measure that? How do you confirm that it's audible and feels like what you're feeling? This seems like an even more complicated issue given how the FR will also change significantly, so we'll never be completely sure that change is really how freely the driver moves and how "fast".
On the other hand, there is ample evidence that different pads can have very audible FR impacts. It's relatively easy to measure and I do it(badly and for the lolz) every time I get new pads. There is also ample evidence that you can change various impressions of various instruments with an EQ(there's even a job about that). The settings used in a mix for snares may be more aggressive than what 2 different pads would do(luckily because we wouldn't want that kind of EQ on the all song), but audibility and subjective impact are hard to reject. At best we might argue that there can be other variables involved, but not that FR change has nothing to so with our subjective impression of ... almost everything.
I understand how intuitively we feel "speed" so we think it's time domain stuff, but I do not know for a fact that it is in this case. Doesn't it just make more sense to treat this as a FR change when all the evidence of FR change and its audibility is in our face?
I'm perfectly fine considering various causes changing the airflow and how freely the driver can move, but without supporting evidence of significant(audible!) impact in that area from swapping 2 pads, I would leave it as an open hypothesis, and stick with the evil I know: FR.