Well reaction times are measured as the time it takes for the MOSFET to switch between 10% and 90% of the difference between Vgs and Vds:

If you consider the times (td, tf) as independent, fixed values, then they create a slope that is dependent on the difference between Vds and Vgs. The higher the difference, the steeper the slope is. Since audio applications, especially headphones ones, don't need the full switching from 10% to 90%, you can cheat by raising the voltages and only using a part of the slope. This way you can match the slopes of different parts, even if their rise and fall times aren't the same.
But then I'm unsure if the times are independent on the difference between the gate and the drain. The datasheets suggests so, anyways.
In reality though, NFB should take care of all of this...
Thinking about it... switching times are probably just not a concern here. We're talking about differences in the nanoseconds. Just to put that in perspective, one nanosecond is the cycle time of a 1 Ghz waveform... 