well you push a fixed direct voltage (instead of the alternating mess that is music). so the driver is pushed away from its resting position.
membrane's speed depends on how much power is used to move it(obviously it will be fast nough to make high frequencies so pretty fast stuff), and also how much the membrane weights and resists(trying to go back to the resting position), how much air it's trying to move, the type of coil(do I forget some stuff?).
when it overshoots I would guess one of the reason could be because the membrane has momentum and doesn't succeed in decelerating fast enough.
why it slowly goes down could be that once the membrane has lost it's momentum, it tries to get back to its resting position by a mechanical pull. I would guess that induction plays a part too, as the more the voltage, the more the inductance, the more opposition to the voltage. but I don't know the scale of such an effect. maybe it doesn't actually matters at all?
I'm going on a limb here, but maybe with a lasting voltage, some coils get warmer so the impedance increases slightly, being a reason for another parameter of changes?
Note: my background is in electrical engineering, so if you want more information, you might ask a mechanical engineer.
What you're describing is pretty correct. Mechanical systems are often defined as mass-spring-dampener systems for easier analysis. EE's do the same thing with RLC, or Resistor-Inductor-Capacitor equivalents. Basically, you can use three elements in a second order diffrerential equation and do a pretty common analysis. You don't have to have the actual elements in your system to simplify to such a model. Things like air resistance will create dempening effects, but you can just include that using a measured response.
Anyway, the mass element plays into inertia and momentum, meaning that you need power to get it moving (more mass = more force to move), and once it's moving, it takes some effort to stop (more mass = more momentum = more stopping force). Springs exert force based on displacement, so if you stretch or compress a spring, you get a force trying to push it back to the resting position. Dampeners resist movement (velocity), and like springs, they resist no matter which way you go.
I don't remember the equations (sorry), but you can tell a few things about the system from a square response (which is basically a step response). The overshoot happens when you are under damped. If you are over damped, it will take a long time for the system to reach steady state--in this case, where the voltage is trying to put the diaphragm.
You'll have to ask someone who does this for a living about what is ideal...obviously you don't want to be too much over/under damped because the transient responses will alter the sound, either cutting the audio power (over damped) or creating harmonics (under damped). Square waves are useful because you can measure these responses, but you can also change the frequency to see if and how the response/dampening changes.
I would guess that induction plays a part too, as the more the voltage, the more the inductance, the more opposition to the voltage. but I don't know the scale of such an effect. maybe it doesn't actually matters at all?
Side note, inductors basically act as resistors to current change. The inductance is determined by the magnetic core, number of turns, and shape, and this value stays the same. However, the inductor will create a voltage response opposing a change in current passing through it. So a higher frequency current will create a more dramatic change than a lower frequency one. The applied voltage only affects the magnitude of the current (and it would be a linear effect on the response).
I do think you are onto something, WRT the square response. After the initial bump, the inductor will settle back to zero (volts across the inductor), and this probably will change diaphragm's response. After all, the driver does act as a separate system between your square wave generator and your diaphragm. If you are measuring diaphragm displacement, the input to your diaphragm is actually the output of the driver, which has its own response to the square wave input.