Impedance is a property of the headphone itself. It is not dependent on the voltage you pass across it.
Impedance (Z) has 2 components, resistance (R) and reactance (X) such that
Z=R+X *
R is the resistance of the voice coil, caused by it being a long thin wire. Higher impedance headphones** use thinner wire for the voice coil and/or more coils of it. You can check this with a multimeter.
X is the reactive part due to the wire being in a coil, so it acts as an inductor. It's not strictly correct to say it's affected by the magnet, as it would have the same reactance even if it isn't in the magnet - the magnet is just the reason it moves. X is frequency dependent.
* Normal addition doesn't work here since Z, R and X are vectors (have direction). R and X are perpendicular to each other, so 3 ohms of resistance + 4 ohms of reactance = 5 ohms of impedance, like if you drove 3km west and 4km north you'd be 5km from where you started.
** I think most manufacturers (incorrectly) quote the impedance as the resistance. Impedance is frequency dependent so it would be a graph rather than a single ohmic value. Some reviews with measurements show how impedance changes with frequency. Dynamic drivers don't fluctuate too much. IEMs (balanced armatures) fluctuate wildly. Also, in these real tests output impedance from the amp also comes into effect.