How exactly does a higher sample rate, when converted to an analog electrical signal, cause distortion or overwhelm a headphone? That doesn't make any sense. Applying that logic to a pure analog recording, which would theoretically have a sample rate of infinity (because there is no sampling) would mean that all analog recordings are equipment killers.
Analog recordings are still bandlimited, moreso than your standard digital file. Way more than a 192kHz digital file. They have quite a bit of treble rolloff.
The distortion would arise when a headphone is trying to reproduce a frequency that it is not really engineered to reproduce, which a 192kHz file is capable of producing. Really a worse case scenario though I think.