Basically the lower the impedance the easier it is for an amplifier to drive something, but the more reactive the amplifier is to the speaker's fluctuating impedance. All speakers have mild variations in impedance over frequency ranges. With very low impedence speakers these variations are large in comparison to the base impedance. With higher impedance speakers, these variations are typically much smaller as a % of overall impedance resulting in less variation over the frequency range due to the variations in impedance.
In order to reduce the impact of variations of impedance in the speaker, a low impedance in the amp helps. If an amp's impedance is at or near the same as the speaker, then these variations will have a large effect on the frequency response curve. If the amp's impedance is much lower then the variations in impedance won't have much if any effect.
Higher impedance speakers require higher voltages to drive them at the same power level as an equivalent lower impedance speaker. So, most higher impedance headphones are designed to be driven by amplifiers designed for this higher voltage output, like studio equipment or headphone amps. Amplifiers like the ones built into phones, tablets, and laptops do not provide a high enough voltage to drive headphones much over 80 ohms very well.
What this comes down to is pretty simple. As the impedance of your headphones goes above 60 to 80 ohms you start to need an amplifier designed to drive high impedance headphones. Generally speaking a headphone amp will provide a cleaner signal than the built in amp in a mobile device and most headphones sound noticeable better with a good amp. So often if your choice is a low impedance version of the same headphones with no amp for $250 and the higher impedance version of the same headphone with an amp for the same price, then the higher impedance version is usually going to sound better, but you're going to be stuck walking around with a mobile amp for it.
Most mobile amps can handle ~220 ohm headphones just fine, but will struggle with 600 ohm headphones to reach a good volume level.