Output impedace affects damping factor and frequency response.
Damping factor is the ratio of the transducer impedance over the amp's output impedance, and represents the amount of control or "grip" that the amp has on the transducer (there's an article over at innerfidelity that explains this quite well). As a general rule of thumb in the speaker world, you want a damping factor of 8-10 or higher. In the headphone world, the effect of damping factor is arguably not as important since the drivers are relatively lightweight and so the whole electromotive back emf yadda yadda pixie dust may not be as relevant. Read up on those argments at your discretion. Impedance for planar magnetics is also another funny thing.
Changes in frequency response might also occur if the headphone impedance tends to swing wildly. Sennheiser and Beyerdynamic are the most well known examples here, with impedance spikes potentially 2-5x higher than the nominal rating in the midbass region (typically around 80-100Hz). This results in a midbass boost (usually only a couple dB), which most people find is actually a somewhat pleasant warm sound, though there's an accompanied "boomy" feeling (whether that's from increased levels and/or from decreased damping factor at those frequencies, well I'll leave that for you to decide).
My stance is this: If you're looking for a sub-$100 amp to drive a $200-300 headphone, don't worry about it. Most headphones are actually not too crazy on the impedance curve. Chances are you're not going to notice anything related to what I just said anyways. Although now that I said it, placebo will kick in. Sorry.
I've run grados (32ohms) off a Bottlehead Crack amp before which has an output impedance of 120ohms. It was fine.
The only time you absolutely want a very low output impedance is with something like multi driver iems which start atrociously low like 16ohm and swing wildly along the spectrum. But then if you had one of those, you wouldn't be looking at an amp like this anyways.
