A DC offset is a constant voltage superposed to the (AC) signal, relative to signal ground. The voltage level at no output is shifted, thus the term DC offset. If you want to connect headphones or speakers via an amp that's entirely DC coupled, you do not want this to be too large to avoid fried voice coils. The output coupling caps are there to eliminate any DC offset; single-digit mV readings across the cap should be safe (after the voltage gain stage, if any - you don't want it to be amplified, after all). Now coupling caps are not the most linear devices, particularly electrolytics (you need them to have a certain capacity to drive loads down to 600 ohm, and comfortably so) and those particularly when they are not biased (some DC across the cap - like a few volts - reduces the hysteresis considerably; yes, these things can distort like transformers). That they work as well as they do is only due to being overdimensioned quite a bit, thus very little voltage dropping over them in the frequency ranges that interest us. (Try building a simple R-C highpass with a 'lytic and measure the distortion for tones around and below the corner frequency. That won't be looking good even for nonpolar types I bet.)