May be warmth, but AFAIK there's also a safety aspect - for your drivers I mean. Pure DC coupled amps - meaning no cap or other protection - could mean your speakers got damaged when/if upstream equipment got switched on. At clipping, pure DC gets passed through too. Either way, your sensitive drivers - which spend most of their time running at (much) less than a watt (speakers) - could suffer a large excursion that results in physical damage. Not good for future performance.
The flip side is that introducing a cap might reduce SQ, as in 'another component in the signal path'.
BTW please take with a grain of salt. All this is vaguely remembered stuff I read and heard years ago when I first got into speakers. One of my audiophile friends is an EE, but most of what he used to say back then went over my head! And, there may be other reasons for using and not using caps too, such as jazzerdave suggests.
I have no idea how or whether this applies with headphones.