Sorry about that, I accidently typed C5. It's C2 on the schematic. Therefore, will a polarized cap work?
Well, again - it depends on the application. In this case, C2 is used as an audio signal
input capacitor. It's there to block tiny voltage offsets that may exist with portable sources. At the time the CMoy was developed, it was intended to block any DC voltage that might be on the output of a portable CD player. The same reasoning applies today with iPods and smartphones. What may be an acceptable DC voltage offset to a pair of headphones, earbuds, or IEMs will get amplified 11 times over (gain on a stock CMoy) once it goes through the CMoy circuit to headphones that you have connected on the output. So, C2 blocks any DC voltage before it gets amplified to something dangerous to your headphones.
Now, point in fact - most high-level, audiophile-quality headphone amplifiers do not use input capacitors. You will very seldom find even a portable player today that has any sort of DC offset. If you're in doubt, you can always measure it. So, bottom line, you can delete the caps altogether. Since you're referencing Tangent's website, though, you may want to read this first:
http://tangentsoft.net/audio/input-cap.html . Despite Tangent's discussion on this page, you may note that his formerly offered, highest-quality amplifier was the PPA1 or PPA2, neither of which used input capacitors. You will not see them on better DIY designs that are intended for use with high-quality sources.
Finally, one other factor to consider is the sound quality inherent in capacitors. Every capacitor, no matter how transparent, will affect the sound quality. The absolute worst of these is an electrolytic capacitor (polarized). Even when they are used in the signal path, electrolytic capacitors are often bypassed with smaller (non-polarized) film caps. The idea is that all but the lower bass frequencies pass through the film cap, which is of much higher audio quality. The lower bass frequencies pass through the electrolytic capacitor, where higher distortion in the bass frequencies is not as objectionable. You will also only find them on the output of an amplifier circuit, not at the input. The reason is that the poor quality of the electrolytic cap at the input will get amplified into something even worse.
So, the question is, why would you want to apply an electrolytic at the input stage, where all the inherent distortion that exists in an electrolytic capacitor would get amplified 11-fold? Unless you are referring to a very rare case of a polarized film cap,* this is a very bad idea.
* Auricap wants you to treat their capacitors as polarized, but this does not mean a lot in real practice and is probably a marketing gimmick. You won't blow the cap by wiring it in reverse. In fact, their newest XO caps drop the "feature" entirely.