Alright so I reversed the caps, but I still only get 1.9 V out of the LDO. What's more, I only get 1.9 V at the input of the LDO. I'm guessing it's shorted and pulls the DCP into over current protection. Pulling the LDO out and the DCP outputs 24 V.
Interesting to note is the polarity of those capacitor is not marked on my PCB. The print is simply not there.