The headphone is the single most important component. The source will only make a bigger difference if the source material is really,
really terrible - like 128 bit mp3's played on an SB Audigy. Assuming you encode in 192 or above, you will notice improvement with higher-end phones.
Moreover, the sound signature among the IEM's you listed varies greatly. Having owned pretty much all of them, I can say that the sound signature variation among these IEM's is greater than among full-size headphones, for instance, HD650, DT880, or RS-1. On top of that, the sound signature of one IEM can vary greatly depending on what tips you use. The UM2 is the best example of this - it sounds like one headphone with the Comply tips and another headphone entirely with modified Ety triflanges. On top of that, with the same tip, a canalphone can sound differently at different times depending on the quality of the seal that you get. This isn't as evident with the likes of the UM2 and E5c, since getting a good seal with these is easier, but the ER-4P and E4c are extremely seal-dependant, and require a lot of fairly precise insertion to sound right. The difference between a good seal and a not-so-good seal can be greater than the difference between the same file played on one player and another.
I also think that the quality of the file is more influential on overall sound quality than the choice of DAP, assuming that the DAP is in the top tier of sound quality overall, and you're not using an ancient 256mb flash player with a 10 kHz treble cutoff. The difference between a DAP and a good standalone source, however, is very pronounced, but still less so than the difference between one headphone and another.
Ultra-revealing headphones do change the nature of the game somewhat. A Stax system, for instance, is as revealing as a headphone gets, and it has a certain "minimum requirement" as far as source components go. When you're above that minimum requirement, then different sources present different sonic flavors, but the difference isn't as extreme as switching to, let's say, a dynamic headphone out of the same source. But once you fall below the minimum requirement, the Stax will sound absolutely horrendous - telling you just how bad your source really is. This is true for most very revealing headphones.
So... in the end, everything matters.
But headphones matter more... unless you're a Stax person... then you're screwed... or more precisely, your wallet is!
Edit: hard to drive phones also change the game a lot. Here, you have an amplification requirement. If you're above it, then the difference in sound signature between one solid-state amp and another isn't as pronounced as the difference between one headphone and another, though the difference between tube and solid-state amps can be. If you fall below the amplifictation requirement, however, then your headphone will let you know really fast! The soundstage in most cases will collapse and dynamic range will suffer terribly.