For certain IEMs, it is better to be amped, example PFEs. These IEMs/earphones have higher impedance (more 32 ohms) really do need more more power to shine.
But, if you have a DAP which have higher output, and high sensitivity IEMs/earphones, then no need to have amp. I tried iPod Touch 2G (I think) with PFEs, and, Touch do can drives PFEs efficiently. At 75-80% is very loud enough, but iPod nano 1G cannot drive PFEs as good as Touch 2G. So sometime it depends on the DAPs too, unless you want something differents with amp, such as better SQ, more power or more bass (bass-boost.
BUT, amped is NOT always better. You may get a good result with certain amp, but sometime worse with other amp.
LOD or line out, is the choice for amping. Supposedly it should provide cleaner signal compared to headphone-out.