Or try considering other IEMs before getting the XBA Z5. For that price you can choose/invest more on the IEM and when you got another budget, then go for the amp dac. I suggests you look at Heir Audio, Lear or others because it is a lot cheaper not to upgrade rather than buy hastily and waste...

Sony PHA 1A or Oppo HA 2 SE. Well within the "strict budget" you said. 

To each is own is quite right. As JoeDoe said: I can hear the difference! Me too! Never went back to lossy files after hearing lossless files. Investment is a right term. I can tell you this, GO FOR IT! Money can't buy happiness hearing heavenly tunes in your ears. that's how I describe lossless...