Isn't that the reason people spend more and more on ever more insanely priced flagship IEMs? For another few percent here and there?
FWIW I firmly believe the transducers make a bigger difference than the source, but once you reach a certain point, even with transducers, the wall of diminishing returns hits hard and steep, so the extra $1000 from a $2000 IEM to a $3000 IEM is only buying you single digit improvements, if that. And it's mostly changes in tuning and tonality than actual technicalities.
So it's really up to you to decide how much you want to spend on "the last mile", and even then there'll always be something more expensive just around the corner.
My view on sources is this: find the DAP that has the best synergy with your IEMs - that makes your IEMs sound best to you. Differences between DAPs are fairly small, but there are differences. I just spent a few days testing the Lotoo PAW Gold Touch and Hifiman R2R2000 with my IEMs, comparing them to the HiBy R6 Pro, and preferred the HiBy almost every time, even though the more expensive DAPs were technically better. Synergy is everything, and the differences aren't night and day, despite what many would have you believe.