Going with an adapter you also wouldn't have a physical volume control which I really wouldn't recommend. Best bet would be to go RCA out to your E9 if you wanted to go with the Bifrost. But yeah it goes back to the question of which is more important to you, 24/192 or an integrated headphone amp?
I would say 24/192 or basically the DAC component. If one is spending this much money you might as well just go for the best anyways. The E9 is a good amp and if need be I can always upgrade to a better one.