I'd say the N-50 is not too far behind the D100 in terms of sound quality. D100 is better, but not by a massive degree. I wouldn't hesitate to recommend it, especially now that they have made some updates via firmware which address a few issues. I have no experience with the Fostex for comparison though, sorry.
It's funny - at one point I saw over at Erji.net that Yulong (the designer himself) had an N-50 and liked it. He thought the DAC section was simple but good, though he eventually did some mods to his unit. I can't find the thread though.