Thanks for all your suggestions.
I'll most likely get a Clip Zip and a fast 32GB microSD card (which I'll fill with an USB card reader).
I'm hesitant to get another Clip+ due to the problems I'm having with mine, even though it seems to be a hardware issue. I don't want to get another just to have it fail on me in exactly the same and annoying manner.
The Cowon C2 isn't appealing to me due to the lack of physical buttons and use of proprietary connector. The Clip also might have a better sound and I prefer the small form factor.