Let me get this straight. You put something that has mass and weight on a band that's not designed to take on additional mass and weight, and you're surprised it's not to your liking? You're surprised?
No wobbles here when I tried that with an even heavier Cowon J3, try exchanging your V80. I wasn't so lucky with M50. Then again, M50 doesn't have M80/V80's vice grip. Well, a normal M80 or V80, that is.