As has been said before, the DJ interfaces are built for headroom and stability. I've been doing a split between my Audio 2 DJ, Roland UA 1ex and Fiio e10. All of these are about $80-$100, but with different use cases, so I thought it might be interesting.
Source material: The Cinematic Orchestra - Every Day ~ flac
Headphones: Shure SE215, Sennheiser HD212 Pro and Sennheiser HD25
The Audio 2 DJ is dry as a bone, but with gobs of volume. It's no fun to listen to at all, lacks character and I feel it sucked the life right out of the recording, especially on the SE215's.
The Roland is fast and responsive and dead flat, pretty much what I'd expect since I use it as a studio sound card. Black as night on this though, very nice. Will also get tiring to listen to after a while, since it doesn't do the source any favours. Sounded better on the 212s, since they are predisposed to add loads of bottom end.
Fiio E10 is the most pleasant to listen to of the three. It's more flattering to the sound, but doesn't color it too much. My favourite for actually listening, sounded good on all three, but the best on the HD25s, with the 215s a fraction behind (the pesky bump at 4k made them unbearable at high volumes for long periods)
So, you could use a DJ soundcard or a studio soundcard to listen to your music, but the question is rather if you would want to. IMO it's better to get something built for the job