I've had the Sony NW-A57 for a a couple of years, which I'd used with both a pair of inexpensive IEMs and my Bose headphones. I found it to have a few advantages over my phone, which others have mentioned:
1. It's smaller, so around the house and out and about while running it's just easier to carry.
2. The battery life is better than my phone, and when traveling I didn't have to worry about killing my phone battery on long flights and then getting off of a plane to recharge before making phone calls or figuring out how to get to my hotel.
3. I'm able to load up a microSD with all of my music and pick and choose (the Sony has pretty good support for genre classification and browsing is fast) what I want while I'm out and about.
4. It's easier to get and play FLAC files than it is with a phone (you can do it on Android, but moving files is fussy, and storage can be tight).
5. The dedicated headphone jack.
When I made the decision, the clinchers were #3, #4, and #5. I've pleasantly discovered #1 and especially #2.
That's not to say that I don't stream, but when I do, I'm usually at my desk. If I want to around the house, I just use my phone (without a DAC dongle, usually); battery life isn't such an issue because it's on WiFi and I plug It in at night anyway.
The phone definitely has some advantages around streaming, listening to podcasts (which aren't usually hifi anyway, but I do wish I could keep them on the same device), speed/fluidity of UI at times, and the fact that sometimes I didn't want to fuss with carrying two devices.
But I'm glad I went the DAP route.