I recently bought the D3 and so far I am loving it. I'm no expert, but here are my 2 cents.
I have heard a J3 before, and all I can say from first hand experience is that the difference, if any, is too small for me to notice (at least with my current headphones).
Prominent members of this forum and other audiophile forums argue that the J3 and D3 sound exactly the same, and any perceived difference is placebo.
Multiple, on-the-go playlists was one major reason I chose the D3 over the J3. This can be done by rooting the D3 then installing a third-party player like RockBox or PowerAmp. Personally, I am using PowerAMP.
The device is definitely slow, compared to other Android devices. However, I think the major reviews on the speed/bugs have been overly harsh. Everyone these days has dual-core Android phones, so they expect any device running Android to be that fast. It is a valid point, but if used purely for a music source (i.e. only player app running, wi-fi off, etc.) and maybe some video here and there, I find the device acceptable. Let's just say that even as little as 7 years ago, a device with the speed and reliability of the D3 would not warrant negative reviews.
Conclusion? If you want an Android device for multi-tasking, installing apps, browsing the web, and some music playing on the side, get a Galaxy S III. And as much as I would recommend the J3 over the D3 to about 98% of people, there is the 2% who take advantage of the D3's strengths over the J3: RockBox/PowerAMP and digital S/PDIF output. If those don't mean much to you, get the J3.
addendum: At the time of writing, my D3 (and others) has a bug where third party player apps have much lower volume than the stock player. There are a couple workarounds, one of them being using the S/PDIF output I mentioned. Just a warning though if you plan on rooting it without an amp.