From memory, I owned the Sony X, the J3, the S:flo2 and a Clip (not the plus, but a Rockboxed v1) all at the same time with the SM3s. For my money, I would choose the J3 for a few reasons: EQ (in case you need it, I did), capacity (has the expansion card slot), format choice (plays flac) and has a pretty decent UI, plus good video capabilities if that means anything. SQ-wise, I also thought it sounded best among those players using HP out (but I could not listen to it flat, with no EQ, very dull and uninvolving). It's nice to be able to experiment with the EQ and the SM3s, rather than be limited (in terms of the Clip, unless of course you plan on rockboxing it). The J3 EQ gives you an incredible array of options.
As for the S:Flo2, I just felt that while it sounded very nice with the SM3s, I could just not get past the UI and its EQ is limited, but not bad either.
The Sony X is a relatively warm DAP in my view, and while it has an excellent EQ, and the synergy with the SM3s is good, you have to pay a premium for the broken browser, so it's very expensive relative to its limitations (no expansion slot, only WAV for lossless). However, I did like the EQ, which I needed for the SM3s (treble increase).
But you already have the S:Flo2, no?
With no EQ (if you don't like using one), I would choose the S:Flo2 or the Clip+ (rockboxed).
The strange part is I own none of that gear any longer, except for the Clip, but it wasn't that long ago.