Like all (audio) DACs with no headphone output, it doesn't have the electronics to handle high current. It just has a line output, which is intended to be connected to a line input of another device, which is usually several thousand ohms or more (put simply, current is voltage divided by impedance, so high impedance means not much current is required). You need a specialized design or at least some kind of headphone output / buffer to be able to provide enough current for headphones. There are some other issues too, but it's mostly about the current. You might have a nice clean 2V, but when you connect that to some headphones, it won't be nice and clean, and probably not even really 2V anymore.
Some line outputs can do okay driving headphones, but that's pretty much operating them way outside of specification.
Ah, makes sense. After I posted that conclusion came to me as well.