Types of tube amps:
Output Transformerless (OTL)
The headphones will be driven directly by the tubes. High output impedance. Usually does not work well with low impedance headphones, though I have heard of exceptions. Some examples are Little Dot Mk2, Schiit Valhalla, Bottlehead Crack.
Transformer coupled
A pair of audio transformers are added to achieve low output impedance. Relatively expensive because good audio transformers don't come cheap. Examples are Woo Audio 6, Bottlehead S.E.X., Millett Jonokuchi.
Hybrid
A combination of tubes and solid state. The final output stage consists of a solid state components that have a low output impedance. Examples are Little Dot 1+, Schiit Lyr, Millett MiniMAX.
If your computer's onboard sound chip is no good (noise, background hiss, lack of clarity) then upgrading that will make a big difference. Either get a good internal sound card and install it far away from the video card, or get an external DAC (my preference). But even with noisy onboard computer sound, an amp can still make an improvement. The noise from the computer is at a fixed level. So set the system volume to 100%, then use the amp's analog volume knob to turn it back down. Now the noise will be barely audible because the amp's volume control reduced its volume along with the music. Still better to have no noise to begin with of course.