If you are talking about using headphones and you like the sound of your onboard audio you can probably just plug the headphones directly into the back of the motherboard or the front audio jacks on your pc case. Providing it has enough power to run them if you have power hungry headphones. Then you don't even need to waste the money on a tube amplifier. If you want a warmer sound like many tube amps are said to have then just use a software eq or plug in to achieve that. Equalizer apo and Peace work nice as an EQ and there are some vst plug ins that can emulate a tube pre amp or amp that will probably work with APO. Or use reaper or a DAW that can handle plug ins. I personally think a dedicated amp would sound better and would maybe suggest getting a DAC/amp combo but if you say you can't hear a difference which is quite possible why waste the money. I think I did see a review somewhere of a small tube amp with a built in dac for like under 200 bucks. It got a decent review. I don't remember what it was called or who made it though, sorry. That might be something to consider.
If you are talking about hooking up speakers then yes you will most likely need to amplify it. And you are correct you can get adapters or adapter cables that will convert 3.5mm stereo to rca cables. Then just run those into the inputs on the pre amp/amp. Before HDMI and digital were options on receivers that is how I ran my computer audio to my home theater receiver. I just had a 3.5mm to RCA adapter cable and ran it into my receiver. Now I use the optical out on my motherboard or HDMI for sound because it is an option. Another option would be getting powered speakers and using those.
There is actually a computer audio forum on this site you might want to check out. I think most are going to tell you getting a dedicated amp with a built in dac or something and running off USB or optical would be a better choice and I would tend to agree with them after using both. That being said if YOU can't tell the difference there is nothing wrong with that. I think that is valid opinion, everyone is different. They have different preferences, budgets and wiring concerns. What you want to do will indeed work, I did it for years. I just didn't use a tube amp I used a solid state receiver instead. There are also some pretty decent computer audio solutions out there. As in powered speaker set ups designed to work with a computer. There are also some pretty crappy ones lol.