I like my turntable better than computer audio. So much so, that the computer rig as dismantled and sold off.
It just didn't get any use.
My advice is to not jump into an expensive deck right away. You will be tempted, but resist.
More than any other component (save selection of headphones) a turntable is a somehat fiddly beast with a mind-blowing array of options. When you start looking at them seriously, you'll feel your frontal lobe begin to melt as it jumps through the options and possibilities.
The problem is that you don't yet have a frame of reference. That's what entry level decks are for. I'd recommend buying a good used one locally (there are a couple of threads on this), get it working, learn to do a correct setup, and install a new cartridge. That will get the records spinning and should sell you on the format. Once you get familiar with the quirks and habits of your deck, you can use that to narrow down a higher grade turntable.
I went around and around until I found a good price on an old Rega Planar 3 locally. I loved that deck. It delivered the magic of vinyl and I couldn't wait to get home to spin discs. Eventually, I started to itch to upgrade it with various mods (there are a lot for Regas) and, instead, settled on a used Michell Gyrodec. The Gyrodec was a serious upgrade and I loved it. After some consideration, I gathered the parts needed to turn it into an Orbe (Michell designed it this way) and have been in love since. There are other excellent tables out there, but the Orbe is everything I could want. Of course, you'll probably settle on something else, but I strongly recommend that you take the same path of starting somewhat small and working your way up.
You can find lots of good entry-level tables a Audiogon. The best thing is that sellers there are usually audiophiles who babied the deck and have the original packing boxes. My Gyrodec arrived in its original packaging, disassembled, just like one from the factory. It was less than half the price of a new one.
I also strongly recommend reading through the Vinyl Anachronist articles. I'm posting from the phone and don't have a URL, but Google should bring him up. You'll find great articles on the vinyl obsession, how to set up, and much else. There are over 40 articles, so you'll have plenty to read.
Finally, Grawk is right about used vinyl. It is the best thing ever. I've stumbled across all sorts of wonderful music at thrift and junk stores. Stuff I never knew existed. It completely broadened my musical tastes and it's been nothing but fun.