For cheap, I'd grab an old Technics table and a decent receiver. Many old receivers drive the headphone jack from the speaker outputs and work quite well as a headphone amp. For the table, you want to make sure that it doesn't take P-mount carts... some of the newer technics belt drive models were like this. Their other models are fine. The newer direct drive tables have a -78db noise floor vs the older belt drive models such as my SL-23 which has a noise floor of -70db. However, I find even the SL-23 to be acceptably quiet. Personally, I'd look at an SL-D202 or SL-D303 which are cheap, direct drive models. Be careful with the older stuff; the speed control tends to go out with age. My SL-23 had this problem, but I fixed it with new capacitors.
You will also need a cartridge and/or a new needle. I've had good luck with stanton cartridges, specifically the Stanton 681ee. The 680 is the essentially the same, but much cheaper (it's intended for DJ applications, but internally it's the same - just not calibrated). I also have a Stanton 505 skII which is a DJ scratch cart, but sounds acceptable. If the needle is old or worn, always get a new one.
Beyond this, you'll also need some way to clean the records before playback. If the records are REALLY dirty, a one time bath usually does wonders. After that, clean/dust them with the product of your choice. I use the Discwasher system. However, I've also used a microfiber cloth and that does fine as well. The cleaning solution I use is a small (~1-2%) amount of 99% isopropyl (rubbing) alcohol, a drop of dawn dish detergent, and water. Carbon fiber brushes are also sold for a quick dusting, but I don't believe they work as well.
Overall, you can acquire all of this equipment, besides the headphones, from ebay/goodwill/yard sales for well under $100 and it should sound pretty good!