First, I'd recommend you pick up the
NPR Curious Listener's Guide to Classical Music and the
NPR Curious Listener's Guide to Jazz. Both are terrific introductions for the new listener. They include historical and background information and give you lists of very popular titles to build a library around.
Classical recommendations are tough - the genre is huge, there's so much to cover and the styles are really, really different. Classical might seem staid and stodgy, but it is anything but. You can sample a few hundred years of development, from conservative styles to some of the most radical music ever written.
For a sampling of classical, try Bach's Brandenburg Concertos, Mozart's Symphony No. 41 "Jupiter," Beethoven's Fifth (the Sixth, Seventh and Ninth are wonderful, too), Chopin's Nocturnes, Shostakovich's Firebird Suite, Arvo Part's Alina, and give Holst's Planets a try. That covers a lot of ground, though it leaves out a couple hundred incredible works you also need to hear. But you'll get to those in time.
For jazz, pick up the two jazz albums
everyone loves and go from there. Those are "Kind of Blue" by Miles Davis and "Time Out" by Dave Brubeck. Both have deep, excellent catalogs.
One neat trick to jazz is that if you love an album, pay attention to the other players on it. For instance, if you love "Kind of Blue" (and I think you will), you'll notice that John Coltrane, Cannonball Adderly and Bill Evans play on it. You'll find that all three of them recorded a bunch of other discs, so go check those out. Bill Evans is a real favorite of mine; his "Waltz for Debby" gets a lot of spins and the recording is wonderful, too. So give those two a listen and start branching out from there. Get other discs, learn about other musicians, get their recordings, and just let it branch out from there. There's so much overlap between musicians and their projects that you won't hit any dead ends.
You'll probably hate me for this, but if you want to get into classical and jazz, there's another piece of hardware you should pick up.
See if you can find a good price on a Sony SCD-CE595 SACD player. I got one a couple years ago for about $70. You should be able to find one between $100-$120, sometimes less if Sony is clearing out refurbs. It's not the best player out there, but it's plenty good enough to deliver the benefits of SACD. If you want to build a classical collection, SACD is a terrific way to go. Classical really pays off with hi-rez and the recordings tend towards the excellent. Even better, you can find a bunch of RCA Living Stereo re-releases of absolutely classic albums on SACD for about $10 each, sometimes less. There's also a fair amount of jazz on SACD, but the format really pays off if you love classical.
Oh, and you might also want to keep an eye out for a used AKG K-501.
They do classical and jazz better than just about anything short of the K-1000. They go between $100-$150.
Apologies for working hardware recommendations into a music thread.