oddly enough the Solti set you display was one of my first classical purchases as well......I'll be frank and say I don't think that he performs a single symphony particularly well in that set, with a slight exception being the 3rd which comes off especially grand due to a very broad tempo. Sonically I find the set reference quality and I still listen to it for that purpose.
If you are a fan of Solti's approach to Beethoven, then I just know that Bruckner is going to be your next step.....essentially Bruckner is a hybrid of Wagner and Beethoven, but certainly more Wagnerian. Solti's specialty was Wagner. He's almost pigeonholed as a Wagner conductor because of just how essential his Ring Cycle was (the first stereo studio recording and without question the most famous version of the piece and biggest selling album of classical music).......
I would look into Gunter Wand's recording of the 7th, 8th and 9th symphonies on RCA.
I also think a purchase of Bach's music is pivotal. If you like the orchestral sound then try the Brandenburg Concertos as performed by Tafelmusik. But I feel Bach's greatest achievement is the Goldberg Variations and Well Tempered Clavier. IF you would like to hear these works on piano as opposed to the original intended instrument (harpsichord) try Angela Hewitt or my favorite for the Goldbergs, Murray Perahia. If you would like to hear it on harpsichord definitely go with Trevor Pinnock as a first choice........
Robm mentions the B Minor Mass and I have to agree in this....I personally prefer instrumental music to choral, but the piece itself is essential, should be in anyone's first 20 or so classical purchases. Elliott Gardiner does a very fine version, so does Philippe Hereweghe.
I think Brahms is a composer whose symphonies are as brilliant as Beethoven's without the prospect of being innovative. A great starting set to hear what Brahms is about is the Karajan cycle on DG on 2 cds. I, however think that Brahms piano concerti are his greatest works.....without question my most often played CD of all classical cds (this includes Mahler who is my favorite) is the Emil Gilels & Eugen Jochum pairing of both of Brahms' Piano Concertos. Simply the best performed and one of the best sounding CDs in my collection despite it being a 1970s recording.
Schubert is always a special comparative listen to Beethoven. Schubert idolized Beethoven, and their lives ended essentially simultaneously despite Schubert being 27 years younger. Schubert, even more than Beethoven is the bridge between classical and romantic music. He is Wordsworth to Beethoven's William Blake....Try his String Quartet "Death And The Maiden" performed by the Takacs quartet
and of course Mozart......can't make a list of essentials without Mozart. Sometimes I view Mozart as water. He's the most essential of all, but easy to get bored with. His music is perfection, but sometimes you miss the eccentricities or in the case of water, the flavor of other composers. I am a huge fan of Mozart, and in fact I own more CDs by him than I do any other composer aside from Bach and Beethoven. But I truly don't know how to begin a recommendation into Mozart because I truly believe that his best work was his operas. But at the same time, I don't think listening to Don Giovanni or Figaro is the best way to learn his music for the first time. The piano concertos have already been mentioned, I would add the clarinet concerto which I feel is the most wonderful concerto he ever wrote, and the best concerto ever written for a wind instrument. Check out Thea King's recording on Hyperion
-Dave
j/k My point was that if he likes Solti's performance of the Beethoven 9th, then he may very well like Bruckner's music, because Solti's performances of Beethoven come off very Brucknerian. 
