My strongest recommendation about purchasing classical music is don't necessarilly go with a budget version just because its cheaper. At the same time don't think a version that costs twice as much is better, sometimes naxos has a very good deal. But its important when building a library to get to know which composers certain conductors are good at, or certain violinists are known for, or certain pianists specialize in.
Some pianists such as Alfred Brendel specialize almost exclusively in the Austrogerman composers from the classical and Romantic periods. Specifically Haydn, Mozart, Beethoven, Schubert, Schumann, Liszt, Brahms and a few others. It is rare that Brendel performs Chopin. Even Bach is a rarity for Brendel. If you go for any classical piano piece by any of the composers I listed and choose Brendel then you are buying a performance that is definitely worth a listen.
Herbert von Karajan is a safe conductor in many instances. The are better conductors overall, but Karajan covered a vast amount of repertoire on record. You're fairly safe with any Karajan recording unless its a Baroque composer.
For the Baroque I recommend period instruments. Period instruments mean that the instruments used on the recording are from the period of which the piece was written. Unlike most, I prefer to hear my Bach keyboard works on harpsichord which they were intended for, not piano.
For violin music I recommend Itzhak Perlman as a reference point. His tone is very thick and lush. Perlman shines in almost every romantic violin concerto I've heard him play. Other violinists such as Anne Sophie Mutter or Maxim Vengerov and the great Jascha Heifetz are more interesting in different ways, but as reference violinist try Perlman
For chamber music, specifically string quartets there are many great ones, but I feel a great quartet that is active today and shines in everything they try is the Takacs Quartet. No they don't make the russian composers sing like the Borodin Quartet, and they don't do modern composers as well as the Emersons, but they are really good all round.
Great Chopin pianist: Arthur Rubinstein for the old world style or Maurizio Pollini for a more modern virtuosic approach
Great Beethoven conductor: Carlos Kleiber or Herbert von Karajan. No one does the 6th like Bruno Walter though. And if you want to hear who the world will probably always consider the best Beethoven conductor ever recorded, try Furtwangler, but don't expect sonic glory.
Great Mozart pianist: Murray Perahia for the concertos, Maria Joao Pires for the sonatas
Great Haydn conductor: Sir Colin Davis!
Great Sibelius Conductor: Sir Colin Davis again....Neeme Jarvi is excellent as well
Great Mahler Conductor: There are many great Mahler conductors and I feel Mahler is the only composer which when buying a complete cycle you really do yourself a disservice as I can't think of a single conductor who mastered all 10 symphonies. Bruno Walter, who knew Mahler most intimately along with Otto Klemperer are great starting points for a Mahler collection.
Great Brahms conductor: Bruno Walter is a safe bet with Brahms. Carlos Kleiber's 4th is almost universally considered the winner though.
Great Bach ensemble: Tafelmusik......Tafelmusik tends to be fantastic with any Baroque music.
Great Bach harpsichordist: I personally love Pierre Hantia, but I have to mention Wanda Landowska
Lastly, I think you should try a Wagnerian opera just to have that experience. My favorite is Tristan und Isolde. And the greatest recording ever made in stereo is Karl Bohm's
Everything must go by tomorrow!
