Well, the thing is that microphone, recording tape, and speaker technology was far more mature than headphone technology was then - only recently have headphones begun to catch up.
Give a listen to, say, Blue Train, which was recorded in 1957. Even today it's far better recorded and mastered than 99% of new albums. Not because of the technology, but because most pop, rock, and some jazz & classical albums are compressed and clipped (at the very least) until they sound like crap. Well, that and Rudy Van Gelder is among the best recording engineers of all time (and he built some of his own equipment back then).
Now, there are plenty of horrible, horrible recordings from the fifties and sixties. The new "big thing" then was stereo - well, we all know how that panned out, but at the time it led to lots of poorly done stereo releases of albums that were originally mastered in mono by the bands. There's also plenty of poor recordings that were just made quickly, with low budgets, poor recording engineers, poor equipment, and so on. That still happens today, of course - it just costs less to do.
Regarding playback in that era - the best systems were huge electrostatic panels with limited bass (solved in 1968 with Infinity's development of the subwoofer), and a handful of dynamic speakers that still perform well today. The most famous of those is probably the Klipschorn - designed way back in 1946! You can still buy them new, and only minor changes have been made to the design since then. They're still arguably Klipsch's best speakers today, and although they're not groundbreaking like they were then they're not any lesser for it.
There's honestly not been as much of a revolution in performance since then as you'd think - more like a slow evolution, with the best systems from the 1980s not really entirely eclipsed by the best performing systems today. Since then, the convenience (not so much the quality) of the CD, cassette, and digital music really have shrunk the market for high fidelity. People want to hear their music on the go - most don't sit down and actually listen to the music anymore. So accordingly, the mass market standards have shrunk to give people convenience. How else can you explain soundbars?!?
At the same time, the portable revolution inspired an entirely new market for headphones. It's taken time for portable audio to grow big enough to support a high-end niche market. Sure, there are/were great electrostatic (and of course a handful of great dynamic and orthodynamic) headphones, but they've always been tethered and expensive, so of limited interest compared to stereos. Anyway, so there's generally been this huge explosion in high quality headphones , which still seems to be gaining pace (at an increasingly fast rate, even). I think the exploding portable market is influencing the market for home headphones as well - young people that are used to high quality portable headphones want even better ones at home, leading to an increasing market for high-end home headphones as well. Don't get me wrong - I'm not trying to say that it's all young people or even mostly young people that are buying all the high-end headphones - it's just that I think that the expanding market for high quality portable headphones as a whole (driven by the younger generation) is influencing the home high-end market as well.
Beyond all that, no matter how bad the recording, it can still benefit from more faithful reproduction. Okay, well there's some that could stand to have harshness or other flaws hidden, but you still gain to hear more detail with better headphones (of the right sound signature).