I'm also of the belief that, generally, there is a law of diminishing returns and much that is out there in the world of audiophilia is overpriced to the point of insanity ($120,000 speaker wire, or $10,000 Ethernet cables, anyone?). Yes, you need to spend a bit on the core bits and pieces to elevate the sound and experience, but very good sound can be had relatively inexpensively. I'm grateful to have what I own (many audio snobs elsewhere would sneer and pat me on the head, but whatever) and for companies like Schiit that price their unique offerings so more people can enter the hobby and experience what others pay 5-10x as much to hear and be wowed.
Edit: as others have recently spoke of, take the time to place room treatments, run tools like Audyssey, and if you're so inclined, get a calibrated measurement microphone and learn how to use Room EQ Wizard.