What he said. Headphones make a bigger difference than anything, then amp, then dac and/or source (last two are arguable), then you get into the insane nitty gritty with cables and power and the like.
I agree.
Headphones will make the difference and it should be your first investment.
I do it like this:
I choose an headphone that will play a certain range of music that I like. This is the same way I choose my speakers. In between that I buy other stuff just to mess around with.
I get an amplifier that is way more powerful that what my headphone can handle, just like with my home stereo setup. This way you never run out of head room and in this case you will have bass for days. Distortion(clipping your amp) will kill your headphones/speakers first, before you burning out the voicecoils from over current.
The source has to be good. There is no point of amplifying crap. I said good, not high end. I listened to high end sources and sure they sound good, but the sound difference is just too small for that big of an investment. And since your are EQing, what is the point really.
Dac's, for me it has to be a good one with a sound signature that I like. Again, high end stuff is not needed. My favorite dac is the old Ibasso that I'm still using. Looks funny in my home setup, but dammit I love it. It's just hilarious seeing that little dac/amp hooked up to some fat Monster RCA cables that plugs into a Henry Engineering Match box that converts to a balanced output.
Cables, yea the do make a difference. The thing is that you don't have to spend an arm and a leg for good cables. Silver enhances your higher frequencies a bit. Good copper gives you that nice deep lowend. This is what my ears and my kids ears hear. Now if you can't hear a difference, don't spend the money. I use a variety of wire gauges throughout my system, because I hear no sound difference with certain equipment in my rigs when going to thicker cables.
Connectors, well I personally like tight fitting connectors. Is there a sound difference? Not that I can tell.
So I recently chose to my eyes an ears a decent Planar headphone that will scale up with power and sound processors. Again, I didn't want to spend too much either. So it became the Hifiman 400i. By default it is too "bright" for me, so EQ to the rescue. It plays well with my Fiio E12A. Sounds louder with my Fiio E9, but lacks subbass that you can feel in your cheek bones and lower jaw. Subbass authority was still lacking even with the use of a BBE processor and EQ. So a better amp was needed to supply the needed current to push the Planars properly to the subbass nirvana. I made a variable resistor box that hooks up straight into 40W per channel NAD amp. Let me tell that I can reach the excursion limit of those planars with the sub bass now. You just don't realize how deep you can push your bass when you are not power limited. The planars play clean right up to the limit. Is fun to have a open back headphone that can satisfy your bass head needs and massaging your face properly, with sound stage to boot.
So yes, amps make all the difference, especially for the Basshead group.
This is my Bass station with "pretty" lights. Not mobile ofcourse. My mobile bass setup consist of Iphone 4s, Fiio E12A, BBE app, Kaisertone and Pioneer vibrators.