I've got more seriously involved into headphones after spending many years into building a good-enough speakers system, which has been and still is my primary mean to listen to recorded music.
I don't think I prefer one approach over the other, both have their advantages in specific points. For instance, for the money you can get very good phones providing an excellent level of detail retrieval, resolution, frequency balance, and sensible low distortion level, you can't even dream getting all that from speakers. However the degree of bass punching in your guts (if that interests you, not my case) or the space, I mean the feeling of music sounding in front of you and not inside your head, that quite affordable speakers can provide, can't be had using phones, no matter how much you spend in the whole rig.
IMO both things are enjoyable and have their use for any music lover. If forced to, I could live listening just to headphones, but fortunately I still can enjoy my speakers systems as much as I like, but not at any time of the day hehehe.