I remember seeing Fantasia for the first time at age 6. The visuals that went with the music made a very strong impression on me imprinting them together in my mind...I was particularly into the visuals and bleeding chunks pieced together from Stravinsky's "Rite" and it's conjuring of a point in time incomprehensibly long ago.
I heard the "Rite" performed live in my mid 20's for the first time in Philly, suddenly it all made sense. Instruments that normally carry melodies being used as if they were percussion instruments, percussion instruments providing a melodic flow, horns and woodwinds providing both and at times almost like punctuation, the crazy different simultaneous time signatures. Even the visual aspect played a part, as I viewed the stage the orchestra reminded me of observing the inner workings of a mechanical watch with springs and levers and gears all performing in unison their various functions perfectly, like a living, breathing machine. The greater the complexity the more it sucked me in. The dynamic range of the sound was unlike any reproduction system I had ever heard, (or would ever hear to this day). From that point on I got it.
I bought and heard my first Mahler (the 2nd) at age 26/27. Based on the album cover picture and a recommendation from a knowledgeable sale clerk at the record store (with very few exceptions these don't exist anymore, the stores or the clerks) after my telling him I was satiated with the Beethoven and Brahms stuff. It was Eugene Ormandy and the Philadelphia Orchestra (recorded 1970). My musical world was well and truly "rocked" and I was never the same after that, seeing it performed live the following year branded me a dedicated Mahlerite! Can't tell you how over the moon I was to obtain a CD copy (to add to 2 dozen other's I have) of this performance on a specialty Japanese label pressing a few years back, even has the original artwork.
Pop music figured heavily during the late 1950's for me, especially the "Rock-a-billy" and "Motown" stuff. It was always easy for me to come to a thumbs up/down decision fast on these 3 to 4 minute songs. ("It had a good beat and was easy to dance to, I give it a 95, Dick!") But it was in college with the "Airplane" "The Doors" and other metal guitar oriented groups that followed that I got hooked into that genre and it's (seemingly to me) greater complexity and richer sound. But I continued to listen to classical and pop side by side for a number of years. Somewhere along the way (70's, 80's?) pop music became too pretty-fied, cute-sy for my taste and the last 20 or 30 years or so it all became too overproduced and lost it's edge, and all same sounding to me. "Drive by Truckers-Decoration Day" was my last purchase in that genre. Over the past 2 decades Jazz has become of greater interest for me although classical and the late romantics and Impressionists in particular are still my musical meat and potatoes.
My experiences over 7 decades with music have taught me that I never know when something will "click". I revisited many composers (like Schoenberg and Berg) that didn't initially speak to me but eventually started to make sense and provide enjoyment, especially with the right conductors interpretation and musicians performance. (Although Verdi has just never made the cut for me, but my opera likes are fairly rigid and limited to just a handful of composers with a very few one-offs.) I never thought of the time spent re-listening as a second job, more a labor of love. Reading about the music and it's composer sometimes illuminated aspects that help me to understand where he was coming from and why, and what he is attempting to convey, be it programmatically or in abstraction.
It's about the journey.