Why does the sound become crappy just because there is ambient noise? I wear my Grado SR60 outside walking around. They're open headphones. I'm not blasting the headphones to compensate for the outside traffic noise. I can still hear the car traffic. I can still hear people talking. The music sounds fine. I'm enjoying it. It's not overly loud. I don't consider it in any way to be a crappy experience just because there is background noise. In fact I rather like using open headphones outside and still being able to hear my surroundings rather than using IEMs and becoming oblivious to what is around me. The hyped clarity and sound of the Grado is actually nice in that situation.
Headphones aren't inherently dangerous to your hearing. It's all in how you use them and being aware of the volume. At a reasonable volume you can listen to headphones all day long, even closed headphones or IEMs.
You can use an
SPL meter to measure the volume of full size around the ear headphones. It's simple and a good thing to do so you can get a good reference for what 70 db, 80 dB, 90 dB and even 100 dB sound like on headphones. It's different than listening to those volumes on speakers. Speakers let you know when they are playing loud (you can feel the loud music in a way you don't feel with headphones). So if you are relying on that sort of physical feedback to let you know when the headphones are too loud you will be playing the headphones way way too loud. Learn to realize that headphones get loud differently and you can learn to keep the volume reasonable and safe.
Listen to big speakers in a room that has lots of sound treatments on the walls, a carpeted floor, a comfy plush chair, and a "dead" sound (sound treatments absorb the reverb and extra bass vibration) and you'll find that you are listening to the speakers louder than you expect. That doesn't mean that acoustically treated rooms and speakers are bad for your hearing.