Your last statement has some truth to it - although only in a very limited way - and only in a limited number of circumstances.
In many cases, running an amplifier at a higher output level will cause it to "wear out" slightly more quickly, mostly because it will be generating a tiny bit more heat.
Of course, the actual effect will be relatively small in most cases.
(I don't drive my car every day at 50 mpH... even though I'm sure it wears out a tiny bit more quickly at 55 mpH.)
However, the same is almost never true for input circuitry, or for the output circuitry on most preamps.
In low-power applications, as long as you avoid overloading anything, the signal level has negligible effect, or no effect at all, on either circuit stress or life expectancy.
(If the circuitry is operating in Class A then it would have no effect at all; if it's operating in Class A/B then it could have a tiny effect.)
Yes, a power amp, and a speaker, both "have to work harder" when the average level is higher.
However, that really isn't true for devices like CD players, DACs, and preamps.
Either they can handle the highest peaks without clipping - or they can't.
In fat, in almost all cases, even a moderate amoutn of clipping won't cause any harm - other than a nasty sound.
(I said "almost" because there are a very few small signal circuit configurations where clipping can cause actual damage.)
And, yes, when you're talking about SIGNAL levels, the level you concern yourself with is the maximum peak level...
If your circuit cannot pass the maximum peak levels then it will clip on the peaks...
If your circuit CAN pass the maximum peak levels without clipping, then it will also pass all lower levels without clipping.
The only "real argument" against the current trend is that we find it to sound unpleasant and unnatural.
(And that's enough for me to consider it to be "a bad thing"... but I will concede that as being merely my preference.)
But something else HAS gone up, over time, within that 96dB container we call Red Book: The AVERAGE level, which my repeated mention of here seems to have had thus far zero effect on your understanding of how we hear. You continue to fixate on peaks, which again, while louder than average level, are mostly fleeting in their existence during playback of a recording.
Higher average levels = higher average voltages, which can indeed cause longer-term strain on both input and output stages of even a moderately expensive stereo receiver or amplifier.