I believe that human perception has limits. Those limits might vary from person to person. Some may be able to hear to the absolute edge of human perception, other people may have degraded hearing that limits them to a range below that. Variability extends *downward*, not upward. There comes a point where no human on earth can hear it. That is what we call the point of transparency. If you want to claim that the established thresholds are incorrect, start gathering evidence to prove it and publish it and if you're correct and your proofs are repeatable by others, the thresholds will be modified. If you think you personally can hear things you shouldn't be able to hear, present yourself to the audiology department of your local university and suggest that they test you.
It isn't logical or practical to pile one "fudge factor" on top of another "a little bit to grow on". That is what audiophools do to convince themselves that they need one more zero on the right of the decimal point. That leads to wasting time, energy and money on sound you can't hear. There is nothing better than audible transparency. If you suspect you're missing something, do a blind comparison and see if you can hear it. I bet you can't hear beyond the established thresholds. If you absolutely must build in buffer, then just follow the thresholds using tones. Those are significantly broader than the thresholds when listening to music. But for the purposes of listening to music, that is overkill.