Jitter is consciously perceptible at fairly high levels. Any modern DAC, even little baby Chinese DACs, have a clock that are good enough to reduce jitter to levels that are not perceptible as jitter. So the battle isn't with old-school, extremely obvious jitter.
Instead of just explaining how jitter at low levels matters, let's see what respected industry manufacturers have to say:
MSB (builders of what many consider to be the finest DACs in the world):
"Our years of experience taught us amplitude (DAC) precision contributes to the realism of the instruments and voices while low clock jitter decreases ‘digital harshness’ and increases focus. Jitter simply takes the pleasure out of the music and creates listener fatigue."
Antelope Audio (famous in high end pro audio):
"Leading producers, sound engineers, artists, and movie post production studios worldwide are using the 10M Atomic Clock together with our high-end Trinity master clock. They usually describe it as “mind-blowing”, an “instantaneous improvement”, “superior”, and fundamental in bringing “more detail, ambiance, and naturalness” to the digital signal. The results are described as “wider and more solid” with “crisp transients and a 3D depth of field.”
---
Low-level jitter does have an impact on audio perception. This is well-known not only by manufacturers, but by DIY'ers as well.
A comparison with inter-module distortion is handy. It's known that IMD isn't generally audible in modern equipment -- because the brain filters out IMD artifacts, but IMD is seen as negative because it is theorized to lead to listener fatigue (it's a theory because it involves neuroscience but it's still a widely accepted view). You won't hear jitter, or noise in general below a certain threshold -- but that doesn't mean that it doesn't effect you.
I am certain beyond any shadow of a doubt -- if readers of this thread were to plug their headphones of choice (well, headphones that aren't trash anyway) into my reference system, the difference between it and their DAPs, little integrated units, etc, were would very obvious. A lot of that difference will come down to the effort and expense gone into reducing jitter and noise in the system.