I disagree entirely.
Excluding some niche audiophile products and options, any "competent" DAC is going to have a very flat frequency response and very low distortion and noise. The differences you hear are going to be due mostly to the differences in how they handle transients. Virtually all modern DACs use digital oversampling - and the benefits of doing so far outweigh the drawbacks. However, all digital oversampling filters create errors in transient response. (They reproduce a sine wave perfectly but, if you feed in an asymmetrical and irregular waveform, they change its shape. Mathematically, all of the energy is present at the right frequencies, but some of it arrives at slightly wrong times.) If you look at the brochures for a lot of DACs, they will show you oscilloscope pictures of transients... which look sort of like a short sharp spike with some ringing before and after the main peak. That ringing is energy that has sort of landed in the wrong place - or, more correctly, at the wrong time. Since it's pretty close to when it should be, we don't hear it as "extra sound at the wrong times", but it does affect the overall sound of what we're listening to. In fact, by designing the filters a bit differently, you can trade off various aspects of these errors, but you can't entirely eliminate it. (For example, you can mathematically "trade" "more post-ringing for less pre-ringing".)
Many DACs even offer you the option of choosing between different digital filters, each of which reproduces transients slightly differently. (Typically, if you were to feed a sharp spike into the DAC, one filter will reproduce it with some ringing before and some after, while another option will have no ringing before the spike, but at the cost of more ringing after it, and a third filter may have less ringing altogether - but at the cost of a less flat frequency response.) While certain filter choices may actually affect frequency response, most alter transient response while avoiding changing the frequency response significantly.
If you listen to a commercial DAC which offers multiple filter choices, and switch between them, you will indeed notice a subtle difference. It's quite subtle, and you may only notice it with some recordings, and only with some headphones and speakers, but it is quite repeatable when it's present. Listen to a good recording with wire-brush cymbals; with some filters they'll sound more like actual metal while, with others, they'll sound more like a steam valve going "tssss, tssss, tssss". In some cases, the sibilants that accompany voices will sound different - and more exaggerated with some filters than with others. Remember that, mathematically, all of these choices are "really close" but none of them are "perfect". Of course different DAC products also have different analog circuitry, which also accounts for slight differences in sound.
I would also remind you that these differences are very subtle - and far less than, say, the differences between speakers, or headphones, or phono cartridges. To be honest, if I switch back and forth between filters, while a song is playing, the differences seem obvious - eyes open or closed - but, if I were to walk out of the room and come back, I very much doubt I could tell you which one was playing; that's the degree of difference we're talking about here.
The types of differences you hear between DACs are also rather different in type from thos you hear with amplifiers or speakers, so you may not notice them unless you actually listen for them. (And, yeah, they are about as miniscule as the differences I typically hear between DSD and PCM versions of the same recording.)