That's pretty much how I look at it. I don't imagine that it is an accurate representation of reality, but I do expect it to support a sort of illusion, and it needs to do that well, and avoid anything which, to me, "jarringly breaks the illusion", or even which "distracts from the main story line".
I would even carry the analogy to an ordinary TV picture. What we see on TV probably has very little to do with reality, with virtually every scene in a movie either staged, or built outright in a computer these days. However, it's still distracting if a scene is "too dark" or "too light" or "a funny color".... and, if your TV were to develop a big blurry spot in the middle of the screen, it would be obvious that it was a technological artifact, and there would be little question that "maybe it belonged there". Likewise, there is no mistaking a dirty smear on your car windshield for part of the scene outside.
Perhaps an even better analogy would be the glass over a painting. The glass you normally see over pictures and paintings is relatively clear, but it is still subject to surface reflections, and fingerprints and dirt. You very rarely see a picture frame where you
LITERALLY CANNOT TELL THERE IS GLASS BETWEEN YOU AND THE PICTURE. You may not consciously notice a color tint, and you may be standing at an angle where there aren't too many reflections, but when you look you can almost always tell that "there's glass there". You move your head a bit, and there are reflections, even though your brain may edit them out pretty effectively when you're standing still.
However, if you go to a high-end framing shop, there is an option called "museum glass". It is extremely transparent, with no noticeable color tints, no odd warps or wobbles in it, and is treated with an anti-reflective coating. If it's clean it is quite literally invisible to the human eye. You can shine a flashlight on it, and hold it up to the light at different angles, and you just plain can't see anything there. In short, it carefully avoids pretty much all of the cues that normally tell us that "there's a piece of glass there". You usually don't notice the windshield in your car, or the windows in your house, but you can usually tell relatively easily when one or the other is missing.... because the lack of all those cues you've learned to ignore is actually rather obvious. To me, that's what a good audio system should do with sound, at least to the degree that it's possible. Audible artifacts are the equivalent of "dirty fingerprints on the glass", and a good audio system should minimize them, or hopefully reduce them to the level where they really are inaudible. And, like museum glass, it should also avoid the artifacts that you normally don't notice, or have learned to ignore, but still distract from the experience at other levels. (And, even if your brain does a decent job of "editing out the extra junk", I suspect that many of us enjoy music more when our audio system avoids adding extra distractions. Perhaps we get to enjoy the real details more when our brains don't have to expend as much effort "editing around the flaws".)
I view headphones, and even speakers, as providing something like a virtual reality experience, so I focus on whether they provide a good experience, rather than 'accuracy' in some imagined or absolute sense. Things that can definitely diminish the experience include whacky tonal balance, noticeable distortion, lack of detail, and incoherent imaging. But sometimes the combo of the recording + the headphones + my ears and mood lines up really well, and makes for a sublime musical experience.