I'm under the impression that wired will always sound better than wireless, though.
In general terms, yes.
The design for a wireless earbud has sensitivity as the utmost priority that way it will need less power to get loud, which, along with how that affects battery life, is what the market for wireless will typically look at on par with quality.
For wired designs they tend to sacrifice sensitivity if they have to if it means keeping the response flat, like having one higher freq driver being more sensitive than the low freq driver/s it's paired with, so they program the crossover to weaken the signal getting to the high freq driver. Or just a single dynamic driver that opts for integrity while it moves rather than lightening the diaphragm.
That's not where that ends though and why it's hard to assume the above will always be the case. For one you can make the earbuds so small they can't have a good battery to last longer anyway depending on how loud you listen vs their lab test and reviewers. On the wired side, sure the response of the earphones tend to be smoother or tweaked to compensate for how different IEMs are vs speakers (ie the Harman curve, or at least the left half of it, is more crucial to personal audio than speakers), but if you put it on a source with low damping factor then the power it gets is problematic, on top of which, for many IEMs, being IEMs means they get 105dB/1mW or higher sensitivity so 15mW from a smartphone or tablet goes a long way.
So you can have a fairly decent smartphone driving a decent IEM and not have trouble, but then if you drive it with one where the damping factor drops due to high output impedance, then it can sound worse (ie more distorted, lower volume) vs a wireless unit.
I'm just not sure how much of the difference between S8/wired and S20/TW to attribute to the phone's dedicated USB DAC, and how much is the wire or its lack. I guess they're related, in that the USB DAC is only used for a wired connection...
Your phone doesn't have a USB DAC. It can't even function as one the way the LG module can still be used as such when the phone breaks.
What they have is a DAC or if you used an OTG cable, sending the signal to a USB DAC and a headphone amp. The phones are like an AMD APU that has the motherboard's 4-Pin or 4+4Pin and x+y phase VRM powering it while sharing system RAM, unlike using a CPU that will use the 4(+4)-Pin power and X phase VRMs and having 16gb of memory while the GPU is on a separate board taking PCI-E power through the slot (which takes its power off the 24pin) and its own 8+8 power connector, running that through its own 7+ phase VRM and has its own 8gb of HBM or GDDR6 VRAM.
The difference though is that one has a DAC that still has a simple headphone driver circuit built into it, even that whole audio circuit is built into a Snapdragon chipset, while the other regardless of what audio chip it has doesn't have that output connected to a 3.5mm line out, just the built in speakers on the phone.
It's discouraging to me that my cheaper S8/wired combination could sound far better than the more expensive S20/wireless.
Well that's because you're using wired earphones with totally different driver design that is either designed to maximize sensitivity without sacrificing response, or you're getting the kind of response that you prefer, while the other either sacrificed response or is subjectively better to others or objectively better but not your preference either way.
And besides the added cost there isn't that the wireless is better, it's just because it's wireless.
Think of the S8 and wired earphones you like as a Lotus Elise being what you want - light, agile, fast enough. The wireless is convenient and full of features to be so, like say, a Camry V6.
The S8/wired is the best I've found so far, but... my intention was to invest in a price range and technology that by now must sound even better.
Moore's Law only applies to computer chips, and even then that kind of performance when other factors are involved doesn't work.
I mean that's like expecting a Ryzen 5 4500G to beat an old desktop using an i5-8600K and GTX 1070Ti barring choking the latter so it thermal throttles.
Or expecting a hyper convenient Camry or Accord to beat Lotus Elise with an old 1.8L Toyota engine or an equally old S2000 with a 9000rpm engine when the latter pair are lightweight RWD cars that will go around corners a lot faster.
Is it common to have such a hard time finding what you're looking for?
To put it in perspective, Amazon banned somebody here after buying and returning two dozen headphones or thereabouts.
Is it just because sound profiles are subjective, or is my perspective still incomplete?
Either your wired earphones match your subjective preferences while the wireless might be objectively better, or your wired earphones can be subjectively good while the wireless are also objectively bad; or it could be your wired earphones are objectively better.
That gets worse if you don't want any compromises on convenience and size.
Regardless of which one it is, going with wired earphones is more likely to get something that is objectively better unless you can pinpoint your subjective bias (which may still be easier to find on wired headphones) and you can be directed to that, and then just get a DAC+HPamp to drive it. You'll really just have to accept some compromises, like "larger Q5 Pro" vs "not all-day BTR1 battery life."
