Where could I find a suitable "test set" of audio fragments for use in controlled level matched double blind ABX tests, or somewhat simpler level matched blind tests?
Just take any well regarded "HD Audio" recording and knock it down to the formats you're comparing to... 16/44.1, AAC, MP3, etc. Then bump them back up to HD Audio and normalize them to the same place. Any audio editing program will be able to do that, and you'll end up with files that all look the same, as long as the test participants don't peek at the waveforms.
All true, but your listening choices don’t generally represent the market that streaming is currently targeting. I don’t agree with your assessment that streaming services are largely good for “greatest hits”.
The definition of where the lines for esoteric and "greatest hits" lie on the spectrum depend on where the listener is on his own musical journey. I know that I'm not a typical consumer. Just pointing out that some people are at a point where streaming just can't serve all of their needs any more.
I also think we are going to see a contraction in streaming libraries. It happened in cable TV and home video. At first, they can't put out new content fast enough. Then they realize some things don't have as large of an audience and they start weeding.
Every time a new format comes in, a lot of stuff gets left behind on the old format. There's a vast amount of material that never made it from 78 to LP, from LP to CD, and from CD to streaming. When I was starting out in jazz, there were series of CDs with complete runs of music by a particular artist... Every Fletcher Henderson side in chronological order, every Cab Calloway, Fats Waller, Erskine Hawkins, even John Kirby and Ambrose and his Orchestra. These were hugely important to me in my research into the history of jazz. Now there are some greatest hits CDs, and those are on streaming. The rest of it is all out of print and unavailable on any format. It's sad, because the technology allows for more complete archives, not less. But that isn't how it works. If it isn't being licensed for sale any more, it isn't available. There's no archive of past streaming content like a used record store.
I remember going to Tower Records in the 80s and finding amazing new things every week. I would come home with arm loads of stuff. Then it started getting thinner and thinner until it seemed like everything was like the end caps on the aisles... hit of the day. That may happen in streaming too. They also might start dividing things up into corporate streaming services... Universal will have theirs, Sony will have theirs, Warner will have theirs... and you have to subscribe to all of them, or choose one. Or maybe slimmed down streaming libraries and you have to pay more for "all you can eat".
I really don't care how I get music or what format it is, just as long as I can get to the music I want. My media server library of rips seems to be the best choice to depend on for the long term. I'm really glad I built my CD library when I did. At least half of it isn't on streaming. Yes, the half of it that is streamable amounts to a huge chunk of listening time... perhaps 10,000 hours. But when I want to hear something from that other 10,000 hours that isn't on streaming... say a specific Erskine Hawkins B side mentioned in a book... listening to a greatest hits of Erskine Hawkins won't do the job.
I guess music serves different purposes to different people. I love art books that have catalogs with photos of every painting in the National Gallery of Art or the Louvre or the Hermitage too. The Taschen complete Rembrandt sketches books, or complete Michaelangelo, DaVinci and Schiele are my meat. Best of Van Gogh books don't serve the same purpose to me.
Edit: One side note... part of the explosion of back catalog historical recordings in the 80s had to do with the copyright law. There actually was a public domain back then. When the copyright extension bill passed, and the Naxos case came down, it all changed. Hopefully, with the clock ticking on copyright again, we will see more older recordings again bit by bit. But that will likely be on non-corporate controlled services. I'd like to see the internet go back to being the wild west again. It's good for culture.