LPD is my favorite band (Sonic Youth second). I'm going to see Edward Ka Spel's solos set this week, in fact
My favorite LPD period, though, is the the earlist period (1980-1988), while most people seem to feel that the late-80s to early-90s was the best period (likewise, many feel that sonic youth ca 1987-1990 was the best, while I love the first 4 records most.. plus Goo)..... I love equally but differently the oldest regular releases and and the demo-ish friends-only tape-only releases of LPD. I really dig a lot of the albums after that but I always go back to the oldest releases and cite them as my clear favorites. They churn my imagination the most.
You'll find that there are themes that run through the albums. You have Towers 1-5 on The Tower. Tower 5 ends the album with "You chose your grave. Now lie there.. lie there.. lie there.." And then you find Tower 8 on the Edward KaSpel album Lyvv China Doll.. That track beginning "Lie there face down and be quiet.".... This is only one of many, many threads.. they bind the KaSpelian world together. ..Don't get me started on the Lisa theme.. it'd take a while.
Though it seems you've probably begun shopping already, I say check the song titles of the release you borrowed. Match them to an album and start there. I got many of my LPD/WSD CDs from Revolution and some from Other Music when I lived in NJ and visited NYC a lot. (I hope those places are still around)
Below are my favorites, though I like every one of the 40+ related releases I own.
====My favorites.. Depending on my mood any of these I'll say is my very favorite
Only Dreaming (tape-only, kinda-polished, but the low-fi of tape is a factor there. will be reissued soon on CD)
Under Triple Moons (a reissue of unpolished stuff)
Brighter Now (regular release)
The Tower (regular release)
Curse (regular release)
Chemical Playschool Volumes 3+4
Legendary Pink Box, especially CD2 (a reissue of unpolished stuff plus some polished re-recordings of old songs)
EKS: Chyekk China Doll
EKS: Lyvv China Doll
Prayer For Aradia (more unpolished old stuff)
Traumstadt One (more unpolished old stuff)
====My next favorites.. I love these immensely, but not as much as those above
Stained Glass Soma fountains (more unpolished old stuff)
Any Day Now
The Golden Age
Basilisk
EKS: Tanith And The Lion Tree
Other stuff........................
Coil.. I only have Scatology (love it!)..... I'll get more. I know I should.
C93 I have never gotten into, really. Emblems, Lucifer Over London, Dog's Blood Rising and a few others and I still have not been compelled to listen to any all the way through. I'll try again.
DIJ... I like their stuff and I find that the CDs I own have a lot of duplication. I have (the 2CD that was sold on their 1997 USA tour that was reissued as--) Discrimination and it's very good. Don't know if Doug P (not SIE) is Nazi or whatnot. He's gay, though, which makes his possible Fascist/Nazi leanings a little queer. heh
You didn't mention them/him, but I REALLY REALLY recommend Blood and Flame by Non. To call it noise is correct but misleading. There's structure and it is scary. It's my favorite noise album and it falls into the WSD-type world. It is Noise, though, so it's a bit different from the ones you mentioned above. Boyd Rice (the heart of NON.. well, he IS NON) is possibly of questionable ideology also, but damn, what a great album.
Oh, and Karl Blake/Lemon Kittens.. Wonderful Stuff, IMHO.. The Karl Blake Studio Archives (Prehensile Tales and especially Paper Thin Religion) are great CDs, but they're quite different than most WSD stuff. Very playful.. often quite dark.. quite inspiring if you're making music alone.