it will depend on device / OS etc but 20% is the rough number I use for most part so I don't have to remember this is that, that is this, etc. It may or may not cause performance issue cause there are other factors involved like how many files, folders, or more specifically, in that specific directory where you're trying to access..ie. if all your music is in one directory and sub directory, this will cause slow downs as it's trying to read/search multiple files at once. Also, not sure of Android, but back in like something like XP, NT days of Windows, the more directories deep dive you have to do, performance also tend to crap out too.
So i guess i should've been more specific
1) Try to keep 20% of storage space free - mainly for cached operation reason / fragmentation issue --- if you have larger files, the % can probably be a lot less, but we're dealing with MP3~flac most likely so the blocks are small per file, especially since it's repeatedly and rapidly reading the music file to play them
2) Do not have 1000s of files in one directory (note: FILES, not "directories", although i'm sure directories will affect if there are too many of that too---for me, i probably don't have over 200 lines in a single directory, especially since i categorize my stuff in Japanese so it looks something like JPN/JPOP/#よ(phonix)/@米津玄師(artistname)/Albumname/musicfiles)
3) Try not to make a Deep Directory dive structure (not sure of Android, but I believe Windows 10 and 11 has a limit of something like 512 letters for directories + file name together? i know that the Sony Music Center is even less with like 256 or so <i can be wrong with the numbers but you get the idea>)
If you're unsure, try factory resetting the device and store all your music in the SD card, while keeping #2 in mind (#1 should not be as important when it's on the SD Card, unless if you're running the system of the SD Card--still do not recommend to top if off, maybe try to keep 7~10% of free space at most) -- see if it performs better. If not, you might just have a faulty unit as it does happen no matter how good or bad is..it just happens.