There are different types and quality of "passives" just like different grades of active preamps. One can't lump them all in one b/c I've certainly heard passives that don't sound great, especially ones using poor-quality or long wires internally, poor volume control, witches, jacks, etc.
I've auditioned/owned many passives, including Bent Audio silver TVC (transformer volume control), simple DIY box with high-quality volume pot, Placette passive preamp, Endler passive attentuators, EVS Ultimate Attenuators, EVS "Nude" Ultimate attentuators, etc.
With all passives, even TVC, a "sympathetic" environment is needed for them to strut their substantial benefits. You need sources with stiff output stage and low output impedance, cables with low capacitance, amp with high input impedance, speakers with high sensitivity, or some combo of above that'll work well.
If such setup is achieved, my favorite remains EVS "Nude" Ultimate attentuators, which has an advantage over passive "boxes" b/c it bypasses an extra set of interconnects, connector jacks, internal wiring, etc.
To match or surpass the results of something like that in a synergistic system, one will require an active preamp of exquisite quality, usually very expensive. I'm talking about the likes of Aesthetix Callisto Signature preamp ($11,000), VAC Renaissance Signature pre, etc.