As much as I love tubes, with the advent of low noise high voltage Schottky diodes, it's pretty hard to make a case for vacuum rectifiers anymore. The only remaining advantage of vacuum rectifiers is the inherent soft start, which can be accomplished by other means.
Tube regulators are more of a gray area. Gas regulators are noisy, inaccurate, and bulky. Zeners can be noisy too, but the noise can usually be easily shunted with a large capacitance that cannot be used on a gas regulator. As a pass device in a series regulator, a tube is going to require a seperate filament winding and eat up 100V or more of the B+ and generate a bunch more heat. A solid state device in this position is more accurate, requires only a relatively small voltage drop, has no filament requirement, and is much less bulky. The solid state regulator will be smaller, cheaper, and measure better than a comparable tube regulator (although it is not difficult to build a tube regulator , that at least for audio purposes, has excellent measurements). The tube regulator has the advantage of usually being more rugged than its solid-state counterpart. Some people, me included, feel that solid-state series regulators, no matter how well they measure, can impart a signature (usually referred to as a glare) to some audio circuits.
The underexplored area in the high voltage regulator field is the shunt regulator.It's rarely used in audio circuits, but it might be worth looking at for a "no holds barred" electrostatic amp. Both vacuum tube and solid-state shunt regulators seem to be free potentiental sonic effects. The big disadvantage of shunt regulators of either type is that they consume as much power as the amplifier itself. I'd like to see a professionally designed hybrid shunt regulated power supply that was capable of driving a top tier electrostatic amp. Everything solid-state except the shunt device, which would probably have to be something considerably beefier than a single EL34.