The question is "good or bad for what?", toroids & EI cores (those square ones you refer to) each have their own characteristics, advantages, and disadvantages, so as usual it comes down to what is most important for you in your given design or project. I will have to speak in some generalities here since there's millions of ways to wind transformers and make the various tradeoffs.
A toroid is more efficient, and for a given power rating it will be smaller and lighter. It will have lower magnetic leakage and throw out less electromagnetic interference. On the downside, toroids are a high bandwidth device, which is great if they're being used as audio signal transformers but not so good when it's a power transformer. This would not be a problem if we had perfectly clean AC power, but as we all know that rarely is the case, most AC these days is heavily contaminated and quite noisy. With a toroid, that noise is going to pass right through the transformer and into the rest of the circuitry, which is a really bad thing especially with tubes.
There are ways to reduce noise transmission, instead of winding the primary & secondaries on top of each other as is done in most toroids, we can place the windings on separate ends of the toroid so they don't touch, and if we really want to we can gap the core while we're at it. This will make the transformer less efficient though.
With an EI core there's often difficulty getting sufficient bandwidth in audio signal use, but for power use that's a good thing. The transformer will naturally roll-off and filter out a fair amount of the high frequency noise & grunge in the power lines. We can help this along some more by using a split bobbin, where the windings are side by side instead of on top of each other, and then gap the core for good measure. Efficiency isn't going to be good so the transfomer will have to be large and it will run rather warm. It's also going to throw EMI all over the place so you're going to have to shield the transformer with mu-metal or steel, in some cases several layers of shielding may be required.
Neither one of them is truly "better" in my opinion, rather, they each have their own tradeoffs and it's up to the designer to decide which works better in a given application.
