Both PS5 and XsX controllers will deliver sound through headphones, the XsX one will be louder.
But you can't expect the amps provided in a $60 controller to be decent, they are what they are, they provide sound in an affordable way.
Also, volume control doesn't give you much control, it goes a lot louder or a lot more silent very fast
Adding an amp between the controller (portable or desktop depending on your setup and needs) will not only give you more power to drive better headphones and drive your headphones better, you will have improvements like more impactful bass, better positioning and retrieval of detail, clarity, etc .... And more granular volume control.
I'm not playing the audiophile here, it's just something that is immediately noticeable, it's a totally different experience.
And a lot will depend on the amp as well though I suspect a lot of portable amps will already do the job well.
A dac/amp expands your options even further like I use it with the XsX and optical, PS5 will maybe allow that via USB later
Keep in mind there are other solutions as well: dedicated wireless headsets (I suspect Penrose to be at the top of them whih is why I ordered those), adding a G6 like MLE described. Maybe the SXFI Amp will work as well, I can test that when mine comes in.
As for different headphones: not only the impedance is important, also the sensitivity. Take the $7000 Susvara for example: only 60 ohm but I'm pretty sure you won't get much volume out of those with you controller, people drive them out of the speaker outputs 10W Class A tube amps to sound good.
So yes, you will get sound out of your controller with any headphone but it won't be good. It's a budget solution but it works.