Ok, I now have this sorted.
I've taken one up on Sony working right around their system making the Walkman run on external / interchangeable batteries and Its quite an easy painless process. What you need is a 14500 3.7v battery, the kind used in some high powered torches. Be aware these batteries are not your regular 1.2v or 1.5v AA battery used in common devices but instead 3.7volt. They're however the same size as an AA battery.
1) Here's an example of the battery used: http://www.ebay.com.au/itm/Free-Shipping-2x-1200mAh-14500-3-7v-Li-ion-Rechargeable-Protected-Battery-New-/201143743937?pt=AU_Electronics_Batteries_Chargers&hash=item2ed519f5c1
2) Another thing you'll need is a battery casing to put it all together.
Here's an example: http://www.ebay.com.au/itm/271633240002?ssPageName=STRK:MEWNX:IT&_trksid=p3984.m1439.l2649
(keep in mind you only need one battery, the above casing supports two though I set it up for a one battery assembly)
3) Last thing you'll need is the original battery harness which was attached to the stock Sony battery, this is very important as the Walkmans mother board connection depends on this proprietary plug, without one you'll be pushing to get this job done.
When you have everything, open the Walkman then carefully drill a hole in the back of the Walkman casing about 1cm from where the connection is on the mother board. Poke the wires through and solder the original battery harness to the positive and negative wires, be sure to tape them off or use heat shrink you don't want any shorts inside the Walkman.
There will be a white wire as well on the original harness, disregard it, chop it off clean or tape it off. You don't need this as I'm aware it's just for monitoring charge and temperature purposes. All you want is the red / black wires.
Attach the battery compartment to the back of the Walkman, I used double sided tape for now but will secure it with something else later (probably dual lock velcro). Make sure you don't plug anything in until you have everything soldered taped off and secure!
Here's what the Walkman runs off now, an interchangeable 14500 3.7volt -1200mAh battery, there are others at around 2300mAh I'm going to try. Keep in mind again you do not need two batteries, this was just the best case I could find and I've altered the wiring so the case works with one. You could even try the next size up 3.7volt 18650 batteries which go up to 4000mAh, I simply used what batteries I had here for the time being.
Edited by H20Fidelity - 10/21/14 at 5:21am
And there you have it, one Walkman I've been listening too all afternoon.
See, because the original Walkman battery was 3.7v 1280mAh, we're basically using exactly the same thing (minus 80mAh in this case) so there's no reason why it shouldn't have worked. Best of all, if the Walkman goes flat you can simply hot swap to a fresh battery, hell, you can carry 4-5 if you want in those battery storage cases and take it camping. Now the Walkman can live way past any internal batteries life span. Keep in mind when you connect to PC for file transfer the battery still will not charge, this must be done externally with a 3.7v battery charger.
Run times with 1200mAh 14500 battery are being tested and so far proving well.