Old computer with XP.Soon as you plug in it turns on

this is something i've seen a lot if the toggle on the powersupply itself is left on (on the back of the pc by the power plug) you can flip that off and it shouldn't power on when you plug it in

1) Usually if anything happens when plugged in a fan starts up then the computer stays off. Confirm that it doesn't completely boot up?

2) If it boots all the way up it may be that your POWER switch is stuck? (I'm not sure if this is possible because it should then turn OFF but then the fault is weird)

3) I recommend using HIBERNATION if it works. There's no point in doing a complete cold boot if this function works.

Anyway, if the computer does start all the way up it's a motherboard issue and has nothing to do with Windows.

A cold boot goes like this:
1) Power button pressed (a Capacitor/Resistor or RC circuit acts as as timer, probably one second, which then gives power to the right location to initiate the BIOS program stored in flashable firmware on the motherboard)

2) BIOS is initiated (a mini program that figures out what basic hardware is attached and runs some basic tests, i.e. POST or Power On Self Test, such as testing the RAM)

3) BIOS last stage is to look for an Operating System like Windows or other self-contained program (Memtest, Backup App etc) from places like the hard drive, USB, floppy, CD etc. (hence the boot order)

4) Boots to Windows (Starts loading Windows and other program data into the System RAM and hard drive Pagefile. When completed it initiates Windows)

BIOS Setting:

There's a BIOS setting that applies to a power outage which gives you the option of restarting automatically or doing nothing. I wonder if there's a glitch causing it to restart thinking that the PC crashed due to a power outage?

You can try turning the BIOS option to auto restart to OFF and see if this helps.

If nothing at all helps you can always try to reflash the BIOS. Get the latest BIOS while you are at it.

Before that, double check all connections and jumpers. Mainly you should be looking at things to do with CMOS and BIOS.