AFAIK, you can take a battery completely out and still boot up, just no BIOS settings will be saved and you will have to detect all the hardware every time for POSTing.
I could be wrong, though.
If you have used a PSU that is working on another computer on the broken computer and it still won't turn on, its likely one of two problems
1) The *new* one doesn't have enough juice to run your PC
2) The motherboard is blown and you will have to replace it.
I have fixed probably 600 computers professionally and I don't remember a single time when the problem was the battery.
On the other hand, more than 50% of the time it was a problem of needing a new PSU. Another 25% of the time it was probably the motherboard going bad.
If it is an old computer, I would guess the PSU, because those tend to be the first things to go.
Tell us what PSU you had before that doesn't work (maker/model), the *new* one you tried that worked somewhere else but not on your computer (maker/model) and the maker and model of everything else in the computer and we can give better advice if you need more than this.