Sure it's possible. Find the voltage breakouts for all of the connectors for a standard power supply and build a battery array to feed them (which will be your more power efficient option, but also the most difficult option), or get one massive battery and connect a DC/AC inverter to it to power a regular ATX power supply (which is easier, but less efficient).
Why do you want to do this anyway? They cost of the batteries is going to be more than enough to justify buying a notebook computer.
It would be very impractical to use a battery to power a normal desktop. For just the +12 V side a Deep cycle PbO battery might last 4 to 6 Hrs and that's a He** of a big battery. Of coarse this depends a lot on the GPU. My 9 cell Li-ion battery is rated at 65 WHrs - also only has 10.5 VDC out. A very Low end slim line Desktop normally has a 250 W psu. 3 $100 Li-ion batteries in parallel = about 30 -> 45 min run time (forgetting there output is only 10.8 V) You could build your own as frozenhead said, but it would be BIG and runtime would be short. You would buy individual cells, place them in series to give the desired voltage, and place a number of these in parrallel to yield the Wattage rating.
Also as frozenhead indicated you could run an inverter off of a battery. Again not to practical. If the Desktop only drew 50 Amps (+5V and +12V rails) - 50 amp/hr batteries - well you get the picture.