I'm attempting to build a new PC that I'll be using for playing newer games that my current PC can't handle. I'm on a budget of about $800-$900 and this is what I've come up with. Feel free to give me any suggestions, I'd appreciate it.

Motherboard: EVGA 122-M2-NF59-TR Socket AM2 NVIDIA nForce 590 SLI - $130
Processor: AMD Athlon 64 X2 4600+ Windsor 2.4GHz - $120
Case: XION Solaris XON-406 Black Steel ATX Mid Tower - $40
Power Supply: Antec TruePower Trio 650w - TP3-650 - $110
Video Card: EVGA 320-P2-N811-AR GeForce 8800GTS 320MB 320-bit - $280
Memory: Corsair TWIN2X2048-6400C4 2GB Kit DDR2-800 XMS2-6400 Xtreme Performance Memory - $175
Harddrive: Western Digital Caviar SE WD800JD 80GB 7200 RPM 8MB Cache SATA 3.0Gb/s - $43

With all of the items listed above and some others the total comes to a little over $900 with tax/shipping.

Looks like pretty good choices. You might want to stretch a little bit and get a bigger HDD, 80 gigs will get filled in no time and the 320 gig drives are only $75~$80.
If you want to save a little, get an X2 3600+ and overclock it, or get some cheaper RAM. I have personally used this Geil in an AM2 board and it works great.
I don't know where you're shopping, but Newegg has that EVGA board with a $30 mail in rebate bringing it down to $99.99 after rebate.

I like the Coolermaster Centurion cases better than the Xion, they're about $50 instead of $40 but are nice budget cases.

If you're not committed to AMD, you can get the Gigabyte GA-965P-S3 mobo ($105) and an Intel e4300 ($119) for less and get better performance if you're willing to overclock it, although without SLI support. But you would be able to use DDR2-667 to save a little more $. But anyway, the AMD system should do nicely as well, and will beat a comparably priced Intel system if you don't want to overclock.

I don't know much about overclocking. If I were to overclock my cpu I'd have to buy better cooling for it instead of stock, right?

Depends on the CPU. The Intel Core 2 Duos can overclock a fair amount on the stock cooler. I don't know about the AMD 3600+, I don't have one and I think most of the people on here with them use non-stock cooling - but they also crank it up from 1.9 GHz to around 3 GHz, which is a rather severe OC. If you're uncomfortable with overclocking, the X2 4600+ is still nice at standard speed.

Yeah, I'll probably just stick with what I have and not overclock anything. About the RAM, do you think it'd be in my best interest to find something cheaper than what I have picked out? Currently, the Geil that you linked is out of stock, do you recommend anything else?

That Geil bounces in and out of stock, you might want to wait for it. They claim it will be in stock Friday. If you can't wait, try G.Skill RAM then:
http://www.newegg.com/Product/Prod [...] 6820231098
or some better but more expensive G.Skill:
http://www.newegg.com/Product/Prod [...] 6820231087
or maybe this Mushkin:
http://www.newegg.com/Product/Prod [...] 6820146599

It seems like a bad time to be buying RAM, a lot of the good values are out of stock at the moment. Probably because so many people just built systems with AMD and Intel both cutting prices within the past month. There WAS some nicely priced Corsair available last week but I can't find it anymore. Your RAM selection should work, but I think you'd be wise to get a bigger HDD and some cheaper RAM would allow that within your budget.

Actually on second though I think you picked the Corsair that was cheap and now the rebate has expired.

I've OC'd my proc on stock and its fine, very decent temps, but that could just be me getting a lucky CPU or maybe my cooling case environment is great, idk. I actually plan to OC it back up again soon...

The Evga and foxconn 590 boards have cooling that is not quite sufficient for a nice OC, they small passive hsf is just not enought to keep it cool enough under a moderate - high OC