In case you are wondering I plan on buying a GPU later in the summer when I have more time to game. I will also install Win7 that I can get cheap from my U.
This will be my first build, am I missing anything? If I plan on going to the stock HSF, do I need thermal paste? Is 4GB enough RAM, or should I just go ahead and get 8GB rather than waiting to do it later? Are these prices good, or should I wait on some things? I know quite little about memory or PSU, so any suggestions on those would be welcome.
First thing, you're wasting too much money on a case that doesn't offer anything special. An Antec 300 Illusion is about 50-60 dollars, and offers better cooling as it comes with 4 fans (2 front, 1 rear, 1 top).
With that extra 40-50 you could buy a better CPU, such as an Phenom II X4 955 or 965. Some people argue to get the 955 as its cheaper and you can OC it to about the same level as the 965. Personally I got the 965, but if your budget is tight you wont be disappointed with the 955 (just make you get the newest version with C3 stepping, not the older with C2 stepping)
Motherboard I would suggest the Gigabyte GA-790FXTA-UD5, especially if you plan on doing crossfire in the future, it offers crossfire x16/x16.
Just to make sure, I think this is the RAM you listed but this is the exact link. I would get this specifically for the above motherboard I mentioned as some people have had some issues with G.Skill and the AMD motherboards. But this looks to be compatible. One of the reviewers had the same motherboard and had no compatibility issue.