If you want to overclock you need an aftermarket heatsink, as the stock wont take you all that far. The most I would be comfortable with on the stock would be 3.7Ghz, and even that's dependent on how hot it gets in your particular conditions.
CM Hyper 212 EVO. $35 http://www.newegg.com/Product/Product.aspx?Item=N82E168...
I assume you have HDD, OS and CD drive? Because if you don't you will have to include them and it could kill your budget.
i would swap out the 4g for two 4 gig ram kits that at 50.00 price point like gskills arires kits. (more ram the better your games going to run..) on the mb some people like the asus efi gui over the older non gui mb for over clocking.