If you can't fit a 2nd Ram stick because of your heatsink, I'd buy a different heatsink. Cooler Master Hyper 212 Evo will give you full ram clearance, so would the Xigmatek Gaia. Both are under 30 dollars.
The Radeon HD 7770 is the best you are going to get in that price range. For a bit more, which you will save without the heat sink, you can get a GTX 650 TI BOOST which will be MUCH better. It has 2GB of GDDR5 RAM and is 192bit instead of 128bit