I recommend you buy the following two items from Newegg.ca. The total price with shipping and taking the rebate into consideration is only about $20 - $25 CAD more than that crappy HP computer.
iBUYPOWER Gamer Extreme NE788D3 Desktop PC Intel Core i5 3570k. You can overclock it for better performance. From the pictures it seems you are getting a heatsink fan combo that does allow for overclocking without having to buy a new heatsink fan.
Radeon HD 7750. This will blow away the Radeon HD 7450 that comes with that crappy HP computer. What's the performance difference? It's like a race between a turtle and Honda Civic in a 1 mile race. Well, it's not that bad, but you get the picture.