Thethermaltake psu is junk so send it back if you can . Only use 80+ bronze or better rated psu with a feature called Active PFC . Without this they wont be 80+ efficient
You will get decent gaming from your mb /ram and hard drive once you install a better graphics card
Getting a decent case and psu makes sense now so that you can re-use them with different hardware when you want to do further upgrades .
+ 1 to psu return + I'd also return the case - everything thermlatake made since the Level 10(original) went downhill. If you're buying from newegg - look out for the fractal cases.
You can get feel for what to look for in a PSU in the link in my sig.
HP seem to have some proprietary parts that if messed about with - will end in doom for your entire rig. open case, take pics of innards and post back. But i vote on building a new rig by yourself with some advice from Tom's hardware members
If you look up newegg.ca - you'll find out how much it'll cost.