According to this the 6670 needs 45W more power than your current card. I would hope HP put enough power to cope with 45W extra but there is no guarantee. I suggest you look on the label and check how many amps it rates the 12V rail at. If you have 17A+ then you should be fine, if its 15-16A then it should work but slightly risky, 14A or less then I suggest a new PSU before the upgrade as there are not really any lower power upgrade options.
For the games you're playing (and those you'll want to play when you see what real PC gaming is like) the HD 6670 is great. You will not have to upgrade your PSU - AMD recommends a 400w PSU so that no one will ever be able to overload it using that card, and hold them liable. AMD has to assume a worst-case scenario, like using the card with a 140w CPU, 8 HDDs, 8 sticks of RAM, 6 case fans... You get the picture.
HP uses decent PSUs, and yours will be fine with an HD 6670 and your 95w CPU.