What kind of games do you play and what detail settings and display resolution are you trying to achieve? That APU is OK for most games at low to medium settings at lower resolutions. However, if you want to play on high settings at 1080P it will struggle on many games, especially newer ones. That is mostly due to the limited graphics capability of the built-in GPU. Upgrading the graphics to a discrete graphics card will result in a more substantial improvement in overall game performance.
Another important thing to consider is your power supply. According to the HP website your PSU is 300W. Most likely it doesn't have any PCIe power connectors so you can only upgrade the graphics card to one that doesn't require additional power connectors. That would be a Radeon HD 7750 based card. Unless you plan to upgrade the PSU along with the graphics card, you can't get anything better.