I know I have an AMD Radeon HD 6530 that came with my HP computer, but I want to upgrade to a better graphics card as my pc cannot really handle diablo 3 on max graphics. I'm willing to spend up to $300 on a new card if its worth it. I realize that this computer isn't exactly top of the line and a $300 card could be a waste of money with it. I'm not sure of the power supply or motherboard compatibility issues so could someone please help me figure this out?
You will almost certainly require a new PSU. What physical size PSU does your HP have, Standard ATX, SFX, or TFX? Seasonic and FSP have some nice, capable PSUs in those smaller sizes. Even if you're limited to a 350W TFX, you should be able to run a HD7770. If you have SFX, you can get a 450W unit that will let you run a HD7850.
Ok, that case will take a standard ATX PSU. I don't think I'd get anything stronger than a HD7850 or GTX650Ti as your CPU will bottleneck a stronger card.
Around $70 will get you a solid PSU, either Seasonic-built (520W Seasonic itself) or Super Flower-built (650W Rosewill Hive). The remaining $230 is more than enough for either of those cards; even a HD7870 should fit, if you don't care that it will be held back some by the CPU.