Work together?

Yes, your motherboard has a 2.0 PCIe slot based on the board's chipset, even though the HP specs dont say what generation. The graphics card will work in it. The power supply should work too, but I would recommend a better quality one. You're on newegg, search for a PSU with a higher rating.

You have a PCI-E slot, you have an ATX form factor case with an ATX PSU support. You're good to go, but I have to agree that you can get a better PSU than the one you showed me earlier.