Yes, PSU is power supply, and just cause you bought new doesn't mean anything, there could still be something wrong with it. Case in point, I just bought 2x4GB DDR3-1333 SODIMMs for my wife's laptop and am currently running memtest+ on each stick individually, cause when I have one stick in (4GB) everything is fine, but the other stick in I keep getting BSODs, so each is getting tested individually and I suspect that I will be RMAing one or both sticks by the time it is through (HP laptop, chipset and HP site allow for 8GB RAM).
Near as I can tell, A) The PSU has a problem with the 4pin ATX power (the 4 pin provides power to the CPU socket, so unless you took the CPU out lately and bent pins, I'm guessing we can rule that out), OR B) The plug on the motherboard got spiked or some other anamoly happened and now it isn't accepting power FROM the PSU.
That's why I am thinking take the PSU back, get another and try it, it is the easiest and fastest way to diagnose. Not all PSUs are of the best quality, thus I was wondering what one you bought. If it was an Antec or Corsair, something higher end, I would be more prone to lean toward mobo problem. If it's some Kingwin or lower quality, it could be the PSU.
Let me know what brand and how many watts it is rated for and I'll see what I know about it. I would still consider taking it back, but after I find out what one it is, I can direct you toward a better or same PSU to excahange for.
As for what motherboard, let's cross that bridge when we get there. HP uses proprietary gear on a lot of their builds and depending on what motherboard it is, you might be better served to get a whole new computer, as the 'custom' motherboard HP will try to sell you will be more expensive than a whole new system!