Correct! it sounds like there's too much pull on the power supply when attempting to POST...one component is unable to meet minimum voltage limits on boot...so it reboots until the voltage achieves the minimum threshold for a successful boot.
HP isn't noted for overly robust power supplies in their proprietary machines so you are....with onboard video...most likely looking at a 250-300W psu and it's just not enough!
Check your psu model for wattage and see if it's at or below 300W...if so..I would seriously suspect power as the issue.
Hope that helps a bit...you can get your psu wattage and go to here.http://extreme.outervision.com/psucalculatorlite.jsp
punch in your numbers and see what it comes back with to better guage what power you need available.
Let us know what you find please...we are a bit nosy here:)