Well I would say the PSU may still be the issue, though the 640 demands more power, this is a 5 year old system (the i3 2120 was released in 2011 for use) and 5 years of 'wear and tear' will kill off parts (which is normal, think of Tires on a car or bike, after so many years they 'wear down'). The PSUs are always the 'cheapo' parts, the problem is that HP, Dell, and others sometimes makes (DEPENDING on specific model PC) customized size cases that can't fit 'off-the-shelf' replacement PSUs, as well as customized connections (as in the case of the infamous BTX connectors).
So first test would be to try with a different PSU, even if you didn't 'install it' just sat it next to the PC, swapped the Power cables to the Mobo, and tried to power it up, do you get the same results. If not, then the issue was the PSU and now you need to remove the old one and put in this new one.
The second thing I would check is a BIOS update. Note the i3 2120 was released in 2011, but 750Tis were not produced and sold till 2014, so the BIOS of the HP may not 'know' what a 750TI or other newer card 'is' and can't "talk" to it when it starts up. It may just need a simple BIOS update.
The last thing is check the 750TI is functioning by trying it in another PC. That it does 'boot' and show 'text' is all great, but when it tries to switch to 'graphics' (the HP/Windows logos, etc.) it fails, which can be the GPU itself is damaged, I seen that rarely but that is the last thing to check.
My final note: you state your a console person, and you have a budget of 105: I am sorry your going to have to reevaluate things. See PC side is EXPENSIVE, and if you don't have at least $700 for a 'gaming rig' and the extra cash incase (as your experiancing) things go 'wrong' for other parts or to fix your mistakes (maybe you jammed the card in wrong and 'broke' a connector, well now you need to buy a NEW card they won't give you a replacement free for example) .
If your not able to HIGH budget for PCs, then stick to consoles, because they are heck cheaper, and straight forward (as you can see how complicated this is getting) to use, games ALWAYS work as advertised. I would suggest a PS4, as the best option IMO for cost, performance, and gameplay (you can get refurb one for as little as $249 at Gamestop I seen before, far less than the video card just for a PC).