I be honest I honestly don't expect your PC to be able to play HD Video reliably. I don't believe it will have hardware acceleration for it either, and the GPU/CPU wouldn't be up for the challenge most likely is what I expect. Not to mention 512mb ram won't get you really anywhere now days.
I kind of expect this to be more of a software problem and not capable enough hardware but lets try to eliminate hardware issues.
I want you to stress test the CPU a program such as prime95 will do the trick, stress it for 8 hours to see if you get any errors. Than run furmark to stress the GPU and see if you get any artifacting (don't need much time for this), and keep track of computer temps while doing this I use a program called Speccy which has temps on it. Also want you to run both programs and let it run till the computer heats up to it's fullest and see what the heat levels are at to make sure it's withing normal limits. Than check the HD for errors.
My thought process is that it could be a HD sector error causing the boot up problems, as the CPU/GPU stress testing it was to make sure the temps and everything is functioning properly and not overheating which would cause video playback problems with CPU/GPU throtling down due to heat.
Pentium D's were known to run a bit hot so not too surprising. Just to double check did you put thermal paste on the CPU before applying the heatsink (and remove the old thermal paste)? And make sure it is seated correctly and clear out all the dust in the case and especially the heatsink.
Beyond that your next best option is an aftermarket heatsink, doesn't need to be anything expensive or fancy there are plenty of good cheap heatsinks out there especially if your running at stock speeds.