I would guess that your fan less card is getting hot during HD playback although the HW monitor is reporting the GPU fine it could be the Vram. You can test this with some makeshift cooling (Fan) blowing on the card.
It looks like your system is a tad underpowered to me. That is a high resolution to obtain with just a Pentium with 1 MB Cache and an 800 MHZ FSB combined with a budget GPU of that vintage. I am picking it is time to make that investment.
If there is not enough airflow around the card it just builds up heat, the higher the load the hotter it gets. Your system should be plenty to play HD videos and as you say the jitter starts few min into the playback which points to heat issues.