Your limitations with YouTube is the internet download speed. If you cannot download a 1080p as fast as it needs to be for real time viewing, or if YouTube has too much traffic to upload the feed to you, you will get lag.
Flash can be very inefficient, and that might be causing jerky playback. Update to the latest version if you haven't already.
Go into your browser options and see if you can turn on GPU (might be called hardware) acceleration, that may also help your youtube video playback. Problem is, I think GPU acceleration in most web browsers is restricted to DirectX 11 cards, so you may not be able to do that with your 9800GT, but it's worth a try.
Have you updated flash or your video drivers since then? It could be one of those has broken something related to 1080p youtube playback. You may want to rollback to an earlier version if that is the case.
it could be your flash player. in the past nvidia have lag issues when playing you tube videos in full screen but it has been fix since 285.xx WHQL driver. just now i tried playing some you tube videos (in full screen as well) there is no lag. currently using the latest beta driver.
I have a feeling that YouTube has been playing about with their codec a little bit, possibly trying to get higher bit rates as a lot of uploaders use fairly high end equipment nowadays capable of in excess of 90Mbps
...I still think your hardware is capable of it as it can play bluray and thats around 30-40 Mbps compared to Youtube at around 8 Mbps...
I would play around a little bit more, experiment with settings and different browsers and roll back flash to see if that helps
It's probably the flash player... Youtube dislikes Google Chrome. Also, wi-fi connection could be a problem too... I've noticed that when my laptop is directly connected to the internet, it runs youtube better. IMO, youtube is jacked up... and they keep jacking it up worse and worse... its getting to the point to where its almost a pain in the donkey to watch or even use your youtube account. Flash Player always crashes for me, no matter what PC I am on, even my friend's i7 2600k GTX 580 Rig.