Vista was just frustrating for other reasons. I think it's nicer looking that XP, but I'm just too used to XP and I'm much more comfortable with it. I'm on XP now and I'll report how it performs a bit later.
I understand why you are frustrated. When I first started using Vista there were a few things that urked me too. But now I have had it for 6 months and have yet to get a virus, spyware or even adware or anything for that matter which is nice. Once you get used to Vista it has its perks. I just love having WMC in it for movies.
I'm thinking though even in XP your score wont be much different. Maybe a 100-200 ppoint boost but that CPu is probably just holding you back big time.