One issue is that available memory will decrease. The 4 gig limitation for 32 bit is for all memory that is mapped. If you look at your current available memory you are probably around 3.2 -> 3.5 Gigs depending on the amount of memory your current video card has. Best guess is that it will drop to around 2.8 Gigs. Video VRAM does not subtract on a 1 to 1 bases.
Bottom Line is that it will decrease multitasking performance, that is you may start to see swapping of in/out of virual memory. Virtual ram is at the mercy of your HDD performance. Specifically regaurding runing a game, Hard to say, but as long as you are not swapping real memory for vircual memory - very little effect. If having to use Virual memory then you will take a hit. For 32 bit programs, including games memory requirement is limited to 2 Gigs - and you probably only have 2.8 gigs avail which is for all running programs + windows OS requirement.
Can look at memory usage by Contral alt del and select task manager, then performance tab