Then by all means, I would say go for it. You won't really see any speed improvements on your games, but at least you'll have better stability, memory management and have access to all 4 gigs of your memory.
My machine isn't a monster like yours, but I still run 64-bit and it's been pretty stable. No BSODs for me except when I decide to try the occasional beta driver. [:isamuelson:3]