The 32-bit version of Windows is limited to a maximum of 4GB of memory, and because of the way the video hardware works the memory on your video card actually replaces some of that so that you won't even be able to use all of it.
The 64-bit version of Windows has no such memory limitation - it can handle as much memory as your system can take, and lots more beyond that. And in 64-bit mode the video memory doesn't replace some of your RAM, so you'll be able to use all 4GB of it.
So yes - definitely go with the 64-bit version of Windows. You'll be able to use all of your 4GB of RAM, and you'll be able to easily add more memory later if you need to.