The short answer is that you would hardly notice the difference in performance. When more apps are programmed for 64 this should change giving 64 an edge in performance. Benchmarks between 32 bit Vista and 64 bit Vista are very close, sometimes 64 gets the edge and sometimes 32. The latest gaming benchmarks I have seen have actually been giving the slight lead to 64, FWIW.
Very few apps are programmed to run in 64 bit mode at this time so they simply run in the 32 bit mode contained within 64 bit CPUs and systems. This insures that almost any 32 bit app will run on 64 bit much as it does on 32 bit.
Some apps and most hardware drivers require 64 bit versions to run on a 64 bit OS but fortunately these are usually easy to get. My anti-virus (nod) and my cd emulator (daemon tools) both require you install their 64 bit versions.
If you have any older apps or hardware you simply must run then you should research each one to see if they will work on 64. Otherwise I'd say go 64. It's the future and it will also allow you to run 4 gig of ram and actually use it all. Ram is cheap you might as well get 4 or more, I run 8 myself. Vista 64 is hot with enthusiasts and gamers these days. It's turned the corner and is taking the lead with leading edge users. Cheap RAM and the superior implementation of 64 bit on Vista is making this difference.
If you will be staying with only 2 gig ram then it's kind of a toss up. 32 gets you maximum compatibility but locks you in to an install you cannot later upgrade to 64.