With 8GB Vista 64 has just been rock solid, IME. A few months ago THG put out an article on 8GB of RAM for Vista, disabling the page file, and so on. I turned off my page file and haven't looked back since. For me, Vista64, 8GB, no-page-file just plain flies. I fire up VMWare WKS with 2-3 clients running simultaneously and they work like I'm used to machines working on physical boxes. There's no stops, no hesitations, no questions, its just fast. In fact, just for nostalgia sake I installed, cough, NT4.0 WKS, in a virtual environment and was blown away by the performance...while a Vista32 and XP32bit client were simultaneously running on the Vista64 host.
Vista 64 is finally really getting there. I've had no driver issues to speak of in the past 9 months. Yes, XP64 may be faster, ...and I'd imagine that Win2K would be even faster yet, DOS would be faster still.
With the price of DDR2 as low as it is I'd say spend the ~$160 (if you can) and go for 8GB and enjoy Vista 64's features, interface, and stability.