More memory makes everything load faster. You can keep more things running at full speed at the same time. Switching between applications is faster. Multi-tasking in general is smoother. When gaming, loading the game itself and level loading are both faster. There are less graphical anomalies like missing textures and such.
Corsair blog 4GB versus 8GB Tom's article on the subject (The words in the pics are in German I think, but the gist is that the graphics card uses system memory to store textures. The more system memory you have, the better -- it will have more texture storage space and the rest of the system will have more memory to run at the same time.)
My recommended minimum is 6GB for triple-channel systems and 8GB for dual-channel systems. I will also recommend 8GB for quad-channel systems when they become available later this year.
A system with 8GB of system RAM will load games and game levels faster, as well as having less graphical anomalies like missing floors or walls. Games will also use higher-resolution textures if you have more system RAM. Some games will also allow you to choose higher quality settings because you have more system RAM.
Not only do you have Windows and the game using system memory, the graphics card uses system memory too. The more system memory you have, the more room it has to store textures and the better your games will look. Getting the textures from system memory is far faster than getting them from the hard drive, so games will perform better as well as look better.
Getting 6GB (for triple-channel systems) or 8GB (for dual-channel systems) of system RAM is a no-brainer, especially right now when prices are so low.
Edit: Of course, this does assume a 64-bit OS ... but you shouldn't be buying a 32-bit OS nowadays anyway.