There are two issues:
1) A 32-bit CPU reserves some of the physical addresses for I/O devices. For example a graphics card with lots of memory can use up a lot of addresses that would otherwise be available to access system RAM.
2) The Windows architecture reserves the top 2GB of virtual memory space for the operating system. (Note that virtual memory space applies to the addresses that can be used by the program, not to physical memory). This means that 32-bit programs themselves can't be bigger than 2GB.
There is an exception to (2) - if the program declares itself to be "LARGE_ADDRESS_AWARE" then the virtual memory space is reallocated as 3GB for the program and 1GB for the OS. Such programs can use up to 3GB of virtual memory (and they can actually use all 4GB of the virtual 32-bit address space if they're run on a 64-bit operating system).