Yes and no, that is NOT how GPU memory is addressed.
To your CPU there is a single 32 bit linear address range. The NT Kernal then divides that range logically into two pieces, Kernel and User address space each of 31 bits. That leaves 2GB address space for each program to use and 2 GB of address space to the kernel. The kernel then takes this space and cuts out a section for mapping ROM and external addressable memory (graphics RAM). AGP GART allows GPU memory to be mapped directly into physical space and this is what the kernel will do. Back when GPU's had 32~128MB of memory it was fine to map the entire range, but as cards hit 256MB+ it became impractical so now the kernel will use a page table to map different regions of GPU memory in a single 256 or 512MB section of memory. So having a 2GB GPU memory will not take up 2GB of addressable space, not unless you set the AGP GART to 2GB which is stupidly large (should crash your kernel).
Now notice I said 2GB per program, that is because programs use virtual address space while the MMU use's real address space. Part of a programs memory can be assigned to the paging find on the HDD, even if there is nothing being stored to that address space.
In short a 32-bit NT Kernel will break 4GB down into the following regions.
2GB User land mapped to multiple virtual address spaces
2GB Kernel Memory
Out of that 2GB multiple sections are sliced off, anywhere from 256MB to 1GB worth. Typically the last 512MB is cut out to be used for ROM / PCI card address space. Then more is cut out if a larger GART frame is needed. This isn't memory that is actually being used, just address space that needs to be used for the drivers to work properly.
Now you can get around that be using the /3GB boot option. This will tell the NT Kernel to use 3GB for user space and 1GB for kernel space, this can be VERY dangerous as some drivers expect certain ranges to be free that now might now be. If your BIOS and CPU have support for it you can enable PAE mode. PAE made allows access to memory above 4GB by using a 36 bit hardware address table to map into the 32 bit virtual address table. This means everything still access's their memory inside their respective 32-bit table but the kernel use's a page frame to map different sections of memory beyond 4GB down into a range under it. It's slower as each region must be remapped on each memory access, also programs are still limited to 2GB of virtual address space each.
Going to 64-bit by pass's all the headache of remapping memory blocks and GART tables. The 64-bit range is so large that you can directly map all your physical memory, your GPU's memory, the PCI expansion address's and still have plenty of room to spare.
Synopsis, there are ways to make 32-bit work but their complex and slow. Best choice moving forward is a 64-bit OS.