Its a counting issue; Windows needs to put all data in a unique address so it can be accessed by the CPU, so it treats GPU RAM just like System RAM (just in a different location) as far as Addressing goes.
Its for that reason why on a 32-bit system, GPU RAM starts to eat into the maximum avaliable of System RAM, as 32-bit OS' can only handle 4GB of unique addresses, and GPU RAM takes up a part of that unique space. So in theory, if you had 4GB installed and a 512MB GPU, you should have 3.5GB of RAM avaliable for system use. This is rarely the case though.
The general formula I use is as follows:
Installed System RAM - 512MB (motherboard/Kernal level stuff) - GPU RAM - Other Hardware = Avaliable RAM
In your case: 2GB - 512MB - 512MB = ~1GB (you had 800MB avaliable for this case)
and: 3GB - 512MB - 512MB = ~2GB (you have 1777MB avaliable, so it looks like 640MB of RAM is being reserved instead of the normal 512MB.)
Not that I know except for increasing the vram through software but that was only experimental at best and never was implemented outside integrated graphics which shares from system ram. As for texture its defined by the drivers but most don't know the code and programing language that most gpus run.
The only real way to "increase" VRAM is to use HD space and treat is as RAM (like normal RAM does; how else do you get multiple 2GB+ applications running at the same time?), but that can cause a lot of performance issues with GPU's. Also, almost all Windows drivers are written in Assembly, in order to get the performance needed to communicate with hardware in real-time.