Well you could, but you dont really want to.
That is the amount of memory that "COULD" be used for graphic card information.
Whether it be textures for games, 3D dimentional maps , Wireframe models, and the cpu calculated data that the video card needs .
The memory is still there and can be reallocated as needed. But in graphic intensive mode this area of memory is mapped for cpu video card use.
It is much faster to have a large preset size , than to constantly re-allocate swap file size.
With the large buffer size you get large sequitential memory reads. Not fragmented reads. Fragmented reads are much slower.