Isn't there a sticky on this yet?
Your missing memory IS being shunted aside so that the processor can get to the 512Mb on the video card. Complaints should be made to the OS developer that stacked hardware vectors on top of system ram instead of hardware vectors first....then system ram.
FYI - it's not wasted, it's being used to hold up the top edge of your ram chip.
/pae /3GB switches allow windows servers to utilize the memory above 3GB. It doesn't change issue mentioned, but who runs a 512mb video card in the server anyway?