The MSI website shows ten different mobo's that are named K8N Neo .... So which is yours? And what CPU socket is used? 939? AM2 / AM2+?
I have an AMD Athlon64 X2 4400+ CPU and have realized most of these processor designs have a limit on how they manage RAM. Don't forget that these processors have their memory management system built into the CPU, not in the mobo's chipset. Apparently because of loading of the memory addressing lines, if the CPU has to deal with FOUR DRAM modules with "double-sided" RAM (no, this does not necessarily mean chips on both sides of the RAM board), it will automatically slow down the RAM bus speed from DDR400 to DDR333. So, if you were to leave your 2 x 512 MB modules in place and fill the remaining two with more 512 MB units (total 2 GB) or with 2 x 1GB units (total 3 GB), you'd have more RAM space to use without forcing slow use of the Swap File on disk, but your overall RAM speed would decrease. I'm facing that as I look at going from 2 GB to 4 GB. On the other hand, in your case you could take out the 2 x 512MB units and replace them with a 2 x 1GB pair of DDR400 (aka PC3200) modules to get a significant RAM size increase (and hence some speed-up) without forcing the RAM to run slower.
Without knowing which exact mobo you have I can't confirm Ovrclkr's statement that you can also use DDR2 RAM. And I don't know whether your particular CPU and mobo are among those that slow down RAM when all 4 slots are populated, or whether DDR2 RAM (if you can use it) is subject to the same limit.
EDIT: Now that you say it's a "3800X2 CPU in Socket 939", and you cannot use DDR2, I pretty convinced you are subject to the limit I outlined - 4 full RAM slots will cause all of your RAM to run a bit slower.
Here's something to consider. Since you have a video card with dedicated RAM, let's assume that the addressing space for that will be taken from the top addressing area over 3GB so that, if you have less than 4 GB of RAM, none of the real RAM is "blocked off" by video address use. However, Windows still will reserve for its own use about 0.3 GB of real RAM. So with the 1 GB you have now, there's actually about 0.7 GB usable by applications. If you went to 2 GB in 2 x 1GB modules as a replacement for current RAM, the usable application space would increase to about 1.7 GB - about 2½ times what you have now. That is a significant increase, but you still would be running at full DDR400 speed in the RAM bus. Moreover, many apps only use up to 1 GB themselves, so you only need over 1GB if you have more than one big app running. But you're under 1 GB available now, so you may be seeing slowdowns while the Swap File is used, and you could beat that with 2GB RAM.