I am running Windows 7, 64bit with 12 GB of RAM. My memory usage is usually around 2-3 GB. About once a day, Windows Defender eats all my memory. For a few minutes the computer is slow (even if running i7-980) then Defender releases the memory and everything is back to normal. I know it's Windows Defender because as soon as I see memory going high, if I stop the Windows Defender service, the memory goes right back to normal.
Is there anything I can do except disabling the service?
Just ran a few tests on a system much like yours (16GB ram, win 7 64 bit). And windows defender doesn't use much RAM at any time on my setup.
If you have antivirus software running, and then use defender they may conflict with each other (even MSE). My recommendation would be to disable it and run it on demand if you still want to use it from time to time.
On a side note if you want to investigate which program or DLL is causing the overload use ProcessExplorer from sysinternals. Its free and a great tool