use open hardware monitor, it will tell you exactly how much memory is being used.
I had the same issue with BF4 Beta when using crossfire, it said not enough memory, yet only 6GB/8GB was used. It turned out to be because I disabled the page file (virtual memory) and BF4 didn't like that when I used crossfire. (of course that was just the beginning of the issues, Dice create new issues with each patch).