New build, weird issue

I'm having a very weird issue. I recently built a new system, with an X2 5600. Everything seems to be running great, CPU intensive programs are running normally. But for some reason, my download speeds are being extremely erratic. If I have a browser opened in full screen, I'm getting horrible download speeds, but when I minimize my download speeds jump up to normal. Then if I open up other programs, my download speeds stay stable when it's in full screen but then die down when I minimize. Is it possibly that my CPU is impacting my download speeds? It seems like the CPU is working a little too hard (about 80% use) and the temperature is around 50C. This seems a little high to me. Does anyone have suggestions as to what I may have done wrong? I used the network card on another PC and it's fine, and my laptop is running fine on the same network, so I know that's not the issue. Any help would be appreciated!