Weird problem with upload speed

I'm going to apologize for the vagueness of this post but I am going to do the best that I can. Some background, I have comcast's Blast 50 internet, a DOCSIS 3.0 modem ,some comp specs are :

ASUS P8Z77-V PRO
8 gigs of RAM
650w seasonic psu
new i5 ivy bridge

Now the problem that I am having is what I hinted at, my upload speed. When I speedtest.net it I get ~10 ping, 35 Mbps download, then upload hangs for 10-30 seconds then registers at .1-.3. I have tried to hook my computer directly to my modem and resetting my router to factory settings. I hook my wife's laptop to the router (cat5) and the speeds post 10, 35, and 6.

One thing I did try was connect my computer to the wireless that came on the mobo. Oddly enough, this fixed my upload problem by giving me 10ping, 18 Mbps (down from 35) and the upload quickly jumps to 6 Mbps.

Basically I am wondering whats going on and how I can make my upload speeds work on my cat5. Thanks for reading and your time.