Router slows down after uploading, restart fixes, help??

I run a Linksys BEFW11S4 811.2 B-network with 1 wired and 3 wireless PC's. The router seems like it needs restarts every few days because it gets bogged down and speeds slow down to almost dial-up like. It for sure will happen any time I run a torrent program that downloads and uploads files at the same time. The torrent program functions fine when its first run and I can leave it going, but soon as I close it and use regular internet its way slowed down. Anyone have any ideas what could be the culprit here? A reboot fixes it back to normal, just a pain restarting it all the time.