Help me diagnose Internet/network problems

I've been having a lot of problems with my Internet/Network, though it's mainly my Internet connection. I get lag spikes and dropped connections in games. My calls and conferences get dropped in Skype whether I'm hosting or not. I have no idea how to figure out if the problem is with my ISP (Comcast, 6mb) or if the problem could be with my network.computer.

As I said, I am on Comcast. My computer is on a fresh install of Windows 7 x64 (was using Vista previously and still had problems). Using my motherboard's onboard LAN, my motherboard is an ASUS P5Q Pro.

My computer connects to the Internet (wired) by going through a switch (so I can have my 360 connected too) and then connects to my router which is a Linksys WRT54GL. That then connects to my cable modem which connects me to the Internets.

I find that resetting my modem and router temporarily makes things run smoothly, but this is too inconvenient to be doing this every few days, just so I can get a working connection.

What things can I check or things I can do to improve my connection? Is it something wrong with my Internet or network?