What sort of connection do you have? DSL or Cable? If it is cable, it could be that there are simply more users on the shared connection, i.e. at night. If it is DSL, I agree with Emperus that it could be general traffic related, i.e. more traffic on your favorite sites and such slowing down the servers.