First of all, make sure your Internet connection really reaches the speed you seem to think you have. Running a wireless connection often characteristically lowers the amount of data recieved and transmitted by your computer compared to a regular ethernet connection.
The game size matter aswell, ofcourse. Downloading huge files through any downloader on a single connection to the server takes very long time if you run your network with a ADSL setup.
You have probably already made sure that no one but allowed users use parts of your broadband, which in cases where the bandwidth is limited, lowers the speed for everyone who uses it.
I'd suggest you run a tptest with a nearby server, to define speeds and latencies. Theese can often be found at the webpage belonging to your broadband deliverer. Post the results here