Your system specs are the same as mine and super-overkill for SOURCE :P so i think we can certainly say the problems lies within your connection rather than the graphics card showing a slow frame rate.
Make sure you dont have any downloads running at the time, make sure theres no torrents downloading or uploading (this can be a real bandwidth killer).
Secondly, 54mbps sounds like your using wireless? If thats the case does your home network have a password? It could be that someone else is using your internet connection. Is there enough connection bars (strong connection)? If your too far away from your router this could hinder performance.
Finally, have you tried different servers? Do you get the same high ping on every server you try?