A 20Mbps difference for a connection protocol rated at 5+Gbps is nothing to worry about. That's a 0.4% difference at best.
Even if you meant 20MiB/s (~160Mbps), compared to the 625+MiB/s that's still not a very big drop (3.2%).
Most likely, the drop is because of limitations on the transfer rate from the HDD. In fact, the difference between SATA III & USB 3.0 is actually ~25MiB/s (600MiB/s for SATA III, 625MiB/s or faster for USB 3.0). So it's almost certain to be due to the HDD's limitations.