Hard Drive Speed - Units of measure vs units of measure for nic card

Richard_41

Reputable
Nov 8, 2015
15
0
4,510
I am doing a crashplan local backup(both boxes have 1 gig cards and connected by a 1 gig swtich). I believe my crashplan is the bottleneck . But every measurement I take is in a different unit of measure and is a bit confusing.

Here is my issue. I am using Crashplan and the box to box copy speed as listed in crashplan is 85 Mbps. based on how long it should and the usage percentage on the nics this seems really slow.. The nics show about 10% usage so the Crashplan speed quote seems to match up with what is going over the network.

When i do a straight copy I was hitting 900+ mbps which is in the range of 10 times what crashplan is pushing. . I assume me approaching the 1000 mbps is me maxing the nic? Is this math correct.

A SATA harddrives 3 GB/s is theoretically triple the max of a 1 gig nic? Is that the right units of measure?

I did test my harddrives with HDTune and it was in a differnt unit of measure. My SSDs were a bit fasater by my regular drives were hitting the 120-150 MB/s speed. Is that the equivalent of the harddrive being able to do a bit more then maxing the 1 gig nic or if my unit of measure totally off?

What I appear to be seeing is that Crashplan is copying at ~1/10th of what I could get with a direct copy and crashplan not the hard drive or network are the bottle neck. Is that reasonable?


 
Mbps stands for Mega BITS per second. GB/s stands for Giga BYTES per second. At 8 bits per byte 3 GB/s would be 24,000 Mbps.

There is also overhead because everything has to be packetized to send it over the network and I suspect there is some overhead for validation that what was received was actually what was sent. The receiving software has to unpacketize the data, write it to a storage device, and verify that it was written correctly. If all of that goes correctly it will send back a packet that says it's OK to send the next packet to be copied.

It sounds like the software builds a packet, sends it and waits for a response that the packet was received and written properly. It then sends another packet. If what was received was different than what was sent or if it was not written correctly to the storage device multiple cycles might occur per packet either in the sending/receiving stage or in the write/verify stage or both.

And networks have latency which adds to transmission delays both way - the data and the acknowledgment to send another packet.

The stated 85Mbps would be 85/8 bits per second or 10.625 MB/s. If that were 85Mb/s which are bot lower than the 10% you stated. So there is probably also some data compression and decompression going on.

You would have to know the exact algorithms used but I would say a 10% throughput is fairly good.
 

Richard_41

Reputable
Nov 8, 2015
15
0
4,510
Overall, i think my crashplan issue is CPU. I am pegging one of the cores of the CPU. Crashplan has a de-dupping algo that only sends the changed part of a file. In my case this was the first backup but I assume it creates the same hashes so that they can be compared against later.

They are also likely compressing the data with the 85Mbps likely being the rate reflecting the original file sizes not the amount of data sent over network. So crashplan using 10% of the network bandwidth of me doing a straight copy is not likely indicative that it will take 10 times as long using crashplan.

What will be interesting is that I am doing a full backup so I can replace a single drive with a raid. The restore should be faster as the cpu to uncompress will hopefully be less then cpu usage on compression side

Crashplan is definitely designed to reduce bandwidth over slow/internet connections at the cost of end users CPU