Hi, I am having some problems with data transfer.

CapnTrav

Honorable
Oct 9, 2013
1
0
10,510
Hi, I am having some problems with data transfer. I have 4 tb drives that come in daily and i need to put the data on the server, using windows cmd and robocopy with /mt;16 for hyper-threading. It takes about 19hrs on both esata and usb3.0 docks. Is there any way to zip or compress the data to speed up the transfer?
I know this is vague and i will try to provide as much info as possible to solve this.
 

popatim

Titan
Moderator
That's rather slow considering you have a max of 4tb of data, down around 50MB/s. is this sequential data? ie - a few big files or a lot of small files?
Are these 7200 rpm drives?
How many are you reading at one time?
What are you copying them to? ie - what size and speed drive(s) or array?

but to answer your compression question: yes but compressing the file on their end and uncompressing the file on your end would consume time also. How much time it would take and if you would save any time overall would depend on how compressible your data is in the first place and how high a compression you try to achieve.

once you copy the data from a drive in and back it up, see how long winzip or 7zip takes to compress it back onto the external drive. Writing would be a little slower than reading.