Fastest read/write for gigabits data transfer

The biggest problem is not really the harddisks, you only need a sustained Read/Write of 125MB/s to fill up a Gbit link... You can get that on a high quality single hard drive. Solidstate is definitely not required.

Most import will be your frame size for the transfer, you'll need to use some type of jumbo frames or mulitple streams to reach a Gbit. With 1500 byte MTU you'll usually only see about 400Mbits on a Gbit link

Remember drive transfer rates are usually in GigaBYTES not Gigabits, 8 bits per byte so 1024Mbits=128MBytes

hi tokencode.

can you give example where i can get 1 gigabits write hard disk? says here (http://www.seagate.com/www/en-us/products/internal-stor...) sustained data transfer rate 138MBps. since it doesnt state whether its read or write, i assume its sequential read. i wont be suprise to find the sequential write to be much-much lower. not to mention random write.

now i can get by with this with raid 0 or other striping raid setup. but the seek time is unacceptable. that is why i go with ssd. and from what i read, the best ssd are from intel and ocz. i choose intel.

im not knowledgeable about hte jumbo frames you talked about. from what i understand, i can just a gigabit ethernet network card and i can get gigaspeed. im thingking of fibre 10gb just to make sure the network card wont be a bottleneck. how does jumbo frames is a concern? please enlightned.

thanks.

Just to reiterate what Tokencode wrote, 138 MB/s is 139 megaBYTES per second. A byte is eight bits. 138 x 8 = 1,104, so you are in the gigaBIT per second range.

Actually, most network encoding is 8/10, so you can only transmit one byte per 10 bits, not eight. This is done largely to make sure that the signal is "balanced," not developing a DC current across the wires. See http://en.wikipedia.org/wiki/8/10_encoding

Most of the HDD/Storage manufacture list the transfer rated sequential.
This wont help you much with a real life application
Here is what you can do to get the fast transfer rate from a server:
1- Hardware raid
2- Network teaming - prefer 802.3ad spec.

even with jumbo frames you will only get ~600 mbit /sec

possibly with nic teaming (LACP or 802.3ad) and jumbo frames you can exceed 1 gbit/sec sustained

The PC clients who are reading the data, will their systems even be able to write at 1gbit/sec?

natx, I've maxed out a Gbit link before doing file transfers, no need for NIC teaming. The only reason you can't get about 400Mb without jumbo frames is the latency in the ack/syn of TCP. Send the data via UDP instead and do your own data validation and you can max out a link regardless of the latency. Yes there is a little overhead so you'll never see the FULL 1024 Mbits but you'll be way in excess of 600.

PublicENEMY, jumbo frames are TCP packets with an MTU larger than 1500bytes (standard ethernet MTU) This is the size of the packet being sent on the ethernet network. If you use TCP to transfer your data, for each packet sent you need to get an acknowledgement back before the next packet is sent. If you have any amount of latency, he number of ack/syns per second is limited and since the packet size is also limited your max htroughput is limited. With Jumbo Frams you can change the MTU to let's say 3000 bytes which would effectively double your throughput. This will only work if everything in between the two end-points supports jumbo frames otherwise the packet will be fragmented and the reassembled and can further degrade performance.

Even with NIC teaming, your throughput will be identical if because your latency is reduced. Now keep in mind all of this only applied to a single TCP steam. If you do multiple streams, each stream can support maximum.


Since you stated that drive access time is important I'm assuming this is for some type of database application or something similiar where the read/writes are random and not sequential like when streaming a movie or something?

Actually, im totally aware that 138megabytes are in the range of megabits. The issue is, that occurred under lab test, sequential and most probably read only. in real world, that is hardly the case. i wont go with something that is just enough in the lab test. too risky.

by the way, thanks for reminding me of 10 encoding. that makes my target speed is 80% of 1 gigabit at max(im sure there are other factors to consider)



exactly.



hardware raid solution is too expensive. plus ssd doesnt support raid with trim. network teaming is out of the question since im measuring single network line.



both client and server have the same spec. the thing im doing is actualy for a telco company. they want to prove to their customer that they are giving 1gigs speed to their customer. not best effort, but guarantee.



i did try using tcp and udp but finally settle down with ftp. about jumbo frames, i chose intel fibre channel nic http://ark.intel.com/products/50396/Intel-Gigabit-EF-Du.... there is no mention of jumbo frames. is this the best nic for the task? are there any other brand better than intel?

this is not for database application. this is for proving that the line can truly provide gigabit speed. the read and write is sequential.

thanks to everybody (tokencode, natx808, FireWire2, WyomingKnott)who answer.

FTP is TCP based protocol, if you want to utilize UDP you will need a special utility, there are some free UDP transfer utilities out there but not a lot. We had to actually write our own but I can max out a GigE connection from east coast to west coast with 60ms of latency using it. The Jumbo Frames is really a windows setting. Are the two machines on the same ethernet segment? If so you can try enabling jumoframes. The settings should be under your advaned options on the adapter itself.

http://www.maximumpc.com/article/howtos/how_enable_jumb...

Here's a quick article with screen shots.

i did try some udp tools. none of them are reliable. the best ones on linux and i required windows. i did try create my own upd tools, but im having problems with timing(there is no guarantee timing in windows 7). my final resort is long duration ftp. the two machines are exactly the same. is it possible for you to share the program that you wrote?

thanks.