SMB Concurent File transfers, thread speed barrier

[larek]

I've hit a wall on file transfers between 2 Win S03 SP2 test machines via copy command from a SMB (windows file sharing) share. I can't seem to get faster the 113MB/s

The PC's are directly connected with a 10' factory made Cat6 cable, and all the Disk Volumes can push much more than that.

We've gone through IBM redbook and tried all the optimizations, although most were already active with SP2. The optimizations at the front of the book (up to the reghacks) got us a 50% performance gain.

But
Jumbo frames and all the reghack's don't show any performance gain, and most show a tiny loss of performance. The same is true with hardware offloading. And the Machines don't seem to be taxed or stresses at all, with little disk activity.

I'm posing this question because Most people see lots of gain from >1500 MTU (Jumbo Frames) and HW offloading. We just aren’t.

Note: yes we know SMB2 can be faster, but were not going to Vista/08/7 in the near future


Here are the PC specs, both are running windows server 2003 Enterprise, both have Intel's new AT2 as their primary nic
[fixed]Test Server
GA-x85a-ud9 Main Board
Triple channel DDR3 2200 (6GB) Ram
I7-980X Proc

Test Client
GA-x852-UD3 rev2 Main Board
Triple channel DDR3 1600 (6GB) Ram
17-930 Proc[/fixed]

And after an entire weekend of fruitless tweaking we found a Workaround. By using multiple concurrent copy commands, 2 file copied at the same time only takes about ~1/6 longer but twice the data was copied, giving us 182.5MB/s

Three works too, at only ~1/11 longer than two, giving us 253.4MB/s

It was three am and we had to stop, but we're expecting 4 will likewise work.

So we can break 113MB/s with our network but not with a single thread (Copy command).

Has anyone else seen this behavior? or have Suggestions on what is possibly limiting a single thread?

 

[riser]

Read/write speeds on your hard drives could be involved.

If you have a RAID setup, multiple reads and mutliple writes (more threads). One copy, one thread, can only read so much.

You've done a lot of testing and said the drives can handle more. Have you looked at the builtin performance counter to verify your drives read/write ability is not maxed?

 

[larek]

Were using three different types of disks for the testing.
■3, Acard ANS-9010 in dual-port mode, on a Adaptec 5085 raid card
■Intel X25-M Mainstream SSDSA2M080G201
■OCZ Agility 2 OCZSSD2-2AGTE240G
■2.5G rramdisk

Local file copys of the same test files, put these disks at
■447.2MB/s Read, 136.1MB/s Write
■166.0MB/s Read, 85.5MB/s Write
■142.7MB/s Read, 84.9MB/s Write
■1685.6MB/s Read, N/A -VeryFast


But locked at the speeds were getting, all the disks can push (read) that fast, and the ones that can write that fast get the same results.


The disk speeds are dissapointing. HD-tune Pro rates my Acrad array at 800MB/s Read 450MBS/s Write but we only get the above rates on our test copy.

 

[larek]

You've done a lot of testing and said the drives can handle more. Have you looked at the builtin performance counter to verify your drives read/write ability is not maxed?

I'm assuming you mean perfmon. while pages seem high 29k pages per second. Doing local copies at the same time throw the count up to 49k per second and performance doesn't suffer.

 

[riser]

What are are the read/write times in the monitor?