USB bandwidth question
Last response: in Storage
Howdy,
I have an oddball question -
I have 6 blu-ray duplication towers that connect to a PC via USB 2.0. These towers have standard 500 gig HDDs in them, which are formatted in EXT2 format (the towers use embedded software to allow for ISOs from the PC to go to the HDD so that you can duplicate the ISOs directly from the drive). I have all 6 connected to a PC with a super fast SSD RAID (4x120gig, RAID0, on adaptec 4 port card; all modern stuff). The PC has 12 USB ports, 6 of them are USB 3.0, all the towers are plugged in to USB 3.0 ports.
So my problem is that I'm trying to get rid of the bottlenecks in transferring a single file from the PC to all 6 of the towers at once. Currently what happens is that the transfer speed drops down to 10 MB/sec when all 6 are copying at the same time. I know I won't get 60 MB/sec (USB 2.0 max speed). I thought I could get max bandwidth out of USB because the SSD RAID can easily provide 1500 MB /sec, and I know the max speed of my towers is 60 MB/sec. 6 x 60MB = 360 MB/sec - easily under the limits of the RAID.
I thought that by using USB3.0 ports, that even though they are running in 2.0 speed, the USB bus still has the headroom of USB 3.0. Instead, the USB 3.0 ports act exactly like 2.0 ports, limiting the TOTAL bus throughput to that of USB 2.0.
Sorry for this long winded post, but does anyone know why this would occur, and how to design against it?
I have an oddball question -
I have 6 blu-ray duplication towers that connect to a PC via USB 2.0. These towers have standard 500 gig HDDs in them, which are formatted in EXT2 format (the towers use embedded software to allow for ISOs from the PC to go to the HDD so that you can duplicate the ISOs directly from the drive). I have all 6 connected to a PC with a super fast SSD RAID (4x120gig, RAID0, on adaptec 4 port card; all modern stuff). The PC has 12 USB ports, 6 of them are USB 3.0, all the towers are plugged in to USB 3.0 ports.
So my problem is that I'm trying to get rid of the bottlenecks in transferring a single file from the PC to all 6 of the towers at once. Currently what happens is that the transfer speed drops down to 10 MB/sec when all 6 are copying at the same time. I know I won't get 60 MB/sec (USB 2.0 max speed). I thought I could get max bandwidth out of USB because the SSD RAID can easily provide 1500 MB /sec, and I know the max speed of my towers is 60 MB/sec. 6 x 60MB = 360 MB/sec - easily under the limits of the RAID.
I thought that by using USB3.0 ports, that even though they are running in 2.0 speed, the USB bus still has the headroom of USB 3.0. Instead, the USB 3.0 ports act exactly like 2.0 ports, limiting the TOTAL bus throughput to that of USB 2.0.
Sorry for this long winded post, but does anyone know why this would occur, and how to design against it?
More about : usb bandwidth question
If I understated correctly, you start the ISO copy to one unit, then immediately begin the copy to the second , the third, etc, right?
I don't care how fast your RAID is, unless your read cache is large enough to hold the entire ISO the hard drive heads will begin to bounce around from one track to another by the time you have several copies going, which will greatly slow the transfer speed.
You can increase the system memory in the hope your OS will use more for cache, but depending what OS you have you might need to force it to use more memory for disc caching.
I don't care how fast your RAID is, unless your read cache is large enough to hold the entire ISO the hard drive heads will begin to bounce around from one track to another by the time you have several copies going, which will greatly slow the transfer speed.
You can increase the system memory in the hope your OS will use more for cache, but depending what OS you have you might need to force it to use more memory for disc caching.
USB 2 maximum speed tops out at about 30MB/S if you are transferring 6 files at once this will exceed the capacity of your RAID due to the head movement of your drives. Your RAID speed is peak transfer rate for a contiguous file but having to move the heads about will considerably reduce the speed. It would probably be faster to transfer the files one at a time sequentially or you could get a SSD RAID.
USB's pratical limit is about 20-25MB/s (30MB/s is theoretical, and does not include the high overhead for transfer protocol). Each USB bus/controller will have 2 connectors (dividing ~20MB/s into 2 gives you ~10BM/s per connector). Basically, if you have all of your connectors all next to eachother then you are trying to send 2 data streams down the same pipeline. If you connect one dubber per controller then you should get your 20-25 MB/s per device back. Generally the shared USB ports will be stacked on top of eachother (if you were looking at the mobo flat on a table), so if you space them 1 per stack then you should be good.
Also, USB (and any modern interconnect for that matter) will always default to the lowest common device. Plugging a USB2 device into a USB3 port will not only run that port on USB2, but the controller (so the shared USB3 port as well). There is no advantage to plugging a USB2 device into a USB3 port vs a USB2 port, so you may as well use up the slow ports first, and leave the fast ports for other devices down the line.
Hope that helps, at least you know that everything is working correctly. A lot of dub towers have upgrade-able ports or slots on them, which may take a USB3 adapter (my DVD tower does, but is PCI only, so no USB3 for me). No dobut it will cost an arm and a leg, but if this is something you do often then it may be worth it.
Also, USB (and any modern interconnect for that matter) will always default to the lowest common device. Plugging a USB2 device into a USB3 port will not only run that port on USB2, but the controller (so the shared USB3 port as well). There is no advantage to plugging a USB2 device into a USB3 port vs a USB2 port, so you may as well use up the slow ports first, and leave the fast ports for other devices down the line.
Hope that helps, at least you know that everything is working correctly. A lot of dub towers have upgrade-able ports or slots on them, which may take a USB3 adapter (my DVD tower does, but is PCI only, so no USB3 for me). No dobut it will cost an arm and a leg, but if this is something you do often then it may be worth it.
Thanks for the replies. This is all awesome info, I've been stuck on what was happening here. PhilFrisbie and pjmelect, I am working with a SSD RAID. I thought that this would alleviate the heads bouncing around due to solid state and all. Is what you said relevant to SSDs?
CaedenV, I did not realize that the ports stacked on each other are on the same bus. This is actually great news if it is the problem, since I have 12 ports, I can run each tower on 1 separate controller. Also, my dup towers do indeed have a card slot for upgrades but the manufacturer has to provide the card, and they are behind the times with 10/100 ethernet or USB2.0 only right now. They said they would have upgradeable options in the future for gigabit and USB3.0. They are microboards towers.
RE: the speed limit - I believe you may be incorrect, I've swapped the mechanical drive in the towers for a SSD (a sandforce 2 good one) which gives me 55MB/sec. I plan on swapping to all SSDs in the towers IF I know the bottleneck is not inherent to the host PC but is instead in the towers. I want to milk the performance. Plus when I switch to USB3.0 its gonna fly.
And yes sadly it is something I do very very often. Our group goes through a medium sized fortune in media.
CaedenV, I did not realize that the ports stacked on each other are on the same bus. This is actually great news if it is the problem, since I have 12 ports, I can run each tower on 1 separate controller. Also, my dup towers do indeed have a card slot for upgrades but the manufacturer has to provide the card, and they are behind the times with 10/100 ethernet or USB2.0 only right now. They said they would have upgradeable options in the future for gigabit and USB3.0. They are microboards towers.
RE: the speed limit - I believe you may be incorrect, I've swapped the mechanical drive in the towers for a SSD (a sandforce 2 good one) which gives me 55MB/sec. I plan on swapping to all SSDs in the towers IF I know the bottleneck is not inherent to the host PC but is instead in the towers. I want to milk the performance. Plus when I switch to USB3.0 its gonna fly.
And yes sadly it is something I do very very often. Our group goes through a medium sized fortune in media.
Related ressources:
- ForumHow to test USB 3.0 bandwidth usage
- ForumUSB 2.0 bandwidth question
- ForumUSB 3.0 Hub - bandwidth usage query
- ForumUSB Hubs not enough bandwidth for USB products ?
- ForumHow to increase the USB bandwidth windows7
- ForumPCI-E x1 Bandwidth Question
- ForumHow to increase the USB bandwidth windows7
- ForumMicrophone's USB bandwidth .
- ForumReducing USB bandwidth allocated by USB Composite device
- ForumGigabyte GA-X58A-UD3R Bandwidth Question
- ForumBandwidth issues on a family network.
- Forumbandwidth question
- ForumBandwidth Ideas and Issues
- ForumASUS P7P55D-E LX PCIe Bandwidth concern
- ForumMultiPort Hubs vs. Dedicated Bandwidth ?
- ForumFirst SSD advice?
- ForumQuestion about SSDs and CD/DVD/BD burning
- ForumUSB 3.0 and USB 2.0 Compatible?
- ForumI can see my WD 750gb USB mass storage device letter in windows wista
- Forum[Solved] Question about Hard Drive
- More resources
Read discussions in other Storage categories
!
