USB, Windows XP, RAID, Portable Desktop, and Ultrafast Boot Times?

bobbo124

Distinguished
Oct 9, 2007
4
0
18,510
About a week or so ago while I was messing around on the web I ran across this article which drew my attention: http://www.ngine.de/index.jsp?pageid=4176. So, being bored at work the next day I decided to try it. I followed the guide step-by-step and successfully got a PC up and running, fully functional with Windows XP loaded on one of my 4gb thumb drives. The load times however were a little disappointing. It took probably 10 minutes to boot up and trying to work in windows was even more painful. It was still pretty amusing seeing XP up and running on a computer with no hard drives. Working in IT, the possibilities for such a device would be huge if only it didn't take so long to start. Theoretically, I could do virus removals, browse or save data on a hard drive that can't boot into windows without having to take out the hard drive, run diagnostics or software with preinstalled software, etc. For personal use, one could have windows, all their software, files, games and music all on their thumb drive, take it out of their computer, go to a friends house, work or anywhere with a computer and put it into a different computer and use it as if it were their desktop at home. Of course there would be driver issues and such but keep a few different chipset/ethernet drivers in a folder and they could download whichever ones they needed. The possibilities seem endless... if it wasn't so slow.

Then, I remembered another article I had read. Jason Kohrs from geeks.com successfully made a RAID array out of 4 usb thumb drives and a usb2.0 hub. http://www.bigbruin.com/reviews05/thumbraid_1 He had some pretty impressive results using all 4 in a RAID-0:

chart2.gif


The catch was, he could only get it working in a Linux environment. This got me thinking, If I could get a RAID-0, even a RAID-5 up and running with windows XP off a hub of thumb drives, not only could it be extremely useful but in theory extremely fast and that I could even use a setup permanently for my home PC to have ultrafast boot times and application load times. It would basically serve as a temporary, cheap alternative to the *drool* upcoming solid-state and hybrid hard drives. It got me a little excited so the next day I went out and got 4x 1GB Lexar USB Flash Drives and a 4-port usb2.0 hub and set to work.

I plugged all 4 of the drives into the hub, and plugged the hub into my laptop running Windows XP to make sure they all functioned. They did, so I formatted all 4 of them to NTFS. (They need to be either NTFS or FAT16 to be able to install windows on them.) I opened up the Disk Manager and sure enough, I was unable to create any kind of raid with them. The problem is that Windows is unable to convert any external usb devices to a Dynamic Disk, which is needed to make any disc a part of a RAID. Obviously, the default windows software raid wasn't going to work for my project, so like Jason Kohrs, I moved on to Linux.

Now, I have almost zero experience with linux, so this part was not very fun. After alot of research into linux software raid, I finally got one sort of running using the MDADM program/command on the Ubuntu LiveCD. Still, I did not understand what I was doing, just following guides I found online. I got the array configured and all in a RAID-0 and formatted it to NTFS, all the lights on the drives were blinking in the correct order and it was looking good. I could not however, figure out how to mount the hard drive. (access denied?) I could probably figure out how to get it mounted and working under linux, but there is no point in that considering that windows won't recognize a linux software RAID. So I moved on to investigating ways to get windows to work with an array of usb drives.

This is where I'm stuck. I have several theories to what might work but I need some input since I have no experience with linux or arrays for that matter. My Ideas are these:

1) Find a third-party program that will create software arrays that start before the OS boots and recognizes USB.

2) Find software that allows the conversion of USB devices to Dynamic Disks with Windows XP, create the array, reboot, and try to install windows to the new array.

3) Use 1 larger usb drive and 3 smaller ones. Create a partition on the larger drive equal to the size of one of the smaller drives, install linux on the other partition and install windows XP on a virtual machine using a RAID-0 or RAID-5 made up of the other 3 drives or the 3 drives and the other partition of the larger drive. Heres an illustration of what I mean:

vpc3.jpg


It would boot into Linux from the first drive and then open a virtual machine with XP installed on an array of the blue partitions.

If anyone has any input or opinions about these ideas, knows of any software that can help, or has any expertise with linux or RAID.. please leave a comment or email me: bobbo124@gmail.com

I'll keep you guys updated with progress and ill add a link to download the .iso of the edited windows xp disc you need. Hope you think this is an interesting idea as well =)
 

alex_d

Distinguished
Nov 12, 2007
1
0
18,510
hi, ran across this via google...
yeah, I've been thinking along those lines too. However, there's some big gotchas.


First, USB flash has a significant flaw: if you erase the same spot too many times, it breaks. There's ways around this (called wear leveling), but if you simply install Windows to it then the pagefile (among other things) will burn a hole in the flash.

I've researched this, and I found no ready-made way of doing wear leveling, but it's possible to write a Linux tool that wear levels (ie, scrambles up) the virtual disk file. The problem with this path, though, is that virtual machine graphics won't work well and performance in other areas, including disk access, suffers too.

I think a good enough solution is to get usb flash software raid working post-boot. It can be used for storing Program Files (Games), video, or vms. Wear leveling is still a concern, but it's not so bad with no pagefile or temp files.

Again, I tried but didn't find any software solutions. There was one third-party software raid program, but it had poor performance. Window's implementation has really killed the competition in this niche.

I think the solution is to write something. E.g., implementing your solution #2 shouldn't be hard, but writing a full Raid 5 driver is reasonable too (it could have much higher performance. Windows doesn't use any sse, for example. it could also implement wear leveling.) Anyone interested in working on this with me, or just want to say they like the idea?
 

SomeJoe7777

Distinguished
Apr 14, 2006
1,081
0
19,280


You do not need to worry about this. Flash drives do wear-leveling in hardware on the flash chips. Even if Windows thinks a file hasn't moved from the sectors it supposedly occupies, the flash chip itself has already moved it to equalize wear.