need lots of fast storage for medium business

[gouletpo]

Hello all !!!

The company for witch I work just scored a big contract. We do paper/microfilm scanning for archival purpose. The problem is that the [strike]fu**ing [/strike]stupid client only want uncompress, 24 bits color images (???). The result is files as big as 60+ MB each. And there will be lots of files ...

My boss therefore asked me to look at expanding our storage capacity. For the moment, we would need something in the order of 2-4 terabytes, accessible over the network. Ideally, it would be nice if we could eventually expand that to 5-6+ terabytes. We also need that storage solution to be pretty fast, because some of the scanners can output images pretty fast. Also, we do a lot processing of the images on the network, witch means reading the image on the network, processing them on a computer, and the rewrite them over the network.

Last year, we bough a 2TB Terastation from Buffalo, a NAS that easily integrated our network. Running it in RAID5 meant 1.5TB of available storage on the network, but that thing is really slow...
Now, I'm looking for some opinions on different upgrade possibilities. I looked at some other NAS, but I don't know if it would be fast enough... Maybe I should go for a fast file server, but would I be able to have that much space on a server (4x500GB drive is only 1.5TB in RAID5). I've heard of SAN, but I don't really know what it is. Any other suggestions ?

In summary, here's what we need :
■lots of space (2+ TB)
■expandability for the future (5+ TB)
■data security (RAID5 is enough for us)
■fast data transfer rate

For information, here is an overview of our network :
■Dell Powervault 745N as main server (DHCP, file server, print server, ...)
■Buffalo 2TB Terastation
■20+ computers running windows 2000/XP, with scanner attached to them
■Dell 1Gb switch
■right now, all scanners scan images to powervault (terastation is too slow) and we transfer them to terastation over night if we need room. But our powervault only has 650 GB of storage space (it's getting old...)
■we are running a workgroup instead of a domain because it grew slowly from 5 computers. I know a domain might be better, but we have a lot of XP home witch would require upgrading to pro and we don't really care about permissions.

Now for the price point... the lowest would be best, but if I can justify the additional expanse to my boss for a faster / more expandable solution, it still works. The final price has to be between 1000$ and 15 000$ (only if it his really worth it). I would appreciate any comment / opinions for a solution. Ask me if you need any more information to make a good choice.

 

[IH8U]

Other than Dell, yes I said the hated name (still better than Mac tho) try this:
http://www.newegg.com/Product/Product.aspx?Item=N82E16859105210
($2,700 HP DL320s)

load it down with 12 terabyte drives (max to 9Tb)
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136151
($230 each, x 12 = $2,760)

total $5460 + S&H

 

[hsarc]

For a good NAS if it will fit in your budget we use NetAPP's. These can be quite fast if loaded with the correct hard drives and also even the lower end ones have 4 x 1 Gb ethernet ports for plenty of bandwidth.

 

[ZOldDude]

Grab two old cases with 500W PSU's and two low cost MB's (most have 6 SATA ports).
If your like alot of people you and you buddy already have a old system you can bring in from home.
Stuff them with 500GB Seagate 7200.11's w/32MB buffers ($99-109).

In fact you could just open the side of -any- computer...lay a HD in the bottom of the case,hook it up,fill it with data and repeat.
Place the filled HD's back into the anti-static wrapper and into the shipping boxs for safe storage.

Thats the "cheapest" way.

EDIT: I fill up HD's like eating donuts. I just use a $19 universial USB adaptor (PATA/SATA/Notebook) and don't even bother to install the drives any longer...just copy the data from the system onto the new drive and put it back in it's shipping box.

 

[hsarc]

I would not use large 7200 RPM hard drives. He is much better if speed is important using 15K RPM SAS drives and a good Raid Card. Also something with multiple GigaBit Ethernet ports that can be bound together to offer more data throughput.

NetAPP makes very fast NAS devices but I do not know if they will fit in your budget. They can also scale from small to very large.

 

[GeorgeH]

It seems foolish to give a recommendation when all we know is "fast."

We don't know how much scanning is being done (either burst speed or average speed.

We don't know how often edits are being done.

We don't know the volume of files except that 2TB might be a reasonable guess.

All we know is that a relatively slow server is being used now - the Dell Powervault.

For under $2K anyone can build a very fast Windows machine with a 4TB RAID that would appear to solve the problem.

 

[audiovoodoo]

Grab two old cases with 500W PSU's and two low cost MB's (most have 6 SATA ports).

*SNIP*

Thats the "cheapest" way.

In the short term maybe. In a commercial setting integrity and availablity of the data are crucial. If you want fast, high availability storage then you can't get away from spending more that you would on a domestic solution. Supremelaw makes some very good suggestions on ways to go. A dedicated card with large cache will work wonders.

One thing that does seem to have been overlooked in this quest for storage is backup. RAID != backup.

 

[UncleDave]

Anybody going to dip their toes in the water and offer backup advice? Redundancy is still not catered for in only having one machine.... Maybe upgrading the network cards in the PCs can make a world of difference to your network speeds.... Expected growth from other clients, etc... "archival purpose" well how about some MO drives or maybe DVD on the PCs? How near does the client require data? "we don't really care about permissions" you will once some little n00b hacks your network.

I've said it before but.... while the advice in this thread is (I believe) good I don't think that a holistic approach is being taken - the cheapest and best solution are rarely if ever the same. With all due respect to the posters and OP - get somebody into your shop who can architect a solution not just look at one part of IT requirements.

UD.

 

[SomeJoe7777]

I would want an Enterprise-level solution for this. The previous suggestions of building a machine around an off-the-shelf RAID card doesn't seem like a good idea in this case, even if it is cheaper.

I also don't think that SAS drives are necessary. In fact, it would increase costs tremendously to go SAS. 15K/10K drives are fast at IOPs, which are tasks involving lots of small reads & writes and random accesses. This is for applications like database, OLTP, OLAP, large Exchange server installations, virtual machines/ESX server, etc. This is not necessary in this case. With this application using write-once read-many large contiguous files, we need transfer rate and capacity. Large 7200 RPM SATA drives will work fine.

Since a Dell machine already exists in the organization, they probably already have a sales contact at Dell that handles their account, so I would look at a Dell solution here. The Dell MD3000i iSCSI SAN unit is ideal. A base configuration unit with 4x 750GB drives in a RAID 5 (2.2TB storage) is about $6300. The unit is expandable to 15 drives, so you could later add drives up to around 10TB.

I would then keep the PowerVault 745 as the main server on the network (Domain controller, DHCP, print, e-mail), but move file services to a new PowerEdge 1950 server that would be hooked to the MD3000i. Put 4GB of RAM in the 1950, do dual Gigabit network to the MD3000i, and dual Gigabit to the rest of the network. This is the file server configuration I use at my office (although I use a different SAN, the MD3000i wasn't out when I bought mine), and it works very well. In a base configuration with redundant power supply, RAID 1 for the C:, and rails, the PE 1950 comes to around $2300.

Finally, you need to be able to back all of that up. You could do this two ways:

1) If you just want the most recent copy of the data and don't care about past versions, then adding 3x 750GB drives to the MD3000i and doing a disk-to-disk backup would probably be cheapest. You'd need some decent backup software. I recommend Symantec Backup Exec 11d.

2) A more traditional backup solution would be backup to tape. Few tape drives on the market are suitable to handle several TB of data automatically. Take a look at the HP MSL2024 Tape Library with an LTO-4 tape drive. This unit has 24 slots, and 800GB native per tape, about $7000. If you go tape, get LTO. Don't bother with SuperDLT. Quantum's SDLT products used to be really good, but they've gone down in quality in the latest generation, and LTO has really taken over the market.

You could also spend a little less on the tape drive with the HP 1/8 G2 Autoloader. It's also LTO-4, but has less slots, therefore less total capacity.

There is another, less expensive option. It may be worth investigating Adaptec's SnapServer units. They can be almost as much money as the Dell MD3000i, but they actually have their own integrated NAS head, obviating the need for the file server (PowerEdge 1950 I mentioned above). So it could save some money, if you're comfortable with Adaptec.

 

[hsarc]

I was recommending the NetApp for many of those reasons. It also has SnapMirror options. We have 6 or 7 at this point. They can be loaded with large 7200 RPM drives though you take a noticeable performance hit with large or many small files or our other setup which has 15K SAS drives. Both have 4 Gigabit Ethernet Connections bonded together so plenty of Network throughput. As for expandability well they start at 4TB and go up to 100's of TereBytes and you can add on as needed. They are also a known and respected brand for NAS type devices for commercial purposes. Unfortuntely you can not go online and see the prices. My guess for a lower end head and one shelf of drives you could get it under the 15K mark on his high end it really depends on how you configure it with drives. A series of 7200 RPM drives will be cheaper.