[Solved] 48Tb NAS - Your thoughts? Or a custom build of up to 72Tb?

I know that above equipment is for 16 drives.
But, how does it compare to this 24 bays Sans Digital AccuRAID AR424I?
Or even to the 16 bays AccuRAID AR316I?

I'm quite lost on this.
Could someone help?

Thanks for clarifying FireWire2!
I've been visiting several storage solutions providers and some forums...
But, I'm still quite lost on this as I'm not literate on these subjects.

We are group of 5 persons working on several projects that overall, are related to culture and education.
There are some other projects that we help to maintain.
Frequently, we count on other 10 to 15 persons helping us on our daily activities.
These persons should have access to the NAS on the LAN.

Probably, we'll need that to provide remote access as well on a "root level" at some cases; acess to everything.

On this August, we are going to start to receive a mass amount of files in video, audio, and book formats to be reviewed.
Up to 25 persons will be editing these files, and make them ready for free distribution.
There will be some HD contents; so yes, we are talking about "editing/messing with some SD or HD content".

This distribution will be accomplished mainly through P2P file sharing, and some files through HTTP/FTP.
HTTP/FTP is not our main focus, and we can pay a hosting providers for this.
When it comes to the structure for P2P sharing, we'll be distributing the files through a 200Mbit FTTx connection.

We still don't have the routers, switches and other network equipment for this. Probably, we'll buy some Cisco stuff.
So, we'd like to plan it properly and hear our friends in Tom's Hardware community to do our best with regard to:
- Buy the right "things", keeping in mind the future!

The solution will be also used for our storage/back up needs; and maybe for learning about virtualization technologies (Hyper-V from MS, VMware, etc).

I've edited the first post, as I did some mistakes there:
It is an at least 48Tb solution.
It can be a 2U, 3U, or up to 5U rack mount solution.

It's always fun to DYI, but time and patience it's a must have; especially with a brand new project.

There would be no problem to build a solution from the scratch, since we can have some help.
The time and patience we already have...

That's it fellow:
An scalable/expandable NAS solution.
A SAN would not work for us as it is somehow expensive and would require much more money than a NAS.

I don't see many solutions providing support for SATA III and SAS 6G HDDs. Why?
Also, a solution with SSD support for the O/S is a huge plus, no?

Thanks for being promptly!
What are your rates?

I'm not sure if could understand it.
1. Why it wouldn't support? Could you explain the above statemet?

Quite insightful!
1st choice: FreeNAS; yes.
5th choice: 2008 server; it may be useful for learning, so we may try it also.

I thought the AccuRAID main differences was the 10Gb Ethernet and some other better hardware.
2. So a 10Gb Ethernet card would work at no problem for HTTP/FTP/Etc?

I'm going to study a bit more about the hardware you've advised.
3. What kind of hardware would support Sata/Sas both 6 Gbit/s HDDs? And also SDD "embedded"?

Thanks,
Henri

I've been reading that link on WikiPedia, but I couldn't get into the point and realize it.
Anyway, I'll read it again and again.

1. So, any iSCSI based storage would not fit into requirements, right?



I saw many people talking about this controller card (link) and I can see many other around there; but most does not specify SATA III or 6 Gb/s... That's why the concern. This one I've mentioned seems to support it, but many "built solutions" or even some controllers do not specify it clearly.

2. What about the SSD? Does the SDD HDDs are also plugged on those controller cards? If not, what is needed to also support 2 SSDs for an operating system?
That card on above link seems to support SDD.

Edit: Yes, all SATA/SAS/SSDs HDDs can be connected to a controller card that support it. So, the question two is answered.
Any note about it?



I've been reading this website (link) and a few other to understand how these things works, and what are the possibilities.
That website has a comparision chart:
SAS Expanders vs. SAS (no expander)
SAS Expanders vs. SATA Multilane
SAS Expanders vs. Port Multipliers
BTW: I were not able to fully understand those charts; mainly the diagrams.

Image for reference on the following questions (link)
3. Can those internal SFF-8087 connectors be used for other things than connecting 4 HDDs each? If so, what for?
4. Can I mix SATA/SAS/SSD on each connector with no problem? (curious)
5. What are those connectors in the middle of the back heatsink? What is that used for?

Another image for reference on the following questions (link)
6. The center connector is a SFF-8088. As I understood, it can be used for other 4 HDDs. But, is it also used for connecting to an identical card so we could have another set with other 24 HDDs (making the NAS expand-able)? What are the purposes of that external connector in terms of possibilities?

Entire review (link)

7. What I still don't understand is, if in the future we another set with more 24 HDDs, and then more 24 HDDs; each time we need to extend, do we need to buy all the things again (MB, CPU, Mem, another controller, etc.)? How does it work (I mean, the process of extending to up to 128 HDDs)? How are the "stuff" connected?


Thanks so much for your efforts!

Wow, that's lots of quesitons :-)

1_ iSCSI wont working with your requirements

As far as SATA III and II you may see a different in the raid itself (speed), but the intend system may have 10Gb E, which is very easy to saturated with the 16/24 drives raid card, regardless SATAII or SATAIII.

Addition to it, the SATA III chipsets with 4 ports are not readily available compare to SAS III...

2_ SSD is very good in OS with HDD access regularly like Windows, but application specific like: Open_filer, Free-NAS, Open-E, USS, Open-Samba... After boot it runs from RAM, therefore SSD is not really benefit

Also, once configured, you can save the configuration to a file... which it take seconds to restore


With FreeBSD/FreeNAS there is ZFS feature which is really cool... the person with the BEST knowledge about it is Submesa. You can use the ARC-1300-4x/i and SAS expander to have indiscriminate HDD pool with up to 128 HDD

Other questions:

miniSAS on SAS controller does have dual purposes: Expanding and HDD connection

a_ you can connect to HDD and the SAS controller will see 4x drives
b_ connect to SAS expander, then SAS controller will see all the HDD connect to the expander and other DAISY chain SAS expander's HDD

If you are using ZFS feature then you get the powerful CPU, other wise save the money for the hw raid controller

That is very good MB.

I would use only ONE CPU instead

Look at the Chelsio 10Gb cards. It's VERY GOOD adapter:
http://www.chelsio.com/assetlibrar [...] -SO-CR.pdf
There is low cost 10Gb switch is CX4, but this have limited in cable length, where SFP you can use a long cable (Optical) if need it.

Look at toshiba 10Gb swithes... they are very decent

With 16.24 ports SAS controller then you are good to go

I would go for the CAT6 Intel® Ethernet Server Adapter X520-T2. But you need to check out the 10Gb E switches with CAT6 first

as far the NAS OS goes:

FreeNAS, OpenFiler, Open-E, USS - does offer both iSCSI and NAS functions
So it can be use as Multi-purposes Server.

I would go with a lesser Video card, cuz it's a server and most of the time it runs as headless

If you runs as FreeNAS, then there is no need for SSD :-)

Save those $ get a Redundant PSU good chassis with SAS 6.0Gb/SATAIII support
http://www.datoptic.com/unified-st [...] enter.html

But 24bay instead - call them they have the 24bays chassis with redundant PSU, three modules instead of two

Hello there,

I've read about your build and started learning exactly with that =D
I've read it also on another forum. That's nice!

I've mentioned that video card because we were going to build a workstation to deal with video edition / rendering.
As our primary role won't be playing with videos, and it'll be someway sporadically we are thinking about not to build it.
A workstation would be somehow costly when compared to just throw a video card on that server.
Then, we may use the server for such "intensive tasks".
Again, I may be wrong about it; I mean, about going this way.
Maybe, a "gaming class video card" would fit perfectly, and do a lot more in general terms than the overpriced Quadro cards.
GTX 580's, 590's, HD 6990... Who knows!?
A GTX 590 (spec. link) is priced for $30 less than that Quadro on NewEgg (link), there's also the AMD HD 6990 (spec. link).
Anyway, I know that Quadro's aims on a different market, has different software/drivers optimized, etc.
Just to remember, budget for a video card shouldn't be more than $800 - "Is it a server!?" :-D
I'm going to study these points further.
1. What do you think?

Some links about it: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11; for those who are interested.



For the SSDs, fellow, I'll tell you
The guys here really loves to try every piece of thing, and make changes every now and then.
I'm quite sure that, at some point, we're going to test a lot of things in this server...
So, as I know it, I want to be prepared.

2. Probably it's a dumb question, but, I actually would like it to be a RAID 10, and not 5 as I've told.
That choice is due to 6 x SFF-8087 on the RAID card: 20 for HDDs, 3 for SDDs and 1 for the Blu-ray burner.
What about the motherboard's ports? That mobo has 6xSATA2...
Will it be deactivated when installed the RAID card? If so, is it possible to keep those 6xSATA "active"? Any drawback on it?
I'm asking because we could "plug" the blu-ray burner on the first port of the mobo, and use that freed port for the 4th SDD: RAID10

Thanks for the link and "SAS 6.0Gb/SATAIII support" notification!
3. What do you mean with "three modules"?



Just a side note, in terms of power consumption, with this current build we're working, you'd be able to build other 4 rigs like yours ;-)

Cheers

Three modules redundant gives less stress to the remain module when power fail happen.

Two modules redundant PSU
When run each provide 50% power, when one goes bad the other have to supply 100%, therefore 100% increase power

Where three modules:
One module failed, only 33.3% power is needed to provide from the remain two, therefore each power module only need to increase additional 50% power.

So three modules redundant power supply (RPS) will be more reliable then the the RPS with two modules

Best answer selected by henri br.

It's interesting Daniel.

That Intel X520-T2 has 2 10Gb ports - 2.500MB of throughput - Or those 1GB following that rule.
In this case, probably, a bottle neck could be the disks'/array's throughput.

What makes me ponder at this moment is with regard to upgrades or possible problems. For hardware RAID, when we select a controller card, it seems we're "prisoner of it". If something goes wrong, we may need the very same model to keep the data - If that card or specific model is not available, you're in trouble. If something better shows up, maybe there's no way to go with it without backing up all the data and reconfigure the entire RAID, what means more expenses and/or some real work. What if you don't have a way to backup 48Tb+ for an upgrade/improvements?

Is it actually this way? How do you guys deal with these kinds of things?

Good to know it.
What about dealing with upgrades?
Get a better up-to-date RAID card?
How does it work?

It would not be that nice to buy e.g. an external enclosure and some dozens of disks, backup the data, upgrade the card and 'configure it', and re-transfer the data. Another 24 disks would cost at least $3K, 40 disks ~$5,3K; depending on the volume of data you have.

Also, a RAID controller you selected may not be available on the market in a few months or years; then you must to upgrade. This is the kind of hardware I'd be glad to upgrade if something much better appears.

What would to be a wise/best way to deal with it? I mean, an upgrade.