48Tb NAS - Your thoughts? Or a custom build of up to 72Tb?
Tags:
- NAS / RAID
- Build
-
Storage
- Product
Last response: in Storage
henri br
May 4, 2011 1:01:51 AM
Thecus N16000
We've been working on project which may require us to share a few hundreds educational contents/files in audio, video & book formats. We'd like to share these contents on eMule/bitTorrent similar apps, as well through HTTP/FTP. Also, as we've been in need of a network-attached storage solution I thought this one would be a solution for all these needs. The problem is that this NAS seems to be in "beta version": 1, 2, 3, 4.
The NAS requirements is something that can:
- Handle 15 local persons editing and messing these contents
- At the same time, share these contents on eMule/bitTorrent similar apps, as well through HTTP/FTP
- Support Sata III 3Tb HDDs like this Hitachi
- Support 10Gb Ethernet card
- Any 2U, 3U or up to 5U NAS would be fine
- Be someway future proof, and provide at least 48Tb of storage
This is not a mission critical system, and we are not worried on uptime.
It's more to local use, P2P sharing on a 200Mbit WAN...
The critical part of this project may be hosted somewhere "in the cloud".
What are your thoughts about this Thecus N16000 NAS?
Do you know some other that would fit on above requirements?
Any advice on these as I'm a newbie on this subject?
Thanks, and by the way it's nice to become part on Tom's Hardware!
We've been working on project which may require us to share a few hundreds educational contents/files in audio, video & book formats. We'd like to share these contents on eMule/bitTorrent similar apps, as well through HTTP/FTP. Also, as we've been in need of a network-attached storage solution I thought this one would be a solution for all these needs. The problem is that this NAS seems to be in "beta version": 1, 2, 3, 4.
The NAS requirements is something that can:
- Handle 15 local persons editing and messing these contents
- At the same time, share these contents on eMule/bitTorrent similar apps, as well through HTTP/FTP
- Support Sata III 3Tb HDDs like this Hitachi
- Support 10Gb Ethernet card
- Any 2U, 3U or up to 5U NAS would be fine
- Be someway future proof, and provide at least 48Tb of storage
This is not a mission critical system, and we are not worried on uptime.
It's more to local use, P2P sharing on a 200Mbit WAN...
The critical part of this project may be hosted somewhere "in the cloud".
What are your thoughts about this Thecus N16000 NAS?
Do you know some other that would fit on above requirements?
Any advice on these as I'm a newbie on this subject?
Thanks, and by the way it's nice to become part on Tom's Hardware!
More about : 48tb nas thoughts custom build 72tb
henri br
May 5, 2011 3:22:25 PM
I know that above equipment is for 16 drives.
But, how does it compare to this 24 bays Sans Digital AccuRAID AR424I?
Or even to the 16 bays AccuRAID AR316I?
I'm quite lost on this.
Could someone help?
But, how does it compare to this 24 bays Sans Digital AccuRAID AR424I?
Or even to the 16 bays AccuRAID AR316I?
I'm quite lost on this.
Could someone help?
m
0
l
Henri
Sans Digital AccuRAID AR424I is iSCSI box it wont support HTTP/FTP. File server and Bittorrent, so as AccuRAID AR316I
BTW what do you mean of:
- Handle 15 local persons editing and messing these contents
Are we talking about editing/messing with SD or HD content?
Let me do a little research I will post later where you go or call...
Sans Digital AccuRAID AR424I is iSCSI box it wont support HTTP/FTP. File server and Bittorrent, so as AccuRAID AR316I
BTW what do you mean of:
- Handle 15 local persons editing and messing these contents
Are we talking about editing/messing with SD or HD content?
Let me do a little research I will post later where you go or call...
m
0
l
Related resources
- Custom Gaming PC Build. Thoughts? - Forum
- Home NAS Build, thoughts, ideas? - Forum
- Thoughts on my custom build, please - Forum
- Custom PC Computer Build Thoughts Suggestions & Feedback - Forum
- New Custom Build, would love your thoughts - Forum
henri br
May 5, 2011 8:47:23 PM
Thanks for clarifying FireWire2!
I've been visiting several storage solutions providers and some forums...
But, I'm still quite lost on this as I'm not literate on these subjects.
- Handle 15 local persons editing and messing these contents
Are we talking about editing/messing with SD or HD content?
We are group of 5 persons working on several projects that overall, are related to culture and education.
There are some other projects that we help to maintain.
Frequently, we count on other 10 to 15 persons helping us on our daily activities.
These persons should have access to the NAS on the LAN.
Probably, we'll need that to provide remote access as well on a "root level" at some cases; acess to everything.
On this August, we are going to start to receive a mass amount of files in video, audio, and book formats to be reviewed.
Up to 25 persons will be editing these files, and make them ready for free distribution.
There will be some HD contents; so yes, we are talking about "editing/messing with some SD or HD content".
This distribution will be accomplished mainly through P2P file sharing, and some files through HTTP/FTP.
HTTP/FTP is not our main focus, and we can pay a hosting providers for this.
When it comes to the structure for P2P sharing, we'll be distributing the files through a 200Mbit FTTx connection.
We still don't have the routers, switches and other network equipment for this. Probably, we'll buy some Cisco stuff.
So, we'd like to plan it properly and hear our friends in Tom's Hardware community to do our best with regard to:
- Buy the right "things", keeping in mind the future!
The solution will be also used for our storage/back up needs; and maybe for learning about virtualization technologies (Hyper-V from MS, VMware, etc).
I've edited the first post, as I did some mistakes there:
It is an at least 48Tb solution.
It can be a 2U, 3U, or up to 5U rack mount solution.
I've been visiting several storage solutions providers and some forums...
But, I'm still quite lost on this as I'm not literate on these subjects.
FireWire2 said:
BTW what do you mean of:- Handle 15 local persons editing and messing these contents
Are we talking about editing/messing with SD or HD content?
We are group of 5 persons working on several projects that overall, are related to culture and education.
There are some other projects that we help to maintain.
Frequently, we count on other 10 to 15 persons helping us on our daily activities.
These persons should have access to the NAS on the LAN.
Probably, we'll need that to provide remote access as well on a "root level" at some cases; acess to everything.
On this August, we are going to start to receive a mass amount of files in video, audio, and book formats to be reviewed.
Up to 25 persons will be editing these files, and make them ready for free distribution.
There will be some HD contents; so yes, we are talking about "editing/messing with some SD or HD content".
This distribution will be accomplished mainly through P2P file sharing, and some files through HTTP/FTP.
HTTP/FTP is not our main focus, and we can pay a hosting providers for this.
When it comes to the structure for P2P sharing, we'll be distributing the files through a 200Mbit FTTx connection.
We still don't have the routers, switches and other network equipment for this. Probably, we'll buy some Cisco stuff.
So, we'd like to plan it properly and hear our friends in Tom's Hardware community to do our best with regard to:
- Buy the right "things", keeping in mind the future!
The solution will be also used for our storage/back up needs; and maybe for learning about virtualization technologies (Hyper-V from MS, VMware, etc).
I've edited the first post, as I did some mistakes there:
It is an at least 48Tb solution.
It can be a 2U, 3U, or up to 5U rack mount solution.
m
0
l
henri br
May 5, 2011 9:21:39 PM
henri br
May 5, 2011 10:09:08 PM
There would be no problem to build a solution from the scratch, since we can have some help.
The time and patience we already have...
That's it fellow:
An scalable/expandable NAS solution.
A SAN would not work for us as it is somehow expensive and would require much more money than a NAS.
I don't see many solutions providing support for SATA III and SAS 6G HDDs. Why?
Also, a solution with SSD support for the O/S is a huge plus, no?
The time and patience we already have...
That's it fellow:
An scalable/expandable NAS solution.
A SAN would not work for us as it is somehow expensive and would require much more money than a NAS.
I don't see many solutions providing support for SATA III and SAS 6G HDDs. Why?
Also, a solution with SSD support for the O/S is a huge plus, no?
m
0
l
Best solution
OK - Want to pay me to build it for you :-) LOL
Here this is I would do - base on my experience.
Intel MB with at least TWO PCI Ex 16x V2.0
CPU - your pick. There is not much of CPU usage when run as NAS
Mem - 2~4GB is good enough
16bay 3U rackmount server chassis
500W Redundant PSU (check the 12V line make sure it rates at 28.0A min)
16x port hardware raid SAS6 controller: ARECA, ADAPTEC, HTP, LSI, 3Ware
- I always used ARECA, since I rarely need support from them, but you may choose US base manufacture
16x 3.0TB HDD Ultrastar
10Gb Chelsio Ethernet card
1x USB thumb drives - 4GB
Software
1st choice: FreeNAS
2nd choice: OpenFiler
3rd choise: Chelsio USS
4th choice: Open-E
5th choice: 2008 server
FreeNAS
Create a boot CD
Load FreeNAS to USB as embeded OS
Configure the RAID per your desire RAID5/5, hot spare
Boot from just created USB
Log In FreeNAS
Mount the raid volume
Configure SMB/NFS/FTP/Bittorrent services
Fine tuning the SYTEM change MTU, tcp, udp...
I like FreeNAS because it's an open source. Help is readily available in it forum, just like Tom's Hardware forum
I have not try 10Gb Ethernet with FreeNAS, but Open-E. I think it should work, because FreeBSD supports it
Most of Linux/Unix base NAS runs from RAM after it boot, and it normally runs for months so there is no need to use SSD.
If you are using 2008 server then you may consider SSD
Hope this will get you start
Here this is I would do - base on my experience.
Intel MB with at least TWO PCI Ex 16x V2.0
CPU - your pick. There is not much of CPU usage when run as NAS
Mem - 2~4GB is good enough
16bay 3U rackmount server chassis
500W Redundant PSU (check the 12V line make sure it rates at 28.0A min)
16x port hardware raid SAS6 controller: ARECA, ADAPTEC, HTP, LSI, 3Ware
- I always used ARECA, since I rarely need support from them, but you may choose US base manufacture
16x 3.0TB HDD Ultrastar
10Gb Chelsio Ethernet card
1x USB thumb drives - 4GB
Software
1st choice: FreeNAS
2nd choice: OpenFiler
3rd choise: Chelsio USS
4th choice: Open-E
5th choice: 2008 server
FreeNAS
Create a boot CD
Load FreeNAS to USB as embeded OS
Configure the RAID per your desire RAID5/5, hot spare
Boot from just created USB
Log In FreeNAS
Mount the raid volume
Configure SMB/NFS/FTP/Bittorrent services
Fine tuning the SYTEM change MTU, tcp, udp...
I like FreeNAS because it's an open source. Help is readily available in it forum, just like Tom's Hardware forum
I have not try 10Gb Ethernet with FreeNAS, but Open-E. I think it should work, because FreeBSD supports it
Most of Linux/Unix base NAS runs from RAM after it boot, and it normally runs for months so there is no need to use SSD.
If you are using 2008 server then you may consider SSD
Hope this will get you start
Share
henri br
May 6, 2011 3:38:01 PM
Thanks for being promptly!
What are your rates?
Sans Digital AccuRAID AR424I is iSCSI box it wont support HTTP/FTP. File server and Bittorrent, so as AccuRAID AR316I
I'm not sure if could understand it.
1. Why it wouldn't support? Could you explain the above statemet?
Software
1st choice: FreeNAS
2nd choice: OpenFiler
3rd choise: Chelsio USS
4th choice: Open-E
5th choice: 2008 server
I have not try 10Gb Ethernet with FreeNAS, but Open-E. I think it should work, because FreeBSD supports it
Most of Linux/Unix base NAS runs from RAM after it boot, and it normally runs for months so there is no need to use SSD.
If you are using 2008 server then you may consider SSD
Hope this will get you start
Quite insightful!
1st choice: FreeNAS; yes.
5th choice: 2008 server; it may be useful for learning, so we may try it also.
I thought the AccuRAID main differences was the 10Gb Ethernet and some other better hardware.
2. So a 10Gb Ethernet card would work at no problem for HTTP/FTP/Etc?
I'm going to study a bit more about the hardware you've advised.
3. What kind of hardware would support Sata/Sas both 6 Gbit/s HDDs? And also SDD "embedded"?
Thanks,
Henri
What are your rates?
FireWire2 said:
Sans Digital AccuRAID AR424I is iSCSI box it wont support HTTP/FTP. File server and Bittorrent, so as AccuRAID AR316I
1. Why it wouldn't support? Could you explain the above statemet?
FireWire2 said:
Software
1st choice: FreeNAS
2nd choice: OpenFiler
3rd choise: Chelsio USS
4th choice: Open-E
5th choice: 2008 server
I have not try 10Gb Ethernet with FreeNAS, but Open-E. I think it should work, because FreeBSD supports it
Most of Linux/Unix base NAS runs from RAM after it boot, and it normally runs for months so there is no need to use SSD.
If you are using 2008 server then you may consider SSD
Hope this will get you start
Quite insightful!
1st choice: FreeNAS; yes.
5th choice: 2008 server; it may be useful for learning, so we may try it also.
I thought the AccuRAID main differences was the 10Gb Ethernet and some other better hardware.
2. So a 10Gb Ethernet card would work at no problem for HTTP/FTP/Etc?
I'm going to study a bit more about the hardware you've advised.
3. What kind of hardware would support Sata/Sas both 6 Gbit/s HDDs? And also SDD "embedded"?
Thanks,
Henri
m
0
l
My rate $135.00/hr :-), but today is free :-)
1_ iSCSI is a network storage, it's centralize the data storage. When mount a iSCSI drive in your system like a local C: or D: drive, but not a network drive... more info go here: http://en.wikipedia.org/wiki/ISCSI
iSCSI does not have networks services such as: HTTP/FTP/SAMBA/NFS....
2_ 10GbE, Bonding GbE, or 10/100Mb NIC they all work for HTTP/FTP/NFS/ATP... It just the available of system services and whether it enable or disable.
3_ Most of the new SAS controllers are SAS/SATA 6Gb, to utilize it the PCI Express slot MUST be V2.0
1_ iSCSI is a network storage, it's centralize the data storage. When mount a iSCSI drive in your system like a local C: or D: drive, but not a network drive... more info go here: http://en.wikipedia.org/wiki/ISCSI
iSCSI does not have networks services such as: HTTP/FTP/SAMBA/NFS....
2_ 10GbE, Bonding GbE, or 10/100Mb NIC they all work for HTTP/FTP/NFS/ATP... It just the available of system services and whether it enable or disable.
3_ Most of the new SAS controllers are SAS/SATA 6Gb, to utilize it the PCI Express slot MUST be V2.0
m
0
l
henri br
May 6, 2011 7:18:18 PM
I've been reading that link on WikiPedia, but I couldn't get into the point and realize it.
Anyway, I'll read it again and again.
1. So, any iSCSI based storage would not fit into requirements, right?
I saw many people talking about this controller card (link) and I can see many other around there; but most does not specify SATA III or 6 Gb/s... That's why the concern. This one I've mentioned seems to support it, but many "built solutions" or even some controllers do not specify it clearly.
2. What about the SSD? Does the SDD HDDs are also plugged on those controller cards? If not, what is needed to also support 2 SSDs for an operating system?
That card on above link seems to support SDD.
Edit: Yes, all SATA/SAS/SSDs HDDs can be connected to a controller card that support it. So, the question two is answered.
Any note about it?
I've been reading this website (link) and a few other to understand how these things works, and what are the possibilities.
That website has a comparision chart:
SAS Expanders vs. SAS (no expander)
SAS Expanders vs. SATA Multilane
SAS Expanders vs. Port Multipliers
BTW: I were not able to fully understand those charts; mainly the diagrams.
Image for reference on the following questions (link)
3. Can those internal SFF-8087 connectors be used for other things than connecting 4 HDDs each? If so, what for?
4. Can I mix SATA/SAS/SSD on each connector with no problem? (curious)
5. What are those connectors in the middle of the back heatsink? What is that used for?
Another image for reference on the following questions (link)
6. The center connector is a SFF-8088. As I understood, it can be used for other 4 HDDs. But, is it also used for connecting to an identical card so we could have another set with other 24 HDDs (making the NAS expand-able)? What are the purposes of that external connector in terms of possibilities?
Entire review (link)
7. What I still don't understand is, if in the future we another set with more 24 HDDs, and then more 24 HDDs; each time we need to extend, do we need to buy all the things again (MB, CPU, Mem, another controller, etc.)? How does it work (I mean, the process of extending to up to 128 HDDs)? How are the "stuff" connected?
Thanks so much for your efforts!
Anyway, I'll read it again and again.
1. So, any iSCSI based storage would not fit into requirements, right?
I saw many people talking about this controller card (link) and I can see many other around there; but most does not specify SATA III or 6 Gb/s... That's why the concern. This one I've mentioned seems to support it, but many "built solutions" or even some controllers do not specify it clearly.
2. What about the SSD? Does the SDD HDDs are also plugged on those controller cards? If not, what is needed to also support 2 SSDs for an operating system?
That card on above link seems to support SDD.
Edit: Yes, all SATA/SAS/SSDs HDDs can be connected to a controller card that support it. So, the question two is answered.
Any note about it?
I've been reading this website (link) and a few other to understand how these things works, and what are the possibilities.
That website has a comparision chart:
SAS Expanders vs. SAS (no expander)
SAS Expanders vs. SATA Multilane
SAS Expanders vs. Port Multipliers
BTW: I were not able to fully understand those charts; mainly the diagrams.
Image for reference on the following questions (link)
3. Can those internal SFF-8087 connectors be used for other things than connecting 4 HDDs each? If so, what for?
4. Can I mix SATA/SAS/SSD on each connector with no problem? (curious)
5. What are those connectors in the middle of the back heatsink? What is that used for?
Another image for reference on the following questions (link)
6. The center connector is a SFF-8088. As I understood, it can be used for other 4 HDDs. But, is it also used for connecting to an identical card so we could have another set with other 24 HDDs (making the NAS expand-able)? What are the purposes of that external connector in terms of possibilities?
Entire review (link)
7. What I still don't understand is, if in the future we another set with more 24 HDDs, and then more 24 HDDs; each time we need to extend, do we need to buy all the things again (MB, CPU, Mem, another controller, etc.)? How does it work (I mean, the process of extending to up to 128 HDDs)? How are the "stuff" connected?
Thanks so much for your efforts!
m
0
l
henri br
May 8, 2011 2:11:41 AM
Knok, knok... Anyone home?
Everytime we need a new build, Intel makes us crazy with so many sockets/chipsets.
As we'll use this server to play with VMware, Hyper-V SP1 from Microsoft, or other, I'm wondering what cpu/mb to buy at this time...
And, did I remember to mention TrueCrypt and encrypted volumes?
LGA 1366? What about it's getting retired?
LGA 1155? What are the future's perspectives for this?
LGA 1567? Powerful (low clock speed) processors, quite expensive. Is it worth?
What else?
Bah! What to do, considering the future?
We were thinking about:
X5690 (upgrad-able to 2 processors)
W3690 (single processor option)
or something at E7 family
Mind to share your opinions on this?
Everytime we need a new build, Intel makes us crazy with so many sockets/chipsets.
As we'll use this server to play with VMware, Hyper-V SP1 from Microsoft, or other, I'm wondering what cpu/mb to buy at this time...
And, did I remember to mention TrueCrypt and encrypted volumes?
LGA 1366? What about it's getting retired?
LGA 1155? What are the future's perspectives for this?
LGA 1567? Powerful (low clock speed) processors, quite expensive. Is it worth?
What else?
Bah! What to do, considering the future?
We were thinking about:
X5690 (upgrad-able to 2 processors)
W3690 (single processor option)
or something at E7 family
Mind to share your opinions on this?
m
0
l
Wow, that's lots of quesitons :-)
1_ iSCSI wont working with your requirements
As far as SATA III and II you may see a different in the raid itself (speed), but the intend system may have 10Gb E, which is very easy to saturated with the 16/24 drives raid card, regardless SATAII or SATAIII.
Addition to it, the SATA III chipsets with 4 ports are not readily available compare to SAS III...
2_ SSD is very good in OS with HDD access regularly like Windows, but application specific like: Open_filer, Free-NAS, Open-E, USS, Open-Samba... After boot it runs from RAM, therefore SSD is not really benefit
Also, once configured, you can save the configuration to a file... which it take seconds to restore
With FreeBSD/FreeNAS there is ZFS feature which is really cool... the person with the BEST knowledge about it is Submesa. You can use the ARC-1300-4x/i and SAS expander to have indiscriminate HDD pool with up to 128 HDD
Other questions:
miniSAS on SAS controller does have dual purposes: Expanding and HDD connection
a_ you can connect to HDD and the SAS controller will see 4x drives
b_ connect to SAS expander, then SAS controller will see all the HDD connect to the expander and other DAISY chain SAS expander's HDD
If you are using ZFS feature then you get the powerful CPU, other wise save the money for the hw raid controller
1_ iSCSI wont working with your requirements
As far as SATA III and II you may see a different in the raid itself (speed), but the intend system may have 10Gb E, which is very easy to saturated with the 16/24 drives raid card, regardless SATAII or SATAIII.
Addition to it, the SATA III chipsets with 4 ports are not readily available compare to SAS III...
2_ SSD is very good in OS with HDD access regularly like Windows, but application specific like: Open_filer, Free-NAS, Open-E, USS, Open-Samba... After boot it runs from RAM, therefore SSD is not really benefit
Also, once configured, you can save the configuration to a file... which it take seconds to restore
With FreeBSD/FreeNAS there is ZFS feature which is really cool... the person with the BEST knowledge about it is Submesa. You can use the ARC-1300-4x/i and SAS expander to have indiscriminate HDD pool with up to 128 HDD
Other questions:
miniSAS on SAS controller does have dual purposes: Expanding and HDD connection
a_ you can connect to HDD and the SAS controller will see 4x drives
b_ connect to SAS expander, then SAS controller will see all the HDD connect to the expander and other DAISY chain SAS expander's HDD
If you are using ZFS feature then you get the powerful CPU, other wise save the money for the hw raid controller
m
0
l
henri br
May 10, 2011 11:07:26 PM
FireWire2 said:
Wow, that's lots of quesitons :-)It's true... lol
And you've been of great help here!
I'm researching the parts for this rig, and now the question is:
1. Which 10GbE (dual port) should we buy? SFP, RJ45? Could someone help on this, and post some links?
Here it is an Intel's link for reference only.
Maybe a helpfull faq here (for those who are able to fully understand it - Not my case =D)
Another Intel's link
Jumbo frames is "a must have", no?
We are going to pick some Cisco parts to build this network.
We don't have none network equipment for this build yet.
The "best" motherboard (for this) I've found seems to be the SuperMicro X8DAH+ (link to specs).
It is a LGA 1366 - Maybe I'm wrong about this choice, considering my last post; I'm not sure.
m
0
l
That is very good MB.
I would use only ONE CPU instead
Look at the Chelsio 10Gb cards. It's VERY GOOD adapter:
http://www.chelsio.com/assetlibrary/t4-docs/DS-T420-SO-...
There is low cost 10Gb switch is CX4, but this have limited in cable length, where SFP you can use a long cable (Optical) if need it.
Look at toshiba 10Gb swithes... they are very decent
With 16.24 ports SAS controller then you are good to go
I would use only ONE CPU instead
Look at the Chelsio 10Gb cards. It's VERY GOOD adapter:
http://www.chelsio.com/assetlibrary/t4-docs/DS-T420-SO-...
There is low cost 10Gb switch is CX4, but this have limited in cable length, where SFP you can use a long cable (Optical) if need it.
Look at toshiba 10Gb swithes... they are very decent
With 16.24 ports SAS controller then you are good to go
m
0
l
henri br
May 11, 2011 2:03:14 AM
Yes, we are going to buy only one CPU.
When, and IF needed, maybe we buy a mate for him.
Anyway, that motherboard would give this possibility.
About the network structure...
There is no need for long cables, I mean more than 100 meters long. 20 to 60 meters would be enough.
We still need to study it better, we are going to do it after we decide which server we are going to "assemble".
Note: For those who are also learning about 10GbE, there are some explanation here (link)
Thanks for the Chelsio T420-SO-CR link.
I'm going to study it, and learn more.
And for those Intel's; which would you pick considering you already know what's involved on this project? link
This question is because we're in Brazil, and I'm sure we can get a fast/local support from Intel.
BTW: It seems we are going to have not only a NAS, but a multi-purpose server, which NAS is the main one.
And a note on NFS vs CIFS, URL 2
When, and IF needed, maybe we buy a mate for him.
Anyway, that motherboard would give this possibility.
About the network structure...
There is no need for long cables, I mean more than 100 meters long. 20 to 60 meters would be enough.
We still need to study it better, we are going to do it after we decide which server we are going to "assemble".
Note: For those who are also learning about 10GbE, there are some explanation here (link)
Thanks for the Chelsio T420-SO-CR link.
I'm going to study it, and learn more.
And for those Intel's; which would you pick considering you already know what's involved on this project? link
This question is because we're in Brazil, and I'm sure we can get a fast/local support from Intel.
BTW: It seems we are going to have not only a NAS, but a multi-purpose server, which NAS is the main one.
And a note on NFS vs CIFS, URL 2
m
0
l
henri br
May 12, 2011 4:59:05 PM
Until now I thought of going with the following hardware:
01x ... Motherboard: SuperMicro X8DAH+ (link)
01x ... Processor: Intel Xeon X5690, Westmere-EP (link)
02x ... Memories: CT3KIT102472BB1339 3x8 (link)
01x ... Raid card: Areca ARC-1880ix-24-4G (link)
01x ... 10GbE: Intel X520-T2 (link)
01x ... Video card: Nvidia Quadro 4000 (link) - Yes, we may use the server directly for HD video edition
20x ... HDDs: Hitachi HUA723030ALA640 3Tb 6Gb (link)
03x ... SSDs: Intel 510 120Gb (link) - In raid 5 for O.S./virtualization, maybe?
01x ... Optical drive: Sony Optiarc BD-5740H Slim (link)
1. Am I missing/forgetting something? Should something be included?
2. Will above hardware work for what we are proposing?
3. What would you change in it?
I know it is not included the power supply, as we'll need to sum the power requirements; but I can't remember of other parts right now.
LCD monitor and chassis we'll buy from a local dealer.
Anyway, is there something we should pay attention when it comes to the storage chassis/enclosure?
4. Anything special with it? We have no idea on what kind of thing it requires.
I'm going to start doing it today. I'm going to check some switches.
Thanks A LOT for your GREAT assistance FireWire2!
Parts, quick specification:
--------------------------------
Motherboard: SuperMicro X8DAH+ - Full description link
Dual 1366-pin LGA • dual 5520 Chipsets • 288GB DDR3-1333 • Intel 82576 Dual-GigE • 6xSATA2 via ICH10R • 1 EIDE, ALC888 7.1 HD Audio • 2x IEEE 1394a • 2 PCI-E x16, 4 x8, 1 x4 (in x8 slot)
Processor: Intel Xeon X5690, Westmere-EP - Full description link
6 cores 12 threads • 3.46 GHz - 3.73 GHz max turbo • 12 MB cache • 2 QPI links • 64-bit • SSE4.2 • max of 288 GB DDR3-1333 3 channels memory • 130 W
*If more than 6 memory modules are in use, the frequency decrease to 1066. I'm not so sure about it.
Memories: Crucial CT3KIT102472BB1339 - Full description
2x 24GB kit (8GBx3) • DDR3 PC3-10600 1333 • CL=9 • Dual Ranked • Registered • ECC • 1.5V • 1024Meg x 72
Raid card: Areca ARC-1880ix-24-4G - Full description
1 x SFF-8088 mini-SAS (external) • 6 x SFF-8087 mini-SAS • PCIe 2.0 x8 • 4GB ECC DDR2-800 • RAID 0, 1, 1E, 3, 5, 6, 10, 30, 50, 60 or JBOD • 6Gb/s and 3Gb/s SAS/SATA/SSD drives
10GbE: Intel X520-T2 - Full description & Comparison link
10Gbps Cat 6A up to 100m Copper • PCIe 2.0 • Intel VT-c (VMDq & VMDc) • NO fiber channel over ethernet
Video card: Nvidia Quadro 4000 - Full description
256 CUDA Cores • 2 GB GDDR5 256-bit • DX 11 / OpenGL 4.0/3.2 • 1 Dual Link DVI-I • 2 DisplayPort • 3D Vision/3D Vision Pro • 142W
HDDs: Hitachi HUA723030ALA640 - Full description
SATA 6Gb/s • 3TB • 64 Mb buffer • 7200 RPM • 512 Bytes/sector • Seek Time (read, ms, typical): 8.2 • Dual Stage Actuator (DSA) & Enhanced Rotational Vibration Safeguard (RVS) • 11.3 W
SSDs: Intel 510 120Gb - Full description
120GB • Read: up to 400 MB/s • Writes: up to 240 MB/s • Random 4KB Reads: up to 20K IOPS • Random 4KB Writes: up to 8K IOPS
Optical drive: Sony Optiarc BD-5740H Slim Full description
BD 6X • DVD 8X • CD 24X • SATA • 5.8 Mb buffer
01x ... Motherboard: SuperMicro X8DAH+ (link)
01x ... Processor: Intel Xeon X5690, Westmere-EP (link)
02x ... Memories: CT3KIT102472BB1339 3x8 (link)
01x ... Raid card: Areca ARC-1880ix-24-4G (link)
01x ... 10GbE: Intel X520-T2 (link)
01x ... Video card: Nvidia Quadro 4000 (link) - Yes, we may use the server directly for HD video edition
20x ... HDDs: Hitachi HUA723030ALA640 3Tb 6Gb (link)
03x ... SSDs: Intel 510 120Gb (link) - In raid 5 for O.S./virtualization, maybe?
01x ... Optical drive: Sony Optiarc BD-5740H Slim (link)
1. Am I missing/forgetting something? Should something be included?
2. Will above hardware work for what we are proposing?
3. What would you change in it?
I know it is not included the power supply, as we'll need to sum the power requirements; but I can't remember of other parts right now.
LCD monitor and chassis we'll buy from a local dealer.
Anyway, is there something we should pay attention when it comes to the storage chassis/enclosure?
4. Anything special with it? We have no idea on what kind of thing it requires.
FireWire2 said:
I would go for the CAT6 Intel® Ethernet Server Adapter X520-T2. But you need to check out the 10Gb E switches with CAT6 firstI'm going to start doing it today. I'm going to check some switches.
Thanks A LOT for your GREAT assistance FireWire2!
Parts, quick specification:
--------------------------------
Motherboard: SuperMicro X8DAH+ - Full description link
Dual 1366-pin LGA • dual 5520 Chipsets • 288GB DDR3-1333 • Intel 82576 Dual-GigE • 6xSATA2 via ICH10R • 1 EIDE, ALC888 7.1 HD Audio • 2x IEEE 1394a • 2 PCI-E x16, 4 x8, 1 x4 (in x8 slot)
Processor: Intel Xeon X5690, Westmere-EP - Full description link
6 cores 12 threads • 3.46 GHz - 3.73 GHz max turbo • 12 MB cache • 2 QPI links • 64-bit • SSE4.2 • max of 288 GB DDR3-1333 3 channels memory • 130 W
*If more than 6 memory modules are in use, the frequency decrease to 1066. I'm not so sure about it.
Memories: Crucial CT3KIT102472BB1339 - Full description
2x 24GB kit (8GBx3) • DDR3 PC3-10600 1333 • CL=9 • Dual Ranked • Registered • ECC • 1.5V • 1024Meg x 72
Raid card: Areca ARC-1880ix-24-4G - Full description
1 x SFF-8088 mini-SAS (external) • 6 x SFF-8087 mini-SAS • PCIe 2.0 x8 • 4GB ECC DDR2-800 • RAID 0, 1, 1E, 3, 5, 6, 10, 30, 50, 60 or JBOD • 6Gb/s and 3Gb/s SAS/SATA/SSD drives
10GbE: Intel X520-T2 - Full description & Comparison link
10Gbps Cat 6A up to 100m Copper • PCIe 2.0 • Intel VT-c (VMDq & VMDc) • NO fiber channel over ethernet
Video card: Nvidia Quadro 4000 - Full description
256 CUDA Cores • 2 GB GDDR5 256-bit • DX 11 / OpenGL 4.0/3.2 • 1 Dual Link DVI-I • 2 DisplayPort • 3D Vision/3D Vision Pro • 142W
HDDs: Hitachi HUA723030ALA640 - Full description
SATA 6Gb/s • 3TB • 64 Mb buffer • 7200 RPM • 512 Bytes/sector • Seek Time (read, ms, typical): 8.2 • Dual Stage Actuator (DSA) & Enhanced Rotational Vibration Safeguard (RVS) • 11.3 W
SSDs: Intel 510 120Gb - Full description
120GB • Read: up to 400 MB/s • Writes: up to 240 MB/s • Random 4KB Reads: up to 20K IOPS • Random 4KB Writes: up to 8K IOPS
Optical drive: Sony Optiarc BD-5740H Slim Full description
BD 6X • DVD 8X • CD 24X • SATA • 5.8 Mb buffer
m
0
l
I would go with a lesser Video card, cuz it's a server and most of the time it runs as headless
If you runs as FreeNAS, then there is no need for SSD :-)
Save those $ get a Redundant PSU good chassis with SAS 6.0Gb/SATAIII support
http://www.datoptic.com/unified-storage-server-data-cen...
But 24bay instead - call them they have the 24bays chassis with redundant PSU, three modules instead of two
If you runs as FreeNAS, then there is no need for SSD :-)
Save those $ get a Redundant PSU good chassis with SAS 6.0Gb/SATAIII support
http://www.datoptic.com/unified-storage-server-data-cen...
But 24bay instead - call them they have the 24bays chassis with redundant PSU, three modules instead of two
m
0
l
Here is what I built Dual Gb Ethernet Hardware raid 40TB Media Server consumes 180W of power. It can expand up to 30 HDD, or 90TB RAW with 3TB Drives
http://www.tomshardware.com/forum/265641-32-40tb-server...
http://www.tomshardware.com/forum/265641-32-40tb-server...
m
0
l
henri br
May 13, 2011 3:02:38 AM
Hello there,
I've read about your build and started learning exactly with that =D
I've read it also on another forum. That's nice!
I've mentioned that video card because we were going to build a workstation to deal with video edition / rendering.
As our primary role won't be playing with videos, and it'll be someway sporadically we are thinking about not to build it.
A workstation would be somehow costly when compared to just throw a video card on that server.
Then, we may use the server for such "intensive tasks".
Again, I may be wrong about it; I mean, about going this way.
Maybe, a "gaming class video card" would fit perfectly, and do a lot more in general terms than the overpriced Quadro cards.
GTX 580's, 590's, HD 6990... Who knows!?
A GTX 590 (spec. link) is priced for $30 less than that Quadro on NewEgg (link), there's also the AMD HD 6990 (spec. link).
Anyway, I know that Quadro's aims on a different market, has different software/drivers optimized, etc.
Just to remember, budget for a video card shouldn't be more than $800 - "Is it a server!?" :-D
I'm going to study these points further.
1. What do you think?
Some links about it: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11; for those who are interested.
For the SSDs, fellow, I'll tell you
The guys here really loves to try every piece of thing, and make changes every now and then.
I'm quite sure that, at some point, we're going to test a lot of things in this server...
So, as I know it, I want to be prepared.
2. Probably it's a dumb question, but, I actually would like it to be a RAID 10, and not 5 as I've told.
That choice is due to 6 x SFF-8087 on the RAID card: 20 for HDDs, 3 for SDDs and 1 for the Blu-ray burner.
What about the motherboard's ports? That mobo has 6xSATA2...
Will it be deactivated when installed the RAID card? If so, is it possible to keep those 6xSATA "active"? Any drawback on it?
I'm asking because we could "plug" the blu-ray burner on the first port of the mobo, and use that freed port for the 4th SDD: RAID10
http://www.datoptic.com/unified-storage-server-data-cen...
But 24bay instead - call them they have the 24bays chassis with redundant PSU, three modules instead of two
Thanks for the link and "SAS 6.0Gb/SATAIII support" notification!
3. What do you mean with "three modules"?
Just a side note, in terms of power consumption, with this current build we're working, you'd be able to build other 4 rigs like yours ;-)
Cheers
I've read about your build and started learning exactly with that =D
I've read it also on another forum. That's nice!
I've mentioned that video card because we were going to build a workstation to deal with video edition / rendering.
As our primary role won't be playing with videos, and it'll be someway sporadically we are thinking about not to build it.
A workstation would be somehow costly when compared to just throw a video card on that server.
Then, we may use the server for such "intensive tasks".
Again, I may be wrong about it; I mean, about going this way.
Maybe, a "gaming class video card" would fit perfectly, and do a lot more in general terms than the overpriced Quadro cards.
GTX 580's, 590's, HD 6990... Who knows!?
A GTX 590 (spec. link) is priced for $30 less than that Quadro on NewEgg (link), there's also the AMD HD 6990 (spec. link).
Anyway, I know that Quadro's aims on a different market, has different software/drivers optimized, etc.
Just to remember, budget for a video card shouldn't be more than $800 - "Is it a server!?" :-D
I'm going to study these points further.
1. What do you think?
Some links about it: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11; for those who are interested.
For the SSDs, fellow, I'll tell you
The guys here really loves to try every piece of thing, and make changes every now and then.
I'm quite sure that, at some point, we're going to test a lot of things in this server...
So, as I know it, I want to be prepared.
2. Probably it's a dumb question, but, I actually would like it to be a RAID 10, and not 5 as I've told.
That choice is due to 6 x SFF-8087 on the RAID card: 20 for HDDs, 3 for SDDs and 1 for the Blu-ray burner.
What about the motherboard's ports? That mobo has 6xSATA2...
Will it be deactivated when installed the RAID card? If so, is it possible to keep those 6xSATA "active"? Any drawback on it?
I'm asking because we could "plug" the blu-ray burner on the first port of the mobo, and use that freed port for the 4th SDD: RAID10
FireWire2 said:
Save those $ get a Redundant PSU good chassis with SAS 6.0Gb/SATAIII support http://www.datoptic.com/unified-storage-server-data-cen...
But 24bay instead - call them they have the 24bays chassis with redundant PSU, three modules instead of two
Thanks for the link and "SAS 6.0Gb/SATAIII support" notification!
3. What do you mean with "three modules"?
Just a side note, in terms of power consumption, with this current build we're working, you'd be able to build other 4 rigs like yours ;-)
Cheers
m
0
l
henri br
May 15, 2011 8:56:18 AM
Some storage chassis manufacturers I've found...
http://www.supermicro.com/
http://www.chenbro.com/
http://www.aicipc.com/
http://www.dynapowerusa.com/
http://www.norcotek.com/
http://www.polywell.com/
http://www.istarusa.com/
http://www.urack.com.tw/
http://www.arktechinc.com/
http://www.procase.nl/
http://www.pcicase.co.uk/
http://www.habeyusa.com/
http://www.primesource.in/
http://www.supermicro.com/
http://www.chenbro.com/
http://www.aicipc.com/
http://www.dynapowerusa.com/
http://www.norcotek.com/
http://www.polywell.com/
http://www.istarusa.com/
http://www.urack.com.tw/
http://www.arktechinc.com/
http://www.procase.nl/
http://www.pcicase.co.uk/
http://www.habeyusa.com/
http://www.primesource.in/
m
0
l
Three modules redundant gives less stress to the remain module when power fail happen.
Two modules redundant PSU
When run each provide 50% power, when one goes bad the other have to supply 100%, therefore 100% increase power
Where three modules:
One module failed, only 33.3% power is needed to provide from the remain two, therefore each power module only need to increase additional 50% power.
So three modules redundant power supply (RPS) will be more reliable then the the RPS with two modules
Two modules redundant PSU
When run each provide 50% power, when one goes bad the other have to supply 100%, therefore 100% increase power
Where three modules:
One module failed, only 33.3% power is needed to provide from the remain two, therefore each power module only need to increase additional 50% power.
So three modules redundant power supply (RPS) will be more reliable then the the RPS with two modules
m
0
l
henri br
May 19, 2011 6:01:06 AM
@FireWire2,
What kind of HDD are you using in your 40TB Media Server?
Answer: 20x WD Green 2.0TB WD20EARS (link)
I'm OVERloaded trying to learn so many things in a short period, but it is good.
What is a really effective anti-vibrating solution to add in this new(bie) build for the HDDs?
Let's suppose we'd buy this chassis; not sure, however.
What kind of HDD are you using in your 40TB Media Server?
Answer: 20x WD Green 2.0TB WD20EARS (link)
I'm OVERloaded trying to learn so many things in a short period, but it is good.
What is a really effective anti-vibrating solution to add in this new(bie) build for the HDDs?
Let's suppose we'd buy this chassis; not sure, however.
m
0
l
henri br
May 24, 2011 11:19:36 PM
Daniel925
May 26, 2011 12:08:07 AM
I think if you are running a 10gbit network you should do pretty well. 10gigbit/8 1,250megabits of throughput. The old rule of thumb was something like 40% before networks start to degrade so really with 1,250x40% or 500megabytes per second for 10gigbit, 50megabytes for 1gigbit, and 5 megabytes for 100megbit network links. Divide that by 15 local persons and it starts to get pretty painful doing things over the slower networks.
There is some stuff now with SAS switches that might be faster for you if everyone is located near enough to each other. I have read you can get upto 2400mega bytes/second over sff-8088 cables.
There is some stuff now with SAS switches that might be faster for you if everyone is located near enough to each other. I have read you can get upto 2400mega bytes/second over sff-8088 cables.
m
0
l
henri br
May 27, 2011 2:54:54 AM
It's interesting Daniel.
That Intel X520-T2 has 2 10Gb ports - 2.500MB of throughput - Or those 1GB following that rule.
In this case, probably, a bottle neck could be the disks'/array's throughput.
What makes me ponder at this moment is with regard to upgrades or possible problems. For hardware RAID, when we select a controller card, it seems we're "prisoner of it". If something goes wrong, we may need the very same model to keep the data - If that card or specific model is not available, you're in trouble. If something better shows up, maybe there's no way to go with it without backing up all the data and reconfigure the entire RAID, what means more expenses and/or some real work. What if you don't have a way to backup 48Tb+ for an upgrade/improvements?
Is it actually this way? How do you guys deal with these kinds of things?
That Intel X520-T2 has 2 10Gb ports - 2.500MB of throughput - Or those 1GB following that rule.
In this case, probably, a bottle neck could be the disks'/array's throughput.
What makes me ponder at this moment is with regard to upgrades or possible problems. For hardware RAID, when we select a controller card, it seems we're "prisoner of it". If something goes wrong, we may need the very same model to keep the data - If that card or specific model is not available, you're in trouble. If something better shows up, maybe there's no way to go with it without backing up all the data and reconfigure the entire RAID, what means more expenses and/or some real work. What if you don't have a way to backup 48Tb+ for an upgrade/improvements?
Is it actually this way? How do you guys deal with these kinds of things?
m
0
l
henri br
May 28, 2011 4:03:42 AM
Good to know it.
What about dealing with upgrades?
Get a better up-to-date RAID card?
How does it work?
It would not be that nice to buy e.g. an external enclosure and some dozens of disks, backup the data, upgrade the card and 'configure it', and re-transfer the data. Another 24 disks would cost at least $3K, 40 disks ~$5,3K; depending on the volume of data you have.
Also, a RAID controller you selected may not be available on the market in a few months or years; then you must to upgrade. This is the kind of hardware I'd be glad to upgrade if something much better appears.
What would to be a wise/best way to deal with it? I mean, an upgrade.
What about dealing with upgrades?
Get a better up-to-date RAID card?
How does it work?
It would not be that nice to buy e.g. an external enclosure and some dozens of disks, backup the data, upgrade the card and 'configure it', and re-transfer the data. Another 24 disks would cost at least $3K, 40 disks ~$5,3K; depending on the volume of data you have.
Also, a RAID controller you selected may not be available on the market in a few months or years; then you must to upgrade. This is the kind of hardware I'd be glad to upgrade if something much better appears.
What would to be a wise/best way to deal with it? I mean, an upgrade.
m
0
l
Related resources
- Thoughts on custom pc build. Forum
- Thoughts on my custom, semi-budget first build? Forum
- High-End NAS or Custom rack build? Forum
- Home Server - NAS and streaming and online back-up duties. Is this build fit for purpose? Forum
- SolvedI am building my first custom gaming PC. I would like some advice on how to set up my fans. Forum
- 550$ budget, came up with this build. Thoughts? Forum
- SolvedBuilding My First Gaming Set up (Thoughts On This Set up) Forum
- New Custom Build - Thoughts Forum
- New Custom Build PC Keeps Shutting off before reaching the windows start up screen Forum
- Custom pc build please advice on thoughts Forum
- overclocking q9300 custom build thoughts? Forum
- First ground up build. Thoughts? Forum
- Need Help Custom build boots up and 3 seconds later shuts down then repeats. Forum
- New custom build attempt can't get it to fire up Forum
- My first build is up and running! Need your thoughts! Forum
- More resources
Read discussions in other Storage categories
!