Storage Help (NAS + Media Streaming + RAID + Automatic Backup)

Toggle sidebar Toggle sidebar
Aloise

Aloise

Honorable
Dec 27, 2013
6
0
10,510
Hi everyone.

I'm currently facing a dilemma trying to choose the best storage solution for me. I have just bought a Netgear R7000 (Nighthawk) 802.11ac router and would like to build my personal home cloud based on it.

My NAS-storage requirements are:
- Fast, reliable and "virtually" fail-proof
- Hot swappable features, to allow replacing failed HDs on the fly, automatically rebuilding data (disaster recovery). No data loss.
- Store high definition media, such as Full-HD (1080p) videos, MP3/FLAC audio, RAW/JPG pictures and stream it to my TV, iPad, iPhone, Android devices, laptops on my wifi LAN
- Ability to read/write files anywhere in the world, securely.
- Separated profiles, i.e., for admin, regular allowed users and specific users to prevent access to private, classified information/data. Be able to allow access of specific files for specific users.
- Mandatory ability to allow me to automatically backup all my data. In this case, I was thinking of buying, for example a 6-bay NAS and setup a RAID config that allows me to use 3 HDs/bays for data and 3 HDs/bays for backup, all bays with same HD brand/model/capacity.
- Scalable, allowing me to plug other boxes via eSATA (for example) in the future and expand capacity.

1. I was thinking about either Netgear ReadyNAS 516 (6-Bay) or QNAP TS-870 (8-Bay). Any suggestions amongst these? Any other models other than these?

2. One mandatory feature for me is to have the ability to mirror my data on backup dedicated HDs. I know that there's a tradeoff concerning performance vs security (redundancy). I know very little about RAID and thought that maybe RAID6 would be a reasonable option. I also know that if I fill my NAS with 12Tb, 6Tb would be for useful data and 6Tb for backup (in a nutshell). Does it make sense?

3. Also, my personal desire it that, once a drive has failed, I would like to hotswap it with a new one so that the NAS rebuilds the information and no data loss occurs. Is it feasible or am I dreaming?

4. I know these NASes don't work with SAS HDs. So I'll have to stick to SATA Enterprise Class HDs, such as Seagate Constellation or similar. Higher RPM and internal memory would increase disk I/O. Both Netgear and QNAP sell either diskless or loaded with enterprise class HDs. Any comments/suggestions on this?

5. Are these NASes able to stream media "from factory", of course through my wifi router or will I need to install tons of apps/software to do it? Any light on this?

6. I have an Apple TV box. May I also use it as another option to stream media (videos, audio,photos) from the NAS to my TV?

Thanks in advance for your precious help. I've been reading and studying so much about this that I'm almost deciding to take a Ph.D. exam... :)
 
Solution
ronintexas
There are two big decisions to make here - to RAID or not to RAID is the first. The most typical setups are RAID 0 (striping - multiple hard drives become "one"), RAID 1 (mirrored drives - if one fails, the other will operate) and RAID 10 (combined RAID 0/1).

To determine IF RAID is needed, how big of a single volume to you need for any application? Windows Media Center requires a single drive for recording video (from live TV), but with the use of "Libraries", you can seamlessly integrate multiple volumes into a single library. BTW - WMC does not allow for "networked" volume for recording....

I have 4 computers in use at home, mine, my wife's, an HTPC and a test unit. The HTPC has a SSD for OS, and a single 4TB WD Black Drive -...
Sort by date Sort by votes
ronintexas

ronintexas

Judicious
Dec 10, 2012
5,911
0
32,960
There are two big decisions to make here - to RAID or not to RAID is the first. The most typical setups are RAID 0 (striping - multiple hard drives become "one"), RAID 1 (mirrored drives - if one fails, the other will operate) and RAID 10 (combined RAID 0/1).

To determine IF RAID is needed, how big of a single volume to you need for any application? Windows Media Center requires a single drive for recording video (from live TV), but with the use of "Libraries", you can seamlessly integrate multiple volumes into a single library. BTW - WMC does not allow for "networked" volume for recording....

I have 4 computers in use at home, mine, my wife's, an HTPC and a test unit. The HTPC has a SSD for OS, and a single 4TB WD Black Drive - http://www.newegg.com/Product/Product.aspx?Item=N82E16822236622. I can record/watch up to 3 HD channels at once with my digital cable, plus stream to several devices without issue.

My computer has multiple backup drives in it (a combination of 2-4TB drives to backup individual computers) using SyncBack Free to keep files backed up in case of hard drive failure. My DVD collection is stored on the HTPC, and since the files don't change, SyncBack only backs up the new files.

Second consideration is security. Microsoft Windows 7/8 security isn't that great for standard home networking. For devices that connect to the network and share data, there isn't much protection against "snooping". This is where the NAS can give you a boost - as some have great security features...When compared with the cost of going to the "PRO" versions of Windows 7/8 AND buying Server 2008/2012 - you can spend $2000 quickly getting everything connected and shared....the NAS will give you a break.
 
Upvote 0 Downvote
Solution
Aloise

Aloise

Honorable
Dec 27, 2013
6
0
10,510
ronintexas, answering your questions:

1. I don't have any specific needs in terms of applicaton. I don't record video from live TV. The only need I have is regaring space as my data needs are growing fast. RAID might not be the best solution for my case but, if so, which options I have, using a NAS, to safely back up/mirror my data? I have tons of full HD videos and pictures of my son and there's no money in the world that would pay for it. I want to safely store and back it up, but also want a copy of it to be available for me to stream it on my wifi LAN or reach anywhere in the world.

2. NAS is definitely my choice. Not only due to security, but also to be centralize my data. The question is, which one to get?
 
Upvote 0 Downvote
ronintexas

ronintexas

Judicious
Dec 10, 2012
5,911
0
32,960
I haven't purchased or used a NAS array, but in speaking with several colleagues - all are recommending the Synology brand. Their Hybrid RAID system is great for both non-IT people and IT professionals who want a maintenance free install, and their Media Sharing tech is better than virtually any other brand out there. It has good security built in as well and the ability to create your own "virtual cloud" to access data outside your network.

http://www.newegg.com/Product/Product.aspx?Item=N82E16822108123&ignorebbr=1
http://www.newegg.com/Product/Product.aspx?Item=N82E16822108113&ignorebbr=1

Both of these were recommended to purchase.

I will say this - RAID is not a replacement for backups....so backup the NAS! You don't want to lose your data. I always keep multiple copies of files across my simple home network - take a look at SyncBack Free - it would be a good fit for what you are doing....in concert with the NAS.
 
Upvote 0 Downvote
Aloise

Aloise

Honorable
Dec 27, 2013
6
0
10,510
Backup is really a concern and I take it seriously. I'll check SyncBack Free, thanks for informing.
Regarding Synology brand, I've read a little about it and got really impressed. The only con is that some people reported serious problems in a specific scenario. When one disk fails and you hot swap it, if the restoration hangs by any reason, you lose your array, in other words, you lose your data. I've seen not only one report about it, but quite a few. This put my focus on two other brands: Netgear and QNAP.

In addition, I still have questions about backing up data using NAS. In case I choose a 6-bay NAS, can I use 3 bays for data and the other 3 remaining bays to backup this data, all in one single NAS tower?

Also, If I want to configure a RAID in both "partitions" (data and backup), would it be possible? If the answer is yes, which RAID setup would fit better considering this scenario (3 bays/HDs for data and 3 bays/HDs for backup)?
 
Upvote 0 Downvote
ronintexas

ronintexas

Judicious
Dec 10, 2012
5,911
0
32,960
Most RAID software/hardware configuration utilities will allow you to assign specific hard drives to an "array". So if you had a 6-bay NAS, you install 6 2TB drives, you can put 3 in one array, and 3 in the 2nd (this would limit you to RAID 0 or RAID5 - suggest RAID 0 for biggest bang for the buck).

You can then assign them as "DATA" and "BACKUP". BTW - in almost every RAID configuration I have installed and maintained, if power is lost or the system hangs during a rebuild of an array - your stuff is gone....If you configure two separate RAID arrays, the chances of both of them having an issue at the same time is lower than one array, but if the controller card crashes - you can still lose everything.

The way I like to look at RAID is this. If you have one drive, you have a 5% (this figure made up to make things easy for math) chance of failure at any given minute. If you have 2 drives in an array (doesn't matter which version of RAID), it is now 10%. 3 drives = 20%. 4 Drives = 40%. And so on. Buying good solid drives (I suggest WD Black Drives - I install them in my servers and use them at home - very few issues with them), will keep you out of trouble - and also remember - you should plan on changing out the drives in 5-6 years....Most servers I have a 5 year replacement cycle on drives.

And always have a back up....can't say that enough...Boards are full of people, "HELP! I have lost my data - how do it get it back?". If everyone did scheduled backups, they would know all they have to do is restore the data. Mirrored RAID drives still have catastrophic failures and you lose data on both drives...and rebuilding them are usually 95% successful, but ask those who fail.

Given all the possible problems, this is why I usually buy a 4TB WD Black for backups, and 1TB or 2TB WD Black drives for data. I do not use RAID at home....and BTW - even in these NAS appliances - you do not have to use RAID.
 
Upvote 0 Downvote
Aloise

Aloise

Honorable
Dec 27, 2013
6
0
10,510
Great!
This brings up something new on the table for me. And so more questions... :)

Let's consider not to use RAID on the NAS appliance. In this case, in a 6-bay case, is it possible to fill it with 6 HDs, like the WD Blacks that you mentioned ou Seagate Constellation and use them as separate drives (i.e.: C, D, E, F, G, H and I)?

In addition, I want to save half my capacity for backup (50% data and 50% backup) Do you think it's still a good idea or would you recommend another way to do it.

I'm asking this last one because I still do not have a NAS but plan to get one. The thing now is all about backup and if it's not going to be done on 50% of the disks in that NAS, what other options do I have? Get another enclosure only for backup, having 2 NASes on my cloud: one for data and another for backup?
 
Upvote 0 Downvote
S

smitbret

Distinguished
Aug 5, 2002
768
0
19,060
If you really want your NAS to act as a media server and stream video to your AppleTV, you need to either make sure that the videos are in a compatible format or get a NAS setup that has enough CPU power to transcode incompatible media files on the fly.

I'm not crazy about the backup suggestions here, either. You are infinitely better off with 2 different systems and preferably at 2 different physical locations.

Have you primary NAS and stock it with WD Reds in a RAID 5. That will give you the redundancy to be able to swap out HDDs if one fails and still run your array while it rebuilds in a degraded state.

For the secondary, just get a Synology 212j or something and just shove 2 x4TB whatever brands in it and run a nightly backup to it across your network.

Keep one in the basement and one in the garage or something similar. Backing up to an array in the same NAS just seems crazy. Flood, fire, theft, etc. all leave you hanging.
 
Upvote 0 Downvote
ronintexas

ronintexas

Judicious
Dec 10, 2012
5,911
0
32,960
Most should allow you to configure separate drives in a non-RAID setup. Typically, I use 1TB of drive space per user at home, so a 4TB drive easily backs up the data. DVDs and other videos can take up a lot of drive space - so you may need to get a larger drive for that.

Remember one thing - Libraries allow you do define file locations (multiple locations), so you don't have to remember drive letters and/or folder locations. SyncBack lets you define what gets backed up and when - so in theory, all the drive letters and folders "disappear" to the end user.

You can also build a cheap "server" and install FREE NAS or another operating system (like Ubuntu Server) to hold the data as well. You can build one cheaply (about $200-400), and have a lot of the same features. I use my desktop to host the backup drive for the network, and each machine has a large data drive. Utilizing FREE NAS (http://www.freenas.org/), it will be a lot more configurable, and use standard PC parts.

A quick selection of components - http://pcpartpicker.com/p/2r8YT

You can build it for $400....and you can always make it dual purpose - i.e. an HTPC as well as your server....

 
Upvote 0 Downvote
popatim

popatim

Titan
Moderator
Dec 2, 2009
38,904
980
135,890
Better read whichever NAS you are really interested in's user manual carefully. Many will not back up to drives mounted internally and I would not suggest this either. Its alright for a "hot" backup but dont consider this a backup. Should a voltage spike take out the device odds are it will take out all the drives with it. I'm pretty sure the Synology will only backup to network or external drive.

Always have an offline backup or two. I keep a dvd backup in my closet and another in my sisters basement in addition to 2 'hot' backups plus a backup to external drives. Dvd's are pretty easy to maintain as you just need to backup the updates from the last time you made them. By 'offline' I mean one that has no connections: no power or is removed from power, no network cables, no telephone lines

As for losing the raid should something go wrong, yes thats fairly typical of several array types running in degraded mode. If something happens to another disk while one is already down the whole thing is lost. This is why raid is not a backup, its meant to keep your system running in the event of a disk failure.
 
Upvote 0 Downvote
Aloise

Aloise

Honorable
Dec 27, 2013
6
0
10,510
Ok. So I am convinced not to use the same NAS box to also hold my backup disks. It's obvious for me now.

Considering that I'll get 2 NASes, 1 for DATA and 1 for BACKUP:

1. May I connect them both to backup data from one to another?

2. In case I do not use RAID on a 6-bay NAS for DATA, will I have to point in which HD I would like to store a file or get another for streming?

3. In case I do not use RAID on my backup NAS, in case of disk failure, I could only get data from my DATA HDs that contain the same infromation or I could backup my backup.. but this would be crazy as would seam like an endless backup cascade.
 
Upvote 0 Downvote
ronintexas

ronintexas

Judicious
Dec 10, 2012
5,911
0
32,960
1) I would use SyncBack to backup data - it is simple - you put in the network address: \\nas1\data_folder as the original and \\nas2\backup_folder as the destination. SyncBack reads the data (date/time stamp and size) and compares the folder, if a file needs backed up, it copies it, if not, it skips. You can set this to run at 4:00AM when no one is on the computers.

2) Depending upon the software used to "stream" the data, you define the "library" that contains the files to stream. I use \\htpc\recorded_tv, \\htpc\dvds, \\mypc\videos, etc...

3) Having an "original data" (you read/write to this folder location) drive, the backup is seamless to the backup folder. In the case of disk failure, replace the disk and run a restore with SyncBack and it will rebuild that volume. In the "down time", you can still access all of the data on the backup drive.
 
Upvote 0 Downvote
Aloise

Aloise

Honorable
Dec 27, 2013
6
0
10,510
Firstly, thanks once again for your replies. I appreciate and value both your time and information.

I'm thinking of getting a Netgear 516 for DATA and maybe a Netgear 314 or another 516 for BACKUP. I'm sure that QNAP and Synology NASes are equally good. So far, Netgear sounds like a good choice for me. Would any of you guys have a differente oppinion?

DATA NAS
Although RAID6 is an option, I am not confortable with it as I am sure I'll have to get some assistance in case of HDD failure. So, I'm thinking of going JBOD on it.
I'd prefer to keep backing up the data on this DATA NAS to my BACKUP NAS. In case of an HDD failure on my DATA NAS, I'll have the option to swap the faulty drive and restore data from my BACKUP NAS. Sounds more simple to me...
In addition, I'm wondering that in a JBOD setup it's equally easy to access and stream my media files in my wifi LAN. I'm just not sure if I'll have to install additional software to stream such content to my devices (TV, iPad, android smartphone, PC, etc.)

BACKUP NAS
No RAID will be used here as well.
I only have to design the solution so that I'll be able to have the same capacity on DATA and BACKUP NAS. That's why having two 516 NASes sounds more accurate for me than having a 516 for DATA and a 314 for BACKUP, even with different single HDD capacities on each NAS (i.e.: 6x2TB on DATA NAS and 4x3TB on BACKUP NAS)

HDDs
I've done some research about enterprise class SATA HDDs and found better reference on Seagate Constellation HDDs than on WD (reds, blacks, greens, whatever, etc...)
Other than reliability, my concern here is about I/O. I've read that for faster I/O it would be better to install more HDDs with smaller capacity (i.e.: 600Gb... 1TB at most) but for this, I'll have to buy a bigger enclosure and that's not my point. So, it seams quite a good balance to have a 516 filled with 2 or 3Tb HDDs, but I'm not sure if they are either compatible to ReadyNAS boxes (516, 314 ,etc.) and even if they are on the list of approved/compatible drives by Netgear (what would invalidate any support I might need from them). Here are the models that I'm talking about:

Seagate Constellation - 2TB - SATA - 6Gb/s - ST2000NM0033 (without cryptography)
Seagate Constellation - 2TB - SATA - 6Gb/s - ST2000NM0053 (with cryptography)
Seagate Constellation - 3TB - SATA - 6Gb/s - ST3000NM0033 (without cryptography)
Seagate Constellation - 3TB - SATA - 6Gb/s - ST3000NM0053 (with cryptography)

In case these drives are either not compatible or non compliant, which other Enterprise Class HDDs (2Tb and 3Tb) would fit RreadyNAS boxes (516, 314, etc.)?
 
Upvote 0 Downvote
ronintexas

ronintexas

Judicious
Dec 10, 2012
5,911
0
32,960
If I were building your project - I would use utilize the 2TB/3TB drives individually without RAID or JBOD if possible - if you must RAID0 OR JBOD would work to create a single drive. Since theoretically, you are "cloning" the drives from one NAS to the other, any drive failure, you would replace the drive, reinitialize RAID0/JBOD, then restore.

To get the biggest performance boosts using RAID, you typically need a lot of drives. My SQL server utilizes ten 15k SAS drives (300GB each), RAID 10 for a total of 1.5TB of space. My backup drive is a single 4TB WD Black - as speed isn't as critical for the backup. Even with these 10 high speed drives, if you were to measure the performance of the RAID array read/write times vs. the individual drive read/write times, there isn't a huge difference in performance - and I am writing 600GB database files.

For streaming data on a home network - I can use this as an example - I have an HTPC that has a single 2TB WD Black drive (7200RPM) - it supports 3 PC's streaming plus one XBOX 360, and can record up to 3 HD channels at once with streaming two large TVs. The drive is seldom utilized 100% even with all this disk I/O.

To see a speed differential of more than 25% gain, you would need at least 4-6 drives, and you would see it when copying very large files (in the 20GB+ range) sequentially using robocopy.
 
Upvote 0 Downvote
S

smitbret

Distinguished
Aug 5, 2002
768
0
19,060
Using several smaller drives will give you an I/O boost if you are striping, but since OP isn't going to use RAID, then the data won't be spread across multiple drives. Speed in accessing files will be limited by the particular drive is storing the data.

That being said......

If you get any modern 1TB+ HDD the speed of the HDD won't make a lick of difference. The bottleneck will be your Gigabit network speeds. Even a NAS with some juice like the 516 will not be able to do over 120MB/s since that is the theoretical limit of a gigabit connection. Any of the current drives with 1TB platters will match or clear that. 80MB/s is what I would expect. If you get closer to 100MB/s then congratulations on having a pretty good network setup. You could get some benefit from the multiple drives if you were in an environment where there were many users accessing lots of different info simultaneously and the multiple I/O situation applies but if this is just a household setup, it won't come close to being an issue. And in reality, the only time you'll ever really come close to bottlenecking a Gb network is when you are directly transferring files. Even streaming a Blu-Ray video is only gonna need about 50mbps.

If you have the space and budget, you may want to consider RAID 5 for the primary server. It only takes 1 extra drive and with good hardware it's pretty easy and trouble free to maintain. The benefit is that if a drive dies, the NAS stays online. You simply swap out the bad drive and it rebuilds from the parity drive in a degraded state. It takes several hours to do this, but you'll still have access to everything while it does the rebuild. There is a chance that a 2nd drive fails during the rebuild and the entire array ends up shot but in my very limited NAS/Server experience this has never happened. At that point, you are still able to just replace and restore from the backup... no worse off than the plan you have now. For me, the easy swap is worth the investment in 1 extra drive.

Your backup NAS won't gain anything from RAID.

The last question I wanted to make sure you were aware of was the compatibility of your media files with the devices you intend to play them back with. If your video collection is strictly iTunes .m4v files and your music is all .aac and .mp3 then this probably won't be a concern. If you have a mix of .ogg, .flac, .flv, .mkv, .avi, .mpg, .iso, etc. then you need to think about what you're playback or trnascoding solution to make sure you are able to view the media at the different locations.
 
Upvote 0 Downvote
P

purepense

Distinguished
Jan 11, 2010
5
0
18,510


Can you tell me please what hardware you are using on your server SRV2012
I’m planning to build one with Xeon processor for VM and media server on mini ATX case

Thanks.
 
Upvote 0 Downvote
You must log in or register to reply here.

TRENDING THREADS

Latest posts

Moderators online

Share this page

COMPANY
RESOURCES
FOLLOW
Top Bottom