Use of PAR files Quickpar v. RAID5 or RAID6 or other

Status
Not open for further replies.
Mar 9, 2018
5
0
10
I've got many large family movies which I'd like to confirm won't be subject to bit rot. There are also photos that I'd like to keep safely as well. Thoughts were using a NAS and RAID 5 or RAID 6 (only for a larger unit with 5 drives) and also alternatives. After looking through everything, it seems that using plain old PAR and Windows or plain Linux is the best bet IMHO because in the event something goes wrong with the controller of the NAS or RAID controller, everything is probably lost. It might not be if I can get an identical device to put the drives into, e.g. if you had a DROBO Model XYZ you'd need another one or a Synology 2019PLAY, etc.

My thought was to have a big USB drive that has 4 disks and shows them as individual disks in standard NTFS format. This is much quicker than Ethernet and this is for long term backup, not needing to put on the network. I'd copy files to 3 of the drives and leave the 4th for a Parity drive holding Quickpar or Multipar (PAR2) files which can be used for parity rebuilds should files fail. I can also use Checksum software to create an additional hashing file to determine whether bits have changed / corruption. I'm not sure how I'll automate the PAR creation but, for now, with the videos that I'd add onto the drive I would drop them onto the Quickpar app which would then use my settings to generate PAR files on the fourth drive holding my parity files.

Seems there is no clear consensus but, in the event of major failure, this provides for the most likely chance of success in recovering data in native format off of a hard drive. The ONLY thing I haven't figured out yet is how to get a map that could let me know if any individual files have been corrupt (or possibly I can run the Checksum app to do that automated.)

Would love go hear insights.
 
Mar 9, 2018
5
0
10
No, of course it's not the only copy. For the most important items I have them on a high quality blu-ray (some M-disc) along with an extra copy on a second hard drive. But what I'm looking for is party protection. I'm wondering whether this is a better approach than RAID-5 and RAID-6 or software RAID with parity protection for corruption. That's the issue.
 

kanewolf

Titan
Moderator


None of those statements addressed my suggestion of off-site protection. You could create PARs. CRCs would be another idea. If you have multiple copies, you can compare the CRCs to see if the files have changed. Disks are good for maybe 10 years, If you chose to compare CRCs and rewrite the data every year or so you would keep the bits "fresh".

I am not a big fan of your multiple disk approach. If you want to go with a commercial NAS, then I think you would be OK. RAID doesn't protect from anything except disk failure. Multiple independent copies (with optional CRC) is the only way to ensure data permanence.
 
Mar 9, 2018
5
0
10
The off-site protection are the multiple drives that are stored elsewhere. But that's easy to do. It's the data integrity of the files themselves that is the main problem. The PARs allow for checking and rebuilding of files. Yes, using CRC or another hash is good but that is just checking whether a file has changed, which is good for the check part but the PARs allow for the fix.

What solution does a commercial NAS provide? I'm guessing you might be talking about BTRFS and mapping, something like ZFS, which is what people use. Very intensive, extremely heavy lifting and very expensive.

I'd certainly look forward to solutions. Criticism is easy and regularly forthcoming elsewhere which is why I devised my own solution which seems to work optimally and better than most I've read about. The RAID 5 and 6 also do rebuilding and correction of corrupted files but that can be very intensive and not as focused on individual files so you can repair those without having it impact other files around it, e.g. like a PAR for multiple files or a directory.
 
Status
Not open for further replies.