I have a 56 TB local Unraid NAS that is parity protected against single drive failure, and while I think a single drive failing and being parity recovered covers data loss 95% of the time, I’m always concerned about two drives failing or a site-/system-wide disaster that takes out the whole NAS.
For other larger local hosters who are smarter and more prepared, what do you do? Do you sync it off site? How do you deal with cost and bandwidth needs if so? What other backup strategies do you use?
(Sorry if this standard scenario has been discussed - searching didn’t turn up anything.)
A second offsite NAS (my old one) with the same capacity for the larger files
Backblaze B2 and a Hezner storage box for Really Important stuff.
Lto tape. But I only have 15tb
It quickly becomes cost effective when you actually need the data to be safe. Far easier to have off site backups. I have never had a problem , but I like to have offline backup. Most of the time my data is static. So I am only backing up projects files ans changes for the most part.
If you have 40+ tb of dynamic data I can’t help there.
Edit: I buy used drives that are usually 2 generations old, so I got lto-5 drives when lto 7 was new. The used drives may be less reliable but used drives can be 1/10th the price of the newest ones.
Not all data is equal. I backup things i absolutely can not lose and yolo everything else. My love for this hobby does not extend to buying racks of hard drives.
For me, I only back up data I can’t replace, which is a small subset of the capacity of my NAS. Personal data like photos, password manager databases, personal documents, etc. get locally encrypted, then synced to a cloud storage provider. I have my encryption keys stored in a location that’s automatically synced to various personal devices and one off-site location maintained by a trusted party. I have the backups and encryption key sync configured to keep n old versions of the files (where the value of n depends on how critical the file is).
Incremental synchronization really keeps the bandwidth and storage costs down and the amount of data I am backing up makes file level backup a very reasonable option.
If I wanted to back up everything, I would set up a second system off-site and run backups over a secure tunnel.
So you have 56TB of total storage, but how much of that 56TB is actually used? Take the amount of storage used and add 10-12% to that figure. Now you create a new NAS (preferably off-site) with that amount of storage and that becomes your backup target. Take an initial backup (locally if possible to speed up the process) and then you can use something like rsync to create incremental backups going forward. This is the method I’ve used and so far it has worked out well. I target 10-12% more than the amount of used storage for my backup capacity because my storage use grows reasonably slowly. If your usage grows faster you might want to increase your “buffer” a little more so that you’re not having to constantly add drives to your backup target.
Yeah, this is certainly a viable “brute-force”-ish ooption. While I have 56, I’m only using 26 or so. But I’d actually be hesitant to do anything less than a full capacity mirror because I do expect to eventually use this (and more - adding drives to Unraid).
I’ve balked because of cost and upkeep (maintaining the same capacity, additional chances for drive failure, two separate sites I need physical access to with a high bandwidth connection), so I admit I was hoping I was missing an easier option.
I mean, if you want a full mirror, rolling your own backup target is going to be the cheapest option even with the current high price of hardware. Other options are cloud storage, or using another media like tape. Cloud storage is of course an on going cost which rules it out for me, not to mention privacy concerns. There are certain “cold storage” options from cloud storage hosts which are considerably cheaper but they have limitations on how the data can be accessed and how often. The tape route is possible but it’s not really viable for home use due to the high upfront cost of the drives. Outside of that, backing up a subset of your storage as others have suggested is the only other option. Creating viable backups without breaking the bank is a challenge as old as computers, unfortunately.
As someone who has experienced double failure twice in my lifetime, I seriously recommend doing backups.
The problem is that the only serious backup solution is another HDD for this size. A robot array for tapes or worm drives is probably out of budget.
I’m not sure if I qualify as a ‘larger local hoster’ but I would go through your 20 TB and decide what really is important enough to backup in case the wheels fall off. Linux ISOs, those can be re-downloaded, although it would take a bit of time. The things that can’t be readily downloaded such as my music collection that I have been accumulating for decades, converted to flac, and meticulously tagged, can’t be re-downloaded. So that is one of my priorities to back up. Pictures, business documents, personal documents, can’t be re-downloaded, so that goes on the ‘must back up’ list…and so on. Just cull out what is and isn’t replaceable. I would bet that once you do that, your 20 TB will be a bit more slim, and you’re not trying to push 20TB up the pipe to a cloud backup.
I use BackBlaze’s Personal, unlimited tier for $99 USD per year, which is a pretty sweet deal. One thing about Backblaze to remember is that the drives being backed up must be physically connected to the PC doing the backup/uploading. I get around that because I have a hot swap bay on my main PC, but there are other methods and software that will masquerade your NAS or other as a physically connected drive.
Backup to 2nd nas.
Important stuff gets backed up to cloud storage. Whatever is cheapest.
In my case Synology c2 cloud was cheapest.
c2 seems expensive, I would go with hetzner storage box + restic
Similar to most responses, I backup whatever I created myself, not shared by someone or downloaded from somewhere. I care about pictures that I took, documents, financial records, etc, which don’t take up much space at all.
Backblaze offers unlimited data on a single computer, $99/year.
There might be some fine print that excludes your setup but might be worth investigating.
With another large NAS.
