How to deal with this backup situation involving RAID1 HD.
January 31, 2018 6:15 PM Subscribe
I currently have no full hard drive backup. I obviously need one. I have a RAID1 with a partial back-up. Will this plan work: 1. Remove one HD from raid to preserve the partial backup. 2. Delete all files on the remaining HD and now just copy (using windows explorer) all the files onto that HD for a quick backup. 3. Get backup software. 4. Swap the HDs (take out the one with my file copied backup and put in the other one. 5. Now delete the partial backup and use backup software to create a proper encrypted backup to this HD. 6. Put the second HD back in and count on the RAID to turn it into a mirror of the backup created with software.
So I've been trying to backup with cloudberry and started two weeks before my Crashplan died, but it's error after error and I'm not impressed by their software, their tech support, or their general level of organization as a company (one definitely gets the sense the left hand doesn't know there even is a right hand over there). Crashplan has now ended for me. Cloudberry won't finish backing up. It's about half way through backing up about 1.2 terrabytes of data.
So I'm going to give up on Cloudberry. But now I need an immediate backup and a longterm solution. The above was suggested by a friend. But I'm not convinced that's how the RAID1 HD works. I think the mirrors need to be written at the same time: You can't just fill one HD and then insert another one and expect it to copy over. Am I wrong? Do I need to format the HD that will have the mirror copies on to it? Will I even be able to write to one HD if there's only one HD in the case or will that not work because it will want to mirror and won't be able to?
I"m backing up locally now with the RAID set up as a network drive. Once backup is complete I will move the HDs to an off-site location and backup over the internet.
I am not knowledgeable in these things, at all. A friend helped me get setup so I coudl use Cloudberry to backup to the RAID but for reasons isn't able to do detailed disaster relief support right now. In other words, don't talk to me like I know anything. Talk to me like I know nothing.
Also, what backup software do you recommend? I"m thinking of going to Arq, but it doesn't do real time backup and I like real-time backup.
So I've been trying to backup with cloudberry and started two weeks before my Crashplan died, but it's error after error and I'm not impressed by their software, their tech support, or their general level of organization as a company (one definitely gets the sense the left hand doesn't know there even is a right hand over there). Crashplan has now ended for me. Cloudberry won't finish backing up. It's about half way through backing up about 1.2 terrabytes of data.
So I'm going to give up on Cloudberry. But now I need an immediate backup and a longterm solution. The above was suggested by a friend. But I'm not convinced that's how the RAID1 HD works. I think the mirrors need to be written at the same time: You can't just fill one HD and then insert another one and expect it to copy over. Am I wrong? Do I need to format the HD that will have the mirror copies on to it? Will I even be able to write to one HD if there's only one HD in the case or will that not work because it will want to mirror and won't be able to?
I"m backing up locally now with the RAID set up as a network drive. Once backup is complete I will move the HDs to an off-site location and backup over the internet.
I am not knowledgeable in these things, at all. A friend helped me get setup so I coudl use Cloudberry to backup to the RAID but for reasons isn't able to do detailed disaster relief support right now. In other words, don't talk to me like I know anything. Talk to me like I know nothing.
Also, what backup software do you recommend? I"m thinking of going to Arq, but it doesn't do real time backup and I like real-time backup.
Response by poster: The small business offering would cost me $40/month which is way more than I want to spend and it did expire so I don't think I could transition anymore, anyway. I t think Backblaze deletes your old files? There was something about backblaze I didn't like. But I'm curious why you recommend backblaze though you use arq. Is it because of the continuous backup?
posted by If only I had a penguin... at 9:29 PM on January 31, 2018
posted by If only I had a penguin... at 9:29 PM on January 31, 2018
In theory, the plan could work; when you remove the second drive from the RAID array, the array will continue to operate in degraded mode with a single drive, and you can write to it.
But why? I don't understand the rationale here at all. Why not just create a new backup on the RAID, without all this shuffling? Or, why not just spend $50-$100 on an additional 2TB drive and do your temporary backup to it, and don't break the RAID at all? How much is your data worth?
One thing to be careful about with breaking and rebuilding RAID arrays: Sometimes you have bad blocks that quietly lurk until you access them during the rebuild, and then, partway through the rebuild, when they're accessed for the first time in months, they cause a failure. You always want to have a spare on hand in case that happens.
Another thing that might make this plan fail: Depending on the precise way that the RAID information is stored on the disk, it might be quite difficult to break the RAID in a way that actually breaks it. Even reformatting the drive might not take it out of the RAID, since partition information is written to the start of the disk and RAID information is sometimes written to the end of the disk and sometimes written to places near the front that reformatting doesn't touch. When you finally put the two disks back in the enclosure together, I would not be willing to guess which of the disks it's going to recognize as the good disk which it should rewrite the other disk from unless I had spend some quality time with dd. I would be extremely nervous about doing this, and I've done all sorts of things with dodgy RAID setups in order to save data.
Unless you want to learn enough about data recovery to be fully confident doing this, I would strongly recommend buying another hard drive. It will make your life much simpler and dramatically reduce your risk of completely losing your backup.
posted by clawsoon at 10:06 PM on January 31, 2018 [1 favorite]
But why? I don't understand the rationale here at all. Why not just create a new backup on the RAID, without all this shuffling? Or, why not just spend $50-$100 on an additional 2TB drive and do your temporary backup to it, and don't break the RAID at all? How much is your data worth?
One thing to be careful about with breaking and rebuilding RAID arrays: Sometimes you have bad blocks that quietly lurk until you access them during the rebuild, and then, partway through the rebuild, when they're accessed for the first time in months, they cause a failure. You always want to have a spare on hand in case that happens.
Another thing that might make this plan fail: Depending on the precise way that the RAID information is stored on the disk, it might be quite difficult to break the RAID in a way that actually breaks it. Even reformatting the drive might not take it out of the RAID, since partition information is written to the start of the disk and RAID information is sometimes written to the end of the disk and sometimes written to places near the front that reformatting doesn't touch. When you finally put the two disks back in the enclosure together, I would not be willing to guess which of the disks it's going to recognize as the good disk which it should rewrite the other disk from unless I had spend some quality time with dd. I would be extremely nervous about doing this, and I've done all sorts of things with dodgy RAID setups in order to save data.
Unless you want to learn enough about data recovery to be fully confident doing this, I would strongly recommend buying another hard drive. It will make your life much simpler and dramatically reduce your risk of completely losing your backup.
posted by clawsoon at 10:06 PM on January 31, 2018 [1 favorite]
I also use Arq. It works and is cheap, but you have to set it up.
I also do not understand why you don't just buy a big external drive and make a backup.
Maybe I'm not following something here.
posted by bongo_x at 1:06 AM on February 1, 2018
I also do not understand why you don't just buy a big external drive and make a backup.
Maybe I'm not following something here.
posted by bongo_x at 1:06 AM on February 1, 2018
The reason I thought Backblaze instead of Arq/B2 is simply that Arq is a bit unfriendly and annoying to set up, and you professed ignorance. It's easy to use if you know the technology. If not, confusing.
posted by doomsey at 5:53 AM on February 1, 2018
posted by doomsey at 5:53 AM on February 1, 2018
That is how RAID 1 should work, but don't go pulling drives out of a working system. You might create problems for yourself. I'm with the other folks that suggest that you buy a 2TB external and use that as a temporary backup.
I've used BackBlaze for years. It's solid. They keep files for thirty days after you delete/modify them on your machine.
posted by gregr at 6:58 AM on February 1, 2018
I've used BackBlaze for years. It's solid. They keep files for thirty days after you delete/modify them on your machine.
posted by gregr at 6:58 AM on February 1, 2018
Response by poster: I just bought these two big drives to put in the RAID and I'm on post-top-up maternity leave, so I'm scraping pennies right now. The reason for all the swapping would be to preserve the partial back-up until I have a full-back up and then preserve the make-shift backup until I have a proper backup. But this reminds me, I have an old 1TB HD lying around. That's not big enough for a full back-up, but maybe if I do everything but my pics (which are backed up to Google Photos anyway) it will fit for the make-shift backup then I can just reformat the RAID and backup with Arq.
Can I set up Arq to backup "on schedule" every hour to come relatively close to real-time backup?
OK, warning taken on the usability of arq. I will look into that more closely. But 30 days to keep a file doesn't seem very long to me.
posted by If only I had a penguin... at 7:22 AM on February 1, 2018 [1 favorite]
Can I set up Arq to backup "on schedule" every hour to come relatively close to real-time backup?
OK, warning taken on the usability of arq. I will look into that more closely. But 30 days to keep a file doesn't seem very long to me.
posted by If only I had a penguin... at 7:22 AM on February 1, 2018 [1 favorite]
Came in here to make clawson's point - a raid rebuild is dangerous because you have 0 redundancy during the rebuild. I always sweat when I have a raid5 or raid10 rebuild underway, and I did once lose a 2nd disk during the middle of a raid6 rebuild (yay for 2 disk redundancy), and my boss "back inthe day" did lose a 2nd disk in the middle of a raid5 rebuild.
Sure, monthly patrol reads should limit the likihood of coming across issues during a rebuild, but don't expose yourself to that risk without a strong need.
posted by nobeagle at 7:50 AM on February 1, 2018
Sure, monthly patrol reads should limit the likihood of coming across issues during a rebuild, but don't expose yourself to that risk without a strong need.
posted by nobeagle at 7:50 AM on February 1, 2018
Arq can do hourly, yes, and I have it configured that way. It'll also do automatic thinning so it doesn't keep many many copies of duplicate files around.
posted by doomsey at 8:49 AM on February 1, 2018 [1 favorite]
posted by doomsey at 8:49 AM on February 1, 2018 [1 favorite]
This thread is closed to new comments.
I switched to Arq post-Crashplan. It seems to work well enough. I have it backing up to a local file server, and Backblaze B2. Yes, it doesn't have continuous backup, but it's a good solution and essentially replaces all of Crashplan.
You can absolutely split and rejoin RAID1s. Splitting a RAID1 is simple - remove the drive, it gets marked as failed. RAID1s work fine with a failed disk, so nothing bad happens. Rejoining is equally simple - replace the drive, and tell the RAID software you've done so. It'll "rebuild," which means recopy all the data back to that drive from the "good" drive. Once you split the drive out, you may need to use a different computer to reformat the drive before reattaching it to prevent it from automatically being pulled back into the RAID.
Assuming transitioning with Crashplan isn't an option, I think I'd recommend just doing the copy procedure you described as an interim backup, and then getting Backblaze (the regular consumer software) installed and running. Once Backblaze finishes its backup, you can put your RAID1 back together.
posted by doomsey at 9:10 PM on January 31, 2018