Need reliable external HDDs to back up my data
January 8, 2015 5:17 AM Subscribe
After some research, I think I've found my ideal backup solution. It involves me:
For a while now I've wished to find an ideal solution to back up my files as my primary computer's HDD, which has tons of important stuff on it, is getting old. Instead of pretending that all will forever be fine, I think it's long due time that I back my shit the hell up before it's too late. I oft worry about losing the tons of important stuff that is stored on my HDD and know that once I do this I'll finally be able to have some peace of mind.
Though I currently use free cloud services—DropBox and Google Drive—for storing small stuff, I first looked into cloud services that allow you to store hundreds of GBs of data to but (unsurprisingly) couldn't find any free or dirt cheap ones. So I quickly lost interest in that as I'd rather not have to spend hundreds annually on such a service. Also, there's always the possibility that the cloud service could just vanish out of the blue.
Next I looked into RAID and was about to go through with that until I read that 'proper' hardware RAID setups require one to get a good RAID card and that good ones typically start at $600. Also, I read that RAID isn't the great backup solution that many make it out to be. After reading on the topic for some time I ended up losing interest in that too as it would be too expensive and probably not reliable enough for my liking.
After feeling discouraged for a while, it dawned on me that a couple external HDDs would probably do the trick as it would:
1. Require but a modest fee.
2. Would be reliable and if one of the two drives were to ever die (which would likely take a very long time to happen as the drives would only be in use when performing my weekly backup) I could get another external HDD to replace the broken one with.
3. I could encrypt the data I stored to them so if anyone were to steal them at least my data will be unreachable to the thieves who stole them.
Now the question is: which external HDDs are considered most reliable these days? I've been told that these days Samsung makes the most reliable SSDs and Hitachi / HGST makes the most reliable (spin-type) HDDs. But I've no clue who is purported to make the most reliable external HDDs. Any recommendations?
- Manually creating a Macrium Reflect encrypted image file of my data once a week
- Storing the encrypted image file to two external HDDs
- Placing the external HDDs in two different (safe) physical locations
For a while now I've wished to find an ideal solution to back up my files as my primary computer's HDD, which has tons of important stuff on it, is getting old. Instead of pretending that all will forever be fine, I think it's long due time that I back my shit the hell up before it's too late. I oft worry about losing the tons of important stuff that is stored on my HDD and know that once I do this I'll finally be able to have some peace of mind.
Though I currently use free cloud services—DropBox and Google Drive—for storing small stuff, I first looked into cloud services that allow you to store hundreds of GBs of data to but (unsurprisingly) couldn't find any free or dirt cheap ones. So I quickly lost interest in that as I'd rather not have to spend hundreds annually on such a service. Also, there's always the possibility that the cloud service could just vanish out of the blue.
Next I looked into RAID and was about to go through with that until I read that 'proper' hardware RAID setups require one to get a good RAID card and that good ones typically start at $600. Also, I read that RAID isn't the great backup solution that many make it out to be. After reading on the topic for some time I ended up losing interest in that too as it would be too expensive and probably not reliable enough for my liking.
After feeling discouraged for a while, it dawned on me that a couple external HDDs would probably do the trick as it would:
1. Require but a modest fee.
2. Would be reliable and if one of the two drives were to ever die (which would likely take a very long time to happen as the drives would only be in use when performing my weekly backup) I could get another external HDD to replace the broken one with.
3. I could encrypt the data I stored to them so if anyone were to steal them at least my data will be unreachable to the thieves who stole them.
Now the question is: which external HDDs are considered most reliable these days? I've been told that these days Samsung makes the most reliable SSDs and Hitachi / HGST makes the most reliable (spin-type) HDDs. But I've no clue who is purported to make the most reliable external HDDs. Any recommendations?
This blog post from Backblaze themselves might be helpful:
https://www.backblaze.com/blog/hard-drive-reliability-update-september-2014/
posted by aheckler at 6:10 AM on January 8, 2015
https://www.backblaze.com/blog/hard-drive-reliability-update-september-2014/
posted by aheckler at 6:10 AM on January 8, 2015
If you were going to do this correctly with external hard drives, you'd buy drives from different manufactures to make sure you didn't get drives from the same bad lot. You'd also have to test your backup drives regularly to know if they are still good or not which would give you the time to try and swap out a new backup drive.
At this point, this is where the cloud backup software solutions start to look attractive.
posted by mmascolino at 7:39 AM on January 8, 2015 [1 favorite]
At this point, this is where the cloud backup software solutions start to look attractive.
posted by mmascolino at 7:39 AM on January 8, 2015 [1 favorite]
I'm not sure there's any "most reliable" consumer external hard drive. I'd consider them more like disposable drives, something that's going to last 2–3 years and then need to be replaced. That's OK with a backup drive; you probably won't ever need to read the data anyway. Keep an eye on the SMART status to see any errors before they matter.
My favorite portable drive now is the Seagate Backup Plus Slim, mostly because it's the smallest drive available. 2.5" USB 3.0 is great because you don't need external power, although I suspect that also means it's going to be less than reliable than some larger 3.5" brick. More info on this choice on Wirecutter.
I think you'll find the discipline of rotating two external drives offsite is harder than you might think. You mentioned not wanting cloud-based backups because of cost for all your data. CrashPlan is $50/year for unlimited data, or about 1/4 the cost of two drives., and is worth every penny IMHO.
Seconding what Pipeski says: RAID is not backup.
posted by Nelson at 7:45 AM on January 8, 2015
My favorite portable drive now is the Seagate Backup Plus Slim, mostly because it's the smallest drive available. 2.5" USB 3.0 is great because you don't need external power, although I suspect that also means it's going to be less than reliable than some larger 3.5" brick. More info on this choice on Wirecutter.
I think you'll find the discipline of rotating two external drives offsite is harder than you might think. You mentioned not wanting cloud-based backups because of cost for all your data. CrashPlan is $50/year for unlimited data, or about 1/4 the cost of two drives., and is worth every penny IMHO.
Seconding what Pipeski says: RAID is not backup.
posted by Nelson at 7:45 AM on January 8, 2015
This is slightly tangential to the question, but I think it's an important warning. I would run a few Macrium images, and then check thoroughly to make sure everything got into the backup image. I once ran a Macrium backup on my boss's computer at work prior to wiping and reinstalling Windows. I'd never had any problems before, so I didn't check the backup. Well, Macrium somehow didn't include My Documents in the image it made, and that's where ALL my boss's important files were. I did not figure this out until after I'd formatted his hard drive, and you can just imagine the horror of initially finding that out. I did manage to recover all his files with I think Easeus's recovery software for formatted hard drives, but I was beyond lucky that that was possible.
In summary, I had a single bad experience with Macrium, but it was a big one, so I switched over to DriveImageXML, and I have not had a single issue whatsoever with DIXML. Just be careful.
As far as externals, I'd say buy affordably priced, high capacity Seagates or WDs, but buy many of them. I've actually found some of my best deals at Walmart for external drives. The key is not the quality of a particular manufacturer of drives, because there's no magic bullet for a failproof hard drive. The key is to develop a written strategy and schedule for backups that will work for you, and then rigorously adhere to it. Always make sure you have your backups on multiple HDDs, and if possible keep them in separate places. You may want three, so you can keep two at home and then frequently copy over the files to the other one in a different location.
You may also want to consider Dropbox as another form of backup for important files. For $10 a month you can get 1 terabyte of storage, and this you have could have every device you've ever owned explode simultaneously, and you'd still have the information on the server. Between that and the effortless synchronization, I highly recommend it. The major limitation will be your ISP's bandwidth limitations.
Oh, and one final piece of advice that I've learned from horrible personal experience. If you transport an external hard drive from one safe place to another, keep track of that hard drive 100% of the time. I had an experience where a former co-worker's hard drive needed professional recovery, so I pulled it out his computer, put it in an external enclosure, and put it in my car. I was going to take it to the recovery people in the morning, and I forgot to bring it inside that night. That very night, my car was robbed, and that drive disappeared forever. Turns out that information on that drive would have probably have been quite useful in defending the company against a lawsuit he filed against us, but it was gone forever.
The best backup advice I can give is to develop a good system, and be diligent and paranoid about it.
posted by KinoAndHermes at 8:12 AM on January 8, 2015
In summary, I had a single bad experience with Macrium, but it was a big one, so I switched over to DriveImageXML, and I have not had a single issue whatsoever with DIXML. Just be careful.
As far as externals, I'd say buy affordably priced, high capacity Seagates or WDs, but buy many of them. I've actually found some of my best deals at Walmart for external drives. The key is not the quality of a particular manufacturer of drives, because there's no magic bullet for a failproof hard drive. The key is to develop a written strategy and schedule for backups that will work for you, and then rigorously adhere to it. Always make sure you have your backups on multiple HDDs, and if possible keep them in separate places. You may want three, so you can keep two at home and then frequently copy over the files to the other one in a different location.
You may also want to consider Dropbox as another form of backup for important files. For $10 a month you can get 1 terabyte of storage, and this you have could have every device you've ever owned explode simultaneously, and you'd still have the information on the server. Between that and the effortless synchronization, I highly recommend it. The major limitation will be your ISP's bandwidth limitations.
Oh, and one final piece of advice that I've learned from horrible personal experience. If you transport an external hard drive from one safe place to another, keep track of that hard drive 100% of the time. I had an experience where a former co-worker's hard drive needed professional recovery, so I pulled it out his computer, put it in an external enclosure, and put it in my car. I was going to take it to the recovery people in the morning, and I forgot to bring it inside that night. That very night, my car was robbed, and that drive disappeared forever. Turns out that information on that drive would have probably have been quite useful in defending the company against a lawsuit he filed against us, but it was gone forever.
The best backup advice I can give is to develop a good system, and be diligent and paranoid about it.
posted by KinoAndHermes at 8:12 AM on January 8, 2015
The fact that you're making two identical backups means you don't need to care about which brand of drive is the most reliable.
There is a small but nonzero chance that any drive will go belly-up right when you don't want it to. Every currently-available drive, to the best of my knowledge, makes that chance so small that the kind of double-backup you propose makes it negligible, especially if your two backup drives are from different manufacturers.
Buy on price so you can afford more drives. Two drives is enough to make drive failure unimportant, but using more than two in strict rotation will protect you better against your own mistakes.
posted by flabdablet at 9:16 AM on January 8, 2015
There is a small but nonzero chance that any drive will go belly-up right when you don't want it to. Every currently-available drive, to the best of my knowledge, makes that chance so small that the kind of double-backup you propose makes it negligible, especially if your two backup drives are from different manufacturers.
Buy on price so you can afford more drives. Two drives is enough to make drive failure unimportant, but using more than two in strict rotation will protect you better against your own mistakes.
posted by flabdablet at 9:16 AM on January 8, 2015
I agree the cloud is superior in terms of automatic, hands-free, offsite backup. But what about a recent issue -- namely the CryptoWall virus, whose infected files would replicate on the cloud as well. Any way to mitigate that?
posted by LonnieK at 9:42 AM on January 8, 2015
posted by LonnieK at 9:42 AM on January 8, 2015
Depending upon how many hundreds of GBs of data you need to store, note that 100GB triple-layer writeable Blu-ray media seems to have come down around $10 each at this point.
posted by XMLicious at 10:07 AM on January 8, 2015
posted by XMLicious at 10:07 AM on January 8, 2015
Those of us who have long and bitter experience in these matters - including several people upthread already - will all tell you the same things:
- Hard disks die. It's only a question of when, not if.
- RAID is not a backup.
- One backup is not sufficient.
- The reliability and effectiveness of a backup solution is inversely proportional to the amount of human intervention it requires.
For your proposal, why do you want to create an encrypted disk image once a week? That makes it likely that after you've forgotten to run a backup one week, been sick the next, and offloaded some really important photos the week after that, the disk will fail while you're hitting it hard to create the encrypted image with the un-backed up stuff.
We already have recommendations for Backblaze and Crashplan in the thread. I like Backblaze, and use Crashplan for my wife's machine. It came to under $3 per month when I bought 4 years in one shot, and well, I spend that on coffee every day. Crashplan versions files and keeps all of them in some rolling window, so it is a true archive, not just a mirror. (So that mitigates any Crypto-wall issues.)
If you don't trust online backup, think of ways to simplify your procedure. Use something like SuperDuper to clone your disk, alternating external disks every week. (The encryption step seems like pointless hassle - what's it protecting, really?) Or learn about rsync from the command line. Get a NAS (look at Synology), or share a NAS with a friend - offsite backup!
Doing it manually, you do have to stay on the ball, day after day after day - otherwise, Murphy promises that the disk will fail at the moment that you're most overextended and have the most to lose. But for all that, any backup - anything - is better than no backup.
(That's why TimeMachine was such a big deal to Mac users, and more importantly, to people who care for other people's Macs. For all its flaws, it would run in the background and just keep something backed up. So many averted tears.)
posted by RedOrGreen at 10:24 AM on January 8, 2015
- Hard disks die. It's only a question of when, not if.
- RAID is not a backup.
- One backup is not sufficient.
- The reliability and effectiveness of a backup solution is inversely proportional to the amount of human intervention it requires.
For your proposal, why do you want to create an encrypted disk image once a week? That makes it likely that after you've forgotten to run a backup one week, been sick the next, and offloaded some really important photos the week after that, the disk will fail while you're hitting it hard to create the encrypted image with the un-backed up stuff.
We already have recommendations for Backblaze and Crashplan in the thread. I like Backblaze, and use Crashplan for my wife's machine. It came to under $3 per month when I bought 4 years in one shot, and well, I spend that on coffee every day. Crashplan versions files and keeps all of them in some rolling window, so it is a true archive, not just a mirror. (So that mitigates any Crypto-wall issues.)
If you don't trust online backup, think of ways to simplify your procedure. Use something like SuperDuper to clone your disk, alternating external disks every week. (The encryption step seems like pointless hassle - what's it protecting, really?) Or learn about rsync from the command line. Get a NAS (look at Synology), or share a NAS with a friend - offsite backup!
Doing it manually, you do have to stay on the ball, day after day after day - otherwise, Murphy promises that the disk will fail at the moment that you're most overextended and have the most to lose. But for all that, any backup - anything - is better than no backup.
(That's why TimeMachine was such a big deal to Mac users, and more importantly, to people who care for other people's Macs. For all its flaws, it would run in the background and just keep something backed up. So many averted tears.)
posted by RedOrGreen at 10:24 AM on January 8, 2015
AOMEI's Backupper is free, and good. I recommend it. That said your solution is untenable.
External drives are ALL unreliable. 90% of the data recovery work I do is from external drives, although Seagate 3.5" and WD 2.5" seem to be the most unreliable around here, ALTHOUGH my sample set is small enough to be within anyone's margin of error.
Simply put, you need your data to exist in 3 separate places. You need it to be incremental and you need file versioning, I'm assuming, because most people do. With that in mind, just flipping buy Backblaze or Crashplan. No need to backup the disk images, just a master image of the OS and then all the requisite files. You're looking at $60-125/year, which is less than you'll be paying for a good second external with no worries about remembering to swap the externals every day. Barf.
Also, get a network attached external for the love of pete, so you can automate the backups to it w/o having the device attached to the computer in question.
posted by TomMelee at 11:45 AM on January 8, 2015
External drives are ALL unreliable. 90% of the data recovery work I do is from external drives, although Seagate 3.5" and WD 2.5" seem to be the most unreliable around here, ALTHOUGH my sample set is small enough to be within anyone's margin of error.
Simply put, you need your data to exist in 3 separate places. You need it to be incremental and you need file versioning, I'm assuming, because most people do. With that in mind, just flipping buy Backblaze or Crashplan. No need to backup the disk images, just a master image of the OS and then all the requisite files. You're looking at $60-125/year, which is less than you'll be paying for a good second external with no worries about remembering to swap the externals every day. Barf.
Also, get a network attached external for the love of pete, so you can automate the backups to it w/o having the device attached to the computer in question.
posted by TomMelee at 11:45 AM on January 8, 2015
Nelson: I'm not sure there's any "most reliable" consumer external hard drive.That's silly. Are all cars equally reliable? As mmascolino pointed out, solid research shows the most reliable is the Hitachi DeskStar brand. They cost about twice as much, roughly, as their shitty competition, and I'll never buy a different brand until research shows something else is rivalling them.
flabdablet: The fact that you're making two identical backups means you don't need to care about which brand of drive is the most reliable.... assuming you are filthy rich (so replacing drives 6x as often is trivial) and believe lightning can't strike twice (coughApollo 13cough).
posted by IAmBroom at 12:10 PM on January 8, 2015
what about a recent issue -- namely the CryptoWall virus, whose infected files would replicate on the cloud
I'm not familiar specifically with CryptoWall; is it one of those malware packages that encrypts all your files and then demands you pay a ransom to get them back? The solution for that problem, or for any other file corruption (like "I saved my novel with a chapter accidentally deleted") is versioned backups so you can restore the file from a checkpoint before the corruption happened. CrashPlan provides that; I have daily backups of some files going back a year or more, although it's not 100% complete and a bit awkward to access. TimeMachine on the Mac also does this well, as do most serious backup solutions. I'm not certain about Macrium; if it only is making a complete disk image copy then you have to hope you kept a copy from before the corruption.
As for disk reliability, I should have spoken more precisely. Some drives are more reliable than others, but I would not trust any of them to provide a "most reliable" experience. They all suck, and which manufacturer is best seems to change every few months. Your best bet is to buy the drive with the best warranty. The Seagate Slim drive I recommended is not good by this metric: 2 years compared to 3 years from most other manufacturers.
Boy do those Seagate Barracuda 7200rpm desktop drives look crappy in the Backblaze paper. But I don't trust that necessarily applies to all Seagate drives (although a shorter warranty sure speaks volumes). The huge Google study on drive reliability also notes that reliability varies significantly by manufacturer, but they do not share any manufacturer-specific data in the paper.
posted by Nelson at 12:33 PM on January 8, 2015 [1 favorite]
I'm not familiar specifically with CryptoWall; is it one of those malware packages that encrypts all your files and then demands you pay a ransom to get them back? The solution for that problem, or for any other file corruption (like "I saved my novel with a chapter accidentally deleted") is versioned backups so you can restore the file from a checkpoint before the corruption happened. CrashPlan provides that; I have daily backups of some files going back a year or more, although it's not 100% complete and a bit awkward to access. TimeMachine on the Mac also does this well, as do most serious backup solutions. I'm not certain about Macrium; if it only is making a complete disk image copy then you have to hope you kept a copy from before the corruption.
As for disk reliability, I should have spoken more precisely. Some drives are more reliable than others, but I would not trust any of them to provide a "most reliable" experience. They all suck, and which manufacturer is best seems to change every few months. Your best bet is to buy the drive with the best warranty. The Seagate Slim drive I recommended is not good by this metric: 2 years compared to 3 years from most other manufacturers.
Boy do those Seagate Barracuda 7200rpm desktop drives look crappy in the Backblaze paper. But I don't trust that necessarily applies to all Seagate drives (although a shorter warranty sure speaks volumes). The huge Google study on drive reliability also notes that reliability varies significantly by manufacturer, but they do not share any manufacturer-specific data in the paper.
posted by Nelson at 12:33 PM on January 8, 2015 [1 favorite]
There is no way I would install a hitachi hard drive at any point ever. Sure there's a reason why Dell puts them in so many machines...its because they last exactly as long as the warranty and then go kerploof. Most drives that fail are as simple as MFT's getting written wrong, with data 100% recoverable. It's an OS issue, not a hardware issue. As far as hardware issues, about 75% of the time, it's not actual physical damage, but a firmware bug that causes the drive to lock/fail. 100% recoverable with the right equipment to force a new firmware and reboot. The rest of the time it's heads or platters or armature or some or all, marginal data recovery at a VERY high cost. IME, with Hitachi's, it's never firmware. It's always the head deciding to eat the platters in a cloud of magic and incredibly toxic dust. YMMV, of course.
BUT--
Cryptowall and the variants are EXCEPTIONALLY easy to prevent.
First, use something like CryptoPrevent or group policy or any of a gazillion other rules to prevent executable files running in the temp folder.
Second, turn on shadow copy, which exists but is off by default in Win 7 and 8 and 8.1. "Previous versions." Easy. Flipping. Peasy.
Third, make sure you have file revisioning on your backup solution. Then if you erase/encrypt/whatever your good copy, you've got two previous versions. Also great when you accidentally delete or overwrite your report/whatever. This is why straight, 1:1 mirror backups are a BAD IDEA. Even google drive/dropbox/box/skydrive/whatever will hold revisions if you use sync software (any of several) to access them. Dropbox lets you turn it on for a fee as well, but it's unnecessarily expensive IMO.
posted by TomMelee at 1:05 PM on January 8, 2015
BUT--
Cryptowall and the variants are EXCEPTIONALLY easy to prevent.
First, use something like CryptoPrevent or group policy or any of a gazillion other rules to prevent executable files running in the temp folder.
Second, turn on shadow copy, which exists but is off by default in Win 7 and 8 and 8.1. "Previous versions." Easy. Flipping. Peasy.
Third, make sure you have file revisioning on your backup solution. Then if you erase/encrypt/whatever your good copy, you've got two previous versions. Also great when you accidentally delete or overwrite your report/whatever. This is why straight, 1:1 mirror backups are a BAD IDEA. Even google drive/dropbox/box/skydrive/whatever will hold revisions if you use sync software (any of several) to access them. Dropbox lets you turn it on for a fee as well, but it's unnecessarily expensive IMO.
posted by TomMelee at 1:05 PM on January 8, 2015
I can't really help you on the external hard drives since I think they are all, generally, much of a muchness. For every person who says that brand X is awesome, you'll find 20 people who will have had terrible experiences. I've used Seagate and Western Digital without problems.
However I think you should look at Crashplan. I have the free version running on several computers owned by the family and they all automatically backup the contents and mirror that to all the others.
As a result, if anything happens then I know I've got several offsite locations containing my backup. To speed things up you'll probably want to do the first backup onto a hard drive and then physically take it to the other computers. This post over at superuser.com explains how to do that.
Best of all it's free.
posted by mr_silver at 1:59 PM on January 8, 2015
However I think you should look at Crashplan. I have the free version running on several computers owned by the family and they all automatically backup the contents and mirror that to all the others.
As a result, if anything happens then I know I've got several offsite locations containing my backup. To speed things up you'll probably want to do the first backup onto a hard drive and then physically take it to the other computers. This post over at superuser.com explains how to do that.
Best of all it's free.
posted by mr_silver at 1:59 PM on January 8, 2015
IME, with Hitachi's, it's never firmware. It's always the head deciding to eat the platters in a cloud of magic and incredibly toxic dust. YMMV, of course.
That's also been my experience with Hitachi drives. Unconvinced about the toxicity thing though.
Worst failure I ever saw on a Seagate drive involved the drive becoming completely inaccessible on every bad sector, requiring actual power cycling to bring it back online. There were several hundred bad sectors. Took a while to get all the retrievable data back from that one.
posted by flabdablet at 7:41 PM on January 8, 2015
That's also been my experience with Hitachi drives. Unconvinced about the toxicity thing though.
Worst failure I ever saw on a Seagate drive involved the drive becoming completely inaccessible on every bad sector, requiring actual power cycling to bring it back online. There were several hundred bad sectors. Took a while to get all the retrievable data back from that one.
posted by flabdablet at 7:41 PM on January 8, 2015
This thread is closed to new comments.
RAID isn't a backup. It's just some insurance against hard disk failure. Assuming you have two disks in a RAID1 array, you have two copies of everything, and failure of a single hard disk won't inconvenience you at all - it'll just ask you to replace the defective disk, then sort itself out. You don't need 'a good RAID card' - software RAID, or a cheap NAS like one of the basic Buffalo ones is just fine for most purposes.
Cloud backups are well worth the money. Backblaze (the one I use) is $5 a month, or $50 a year. A couple of external HDDs will cost significantly more than that. Plus you have to worry about carrying your hard disks back and forth to wherever you're keeping them when you're not backing stuff up. A cloud backup is set-up-and-forget. Most cloud backup services also encrypt your data in transit and in storage, and only you have the key.
My own solution is to back up my local files (from two different PCs) every night to a RAID1 NAS, which pretty much ensures that I won't lose data due to a HDD failure, either in a PC or the NAS. And then I have the Backblaze account, which is my offsite backup. I could just rely on the Backblaze backup, but downloading all of my files is going to be time-consuming if I ever need to do it. I don't worry about the company 'vanishing out of the blue' - the odds of that event coinciding exactly with my house burning down are vanishingly small.
Nobody really makes 'reliable external HDDs'. They're just normal drives in an enclosure (or plugged into an adapter or a caddy). This recent thread has some opinions on reliability of drives.
posted by pipeski at 5:57 AM on January 8, 2015 [2 favorites]