I am trying to choose the right backup solution but I have questions.
January 27, 2015 1:44 AM Subscribe
(network attached storage, raid)
I have about 4TB of images backed up on a external hard drives and it’s freaking me out. These are high-res images I want to keep backups of. I would prefer having redundancy with the files being backed up onto two drives. I use macs exclusively and the backup solution should not be connected to the internet. I want them to be on my network because one of my macs only has FW800 and USB 2, not thunderbolt or USB 3.
One solution I looked at but dismissed was Drobo. I read an interesting blog post by someone complaining that the file format was proprietary and that once his Drobo broke the files were essentially locked until he purchased another. That is not what I want. I want my files backed up uncompressed and unchanged. A JPEG is a JPEG is a JPEG.
I have seen the Synology website and like the idea of having a 2-bay network attached storage device that I can connect to my Netgear Nighthawk router via the ethernet port. My suspicion is that the drives would show up on my mac desktop like another hard drive. Is this how it would work? Will the files be backed up from one hard drive to another automatically? Is this complicated to set up and administrate? Also: which Synology device would be the best for me to choose?
I have also read about a recent cryptolocker-type attack that locks files on devices that don’t have up-to-date firmware. Okay, I can update that if I get a device but how easy or difficult would it be to tell a router-connected device to never connect to any device outside my WiFi coverage area (never accept incoming connections from the web)?
Thanks for reading this. I find this topic rather intimidating.
I have about 4TB of images backed up on a external hard drives and it’s freaking me out. These are high-res images I want to keep backups of. I would prefer having redundancy with the files being backed up onto two drives. I use macs exclusively and the backup solution should not be connected to the internet. I want them to be on my network because one of my macs only has FW800 and USB 2, not thunderbolt or USB 3.
One solution I looked at but dismissed was Drobo. I read an interesting blog post by someone complaining that the file format was proprietary and that once his Drobo broke the files were essentially locked until he purchased another. That is not what I want. I want my files backed up uncompressed and unchanged. A JPEG is a JPEG is a JPEG.
I have seen the Synology website and like the idea of having a 2-bay network attached storage device that I can connect to my Netgear Nighthawk router via the ethernet port. My suspicion is that the drives would show up on my mac desktop like another hard drive. Is this how it would work? Will the files be backed up from one hard drive to another automatically? Is this complicated to set up and administrate? Also: which Synology device would be the best for me to choose?
I have also read about a recent cryptolocker-type attack that locks files on devices that don’t have up-to-date firmware. Okay, I can update that if I get a device but how easy or difficult would it be to tell a router-connected device to never connect to any device outside my WiFi coverage area (never accept incoming connections from the web)?
Thanks for reading this. I find this topic rather intimidating.
You can mail hard drives to Amazon and put all your files in an S3 bucket in the cloud. To reduce costs, you can use their Glacier service. The tradeoff for lower costs is greater retrieval time, which is why I mention it despite your qualification about no "internet-connected" — you're basically putting a backup into "cold storage".
Failing that, you might indicate what kind of Mac that you have. Some older Macs (like a MacBook Pro) have ExpressCard/34 slots, which can support eSATA.
You could set up a pair of eSATA- or even FW800-connected drives and use Disk Utility to set up RAID 1 mirroring. The two drives show up as one volume — copying a file to this volume writes it to both disks. If one disk has a physical failure, then the other drive can take over.
RAID 1 isn't a backup. If you delete a file accidentally, it is removed from both disks. But redundancy provides some measure of protection from early hard drive failure.
posted by a lungful of dragon at 2:05 AM on January 27, 2015 [2 favorites]
Failing that, you might indicate what kind of Mac that you have. Some older Macs (like a MacBook Pro) have ExpressCard/34 slots, which can support eSATA.
You could set up a pair of eSATA- or even FW800-connected drives and use Disk Utility to set up RAID 1 mirroring. The two drives show up as one volume — copying a file to this volume writes it to both disks. If one disk has a physical failure, then the other drive can take over.
RAID 1 isn't a backup. If you delete a file accidentally, it is removed from both disks. But redundancy provides some measure of protection from early hard drive failure.
posted by a lungful of dragon at 2:05 AM on January 27, 2015 [2 favorites]
A NAS is a simple and wonderful thing. It plugs into the network, at which point it behaves much like a shared folder (or multiple shared folders) on another machine (which is more or less what it is). You won't see it as a hard drive, because it's not attached to the mainboard of your computer in the way that a USB/Thunderbolt/etc device would be. But a network share works much the same in terms of being accessible from your machine. Almost any NAS will work with almost any router. A Synology box is a fairly advanced kind of NAS - there are many much simpler (and less expensive) options that just work as a NAS, without the many bells and whistles like media servers and FTP and so on that you get with a Synology box.
Usually you can administer the NAS through a web browser, although some manufacturers supply software so that you can administer the device directly from your computer, which may make life easier if you're not familiar with basic home networking. The interface (browser or installed software) allows you to create network shares on the device, set up users and permissions, and choose the type of RAID you want to have.
RAID 1 is the sensible way to go if you're looking to keep your data safe from drive failure. Use the NAS as a local backup, by all means, but don't just put files on it and assume that the two copies of your data due to the RAID 1 is a backup - it isn't. As a lungful of dragon says, deletion removes both copies. So you'll probably want to look at an additional cloud backup service of some kind for total peace of mind.
posted by pipeski at 2:12 AM on January 27, 2015
Usually you can administer the NAS through a web browser, although some manufacturers supply software so that you can administer the device directly from your computer, which may make life easier if you're not familiar with basic home networking. The interface (browser or installed software) allows you to create network shares on the device, set up users and permissions, and choose the type of RAID you want to have.
RAID 1 is the sensible way to go if you're looking to keep your data safe from drive failure. Use the NAS as a local backup, by all means, but don't just put files on it and assume that the two copies of your data due to the RAID 1 is a backup - it isn't. As a lungful of dragon says, deletion removes both copies. So you'll probably want to look at an additional cloud backup service of some kind for total peace of mind.
posted by pipeski at 2:12 AM on January 27, 2015
Best answer: I would advise against raid if you care about protecting against actual hardware failure. Synologies DO break. I'm in IT, and the place i work at uses a bunch of these for minor on-site backup stuff because they're cheap and they work. Mostly we've had dead drives, but i have had a couple die. They only really break if you don't clean the fans enough, or the fans go bad. Almost every time i thought one died the firmware just corrupted when the drive died or something and i was able to perform CPR with a 30/30/30.
But, if you have raid, and it decided to shit its pants, then you're hosed. You just want to set an rsync script or use synlogy's utility("data replicator") to write out your files to BOTH drives.
Personally, if this was my rig, i'd buy the 4 drive one or even bigger and go raid 5... then get a backblaze account and back it all up online too. I know you said you weren't interested in an online solution, but i think "offsite and on different hardware" is a good idea if this stuff is important enough that you're willing to drop a bill or more on backing it up properly at your place.
My suspicion is that the drives would show up on my mac desktop like another hard drive. Is this how it would work?
yep. it'll show up under "shared" in finder, right under devices. It supports AFP, samba, NFS, all the good stuff. Anything that can access network shares will probably see it totally by default.
I have also read about a recent cryptolocker-type attack that locks files on devices that don’t have up-to-date firmware. Okay, I can update that if I get a device but how easy or difficult would it be to tell a router-connected device to never connect to any device outside my WiFi coverage area (never accept incoming connections from the web)?
this is in the main config of the synology when you set it up. you want to select that you have no interest in the internet features or cloud stuff. then once you get in to the main admin screen(lt basically looks like a crappy overwrought mid-2000s linux livecd) go to updates, and check autoupdate.
They're good about patching things very quickly, but people are bad about installing updates. That hack worked like any other, it depended on people not patching a 0day. Enable autopatches and you're square.
Disabled UPNP on your router(WHICH YOU SHOULD HAVE OFF ANYWAYS ITS EVIL) and don't forward any ports. you're fine.
QNAP makes ok NAS stuff too. Avoid drobo, and avoid buffalo or any of the cheapy stuff. Synlogy hardware is very, very well made and everything about it generally shouted quality and good design at me. I've had i think 3 generations of their stuff around now, and every generation seems to be a solid improvement in everything over the last. Better cooling, smarter thermal design, nicer... just everything hardware wise always feels better.
Looking around, this seems to be the most basic model that will do what i was discussing.
Also note that you don't need 4 drives just because it has 4 slots. It will run fine with 2, or 3, or whatever.
Buy real NAS/commercial grade drives. Buy HGST/hitachi. this is why. 3tb hitachis look to be the most reliable drive on the market right now. I'm about to order some myself for a work project.
I can't stress enough though, do not make this your only backup solution. It's fine if this is a primary. but keep writing out the external drives once a month and storing them at your moms house or whatever at the very least. And test them once a month. SMART, copy some files, file system check. Do a full restore every other month.
And that's only if you absolutely refuse to store these files online/on an offsite system out of your control(which is fine, and can have totally legit reasons including "i don't want to"). I really recommend that NAS setup and backblaze.
Oh, as a closing note on preview, the synology OS is smart enough and capable enough that you can plug your external drives in to the USB ports on the unit, and have it clone/update the data on them if you configure it correctly and script it to that at X time. This is pretty cool, if you get it set up correctly. You can set it to try and write to all your external drives, and it'll just succeed on whichever one is connected. Then you log in, unmount it, and take it to whatever offsite location you're storing it at. Good times.
posted by emptythought at 3:05 AM on January 27, 2015 [6 favorites]
But, if you have raid, and it decided to shit its pants, then you're hosed. You just want to set an rsync script or use synlogy's utility("data replicator") to write out your files to BOTH drives.
Personally, if this was my rig, i'd buy the 4 drive one or even bigger and go raid 5... then get a backblaze account and back it all up online too. I know you said you weren't interested in an online solution, but i think "offsite and on different hardware" is a good idea if this stuff is important enough that you're willing to drop a bill or more on backing it up properly at your place.
My suspicion is that the drives would show up on my mac desktop like another hard drive. Is this how it would work?
yep. it'll show up under "shared" in finder, right under devices. It supports AFP, samba, NFS, all the good stuff. Anything that can access network shares will probably see it totally by default.
I have also read about a recent cryptolocker-type attack that locks files on devices that don’t have up-to-date firmware. Okay, I can update that if I get a device but how easy or difficult would it be to tell a router-connected device to never connect to any device outside my WiFi coverage area (never accept incoming connections from the web)?
this is in the main config of the synology when you set it up. you want to select that you have no interest in the internet features or cloud stuff. then once you get in to the main admin screen(lt basically looks like a crappy overwrought mid-2000s linux livecd) go to updates, and check autoupdate.
They're good about patching things very quickly, but people are bad about installing updates. That hack worked like any other, it depended on people not patching a 0day. Enable autopatches and you're square.
Disabled UPNP on your router(WHICH YOU SHOULD HAVE OFF ANYWAYS ITS EVIL) and don't forward any ports. you're fine.
QNAP makes ok NAS stuff too. Avoid drobo, and avoid buffalo or any of the cheapy stuff. Synlogy hardware is very, very well made and everything about it generally shouted quality and good design at me. I've had i think 3 generations of their stuff around now, and every generation seems to be a solid improvement in everything over the last. Better cooling, smarter thermal design, nicer... just everything hardware wise always feels better.
Looking around, this seems to be the most basic model that will do what i was discussing.
Also note that you don't need 4 drives just because it has 4 slots. It will run fine with 2, or 3, or whatever.
Buy real NAS/commercial grade drives. Buy HGST/hitachi. this is why. 3tb hitachis look to be the most reliable drive on the market right now. I'm about to order some myself for a work project.
I can't stress enough though, do not make this your only backup solution. It's fine if this is a primary. but keep writing out the external drives once a month and storing them at your moms house or whatever at the very least. And test them once a month. SMART, copy some files, file system check. Do a full restore every other month.
And that's only if you absolutely refuse to store these files online/on an offsite system out of your control(which is fine, and can have totally legit reasons including "i don't want to"). I really recommend that NAS setup and backblaze.
Oh, as a closing note on preview, the synology OS is smart enough and capable enough that you can plug your external drives in to the USB ports on the unit, and have it clone/update the data on them if you configure it correctly and script it to that at X time. This is pretty cool, if you get it set up correctly. You can set it to try and write to all your external drives, and it'll just succeed on whichever one is connected. Then you log in, unmount it, and take it to whatever offsite location you're storing it at. Good times.
posted by emptythought at 3:05 AM on January 27, 2015 [6 favorites]
To build on emptythought's comment - I would whole-heartedly recommend Backblaze. If you are concerned about privacy and/or security, you can encrypt your data before upload with a key only you know, baked right in to the product. I keep my photographs and other documents on an external hard disk, which is then backed up by Backblaze.
posted by NordyneDefenceDynamics at 3:54 AM on January 27, 2015 [1 favorite]
posted by NordyneDefenceDynamics at 3:54 AM on January 27, 2015 [1 favorite]
Response by poster: I'm in China, so any online backup as well as drives in the mail solutions are not gonna happen as our internet connections here are like sucking a cake through a straw. I have a 2010 MBP 17" with firewire 800 and usb 2 and expresscad and a brand new 15" mbp with thunderbolt and usb 3 but no firewire 800 or expresscard slot. there's a 27" retina imac scheduled to join the family next month.
I am completely willing to drop $1.000 on this if it enables me to have two physical copies on separate drives. this is something I want to solve once and for all and be able to take the drives with me if I ever move to another city, country, continent again. I do not want to use the cloud at all.
posted by krautland at 4:54 AM on January 27, 2015
I am completely willing to drop $1.000 on this if it enables me to have two physical copies on separate drives. this is something I want to solve once and for all and be able to take the drives with me if I ever move to another city, country, continent again. I do not want to use the cloud at all.
posted by krautland at 4:54 AM on January 27, 2015
If you're not using the cloud or any other form of transfer via internet, then your best bet is probably a separate external drive that you store offsite. This is going to make your backups pretty labour-intensive, because you're going to need to remember to physically carry the drive to and from your offsite location.
If portability is an issue and you still want a NAS (because you're backing up more than one machine), then look for the lightest portable option. That probably means not bothering with RAID and just having an onsite NAS + an offsite external drive.
posted by pipeski at 5:16 AM on January 27, 2015 [1 favorite]
If portability is an issue and you still want a NAS (because you're backing up more than one machine), then look for the lightest portable option. That probably means not bothering with RAID and just having an onsite NAS + an offsite external drive.
posted by pipeski at 5:16 AM on January 27, 2015 [1 favorite]
Response by poster: the external drive is the current solution with which I am not satisfied anymore.
posted by krautland at 5:55 AM on January 27, 2015
posted by krautland at 5:55 AM on January 27, 2015
That is not what I want. I want my files backed up uncompressed and unchanged. A JPEG is a JPEG is a JPEG.
It's not quite like that. The files aren't changed: what Drobo uses is a proprietary layout format. You could say the same about hardware RAID: if, say, an expensive LSI controller managing a RAID5 array breaks on a server, then you need to replace it with another expensive LSI controller.
Anyway, for backup purposes, a NAS is a fancy external drive. A better made and much more capable one, for sure, but within the context of backing stuff up, it's not massively different from a two-drive external. Having two copies sitting side by side in the same box protects you against a drive going bad, but doesn't provide redundancy against catastrophic hardware failure or theft or fire/flood/rampaging wildebeests.
So I'd agree with emptythought: get a NAS (4-bay gives you more room to grow than 2-bay), back up to two drives individually, and then get another big dumb external drive for something approximating offsite. Test your backups. As that Backblaze link notes, the jury's out right now on the long-term reliability of 6GB drives, but I'd be okay with one as the rearguard option. That gets you to something like jwz's approach.
posted by holgate at 8:35 AM on January 27, 2015 [1 favorite]
It's not quite like that. The files aren't changed: what Drobo uses is a proprietary layout format. You could say the same about hardware RAID: if, say, an expensive LSI controller managing a RAID5 array breaks on a server, then you need to replace it with another expensive LSI controller.
Anyway, for backup purposes, a NAS is a fancy external drive. A better made and much more capable one, for sure, but within the context of backing stuff up, it's not massively different from a two-drive external. Having two copies sitting side by side in the same box protects you against a drive going bad, but doesn't provide redundancy against catastrophic hardware failure or theft or fire/flood/rampaging wildebeests.
So I'd agree with emptythought: get a NAS (4-bay gives you more room to grow than 2-bay), back up to two drives individually, and then get another big dumb external drive for something approximating offsite. Test your backups. As that Backblaze link notes, the jury's out right now on the long-term reliability of 6GB drives, but I'd be okay with one as the rearguard option. That gets you to something like jwz's approach.
posted by holgate at 8:35 AM on January 27, 2015 [1 favorite]
I would listen to emptythought. I currently have a QNAP TS-459 Pro II (4 bay, raid 5) with enterprise drives. I've had it for a few years and had one drive failure. I purchased another drive, inserted it, and rebuilt the array no problem. Drives have a fixed time-span.
I can't speak for Synology's OS, but the QNAP software that comes on the NAS is linux-like and very stable. The backup station (part of the QNAP OS) also allows external backup through rsync, RTRR, TimeMachine, Amazon S3, Elephant Drive, and connected USB.
Of course, if you want VERY long term solutions, you can look into magnetic tape but that seems overkill.
posted by sk8ingdom at 9:36 AM on January 27, 2015
I can't speak for Synology's OS, but the QNAP software that comes on the NAS is linux-like and very stable. The backup station (part of the QNAP OS) also allows external backup through rsync, RTRR, TimeMachine, Amazon S3, Elephant Drive, and connected USB.
Of course, if you want VERY long term solutions, you can look into magnetic tape but that seems overkill.
posted by sk8ingdom at 9:36 AM on January 27, 2015
One thing I would suggest is having your NAS only accessible over your home network over ftp or something like that. The reason being if it is accessible as a shared folder then any virus/ransomware you get on your machine could get at the NAS as well.
posted by any portmanteau in a storm at 9:49 AM on January 27, 2015
posted by any portmanteau in a storm at 9:49 AM on January 27, 2015
I know you said you're unsatisfied with this solution, but the only way to protect against Cryptolocker, fires, theft and random deletion is to have multiple, offline copies that only change over time. If you have a NAS that mirrors your drives and you get Cryptolockered, or your place burns down or is broken into you're hosed. Those files are gone. If you have three external drives that you rotate in and out, there's always one drive free and somewhere else that you can use to restore.
It's easy enough to automate the data duplication. I wouldn't dissuade you from mirroring your stuff on-site, but you're not accomplishing what you want if the data doesn't exist off-site as well.
posted by cnc at 1:05 PM on January 27, 2015
It's easy enough to automate the data duplication. I wouldn't dissuade you from mirroring your stuff on-site, but you're not accomplishing what you want if the data doesn't exist off-site as well.
posted by cnc at 1:05 PM on January 27, 2015
"the external drive is the current solution with which I am not satisfied anymore."
Maybe you could explain your dissatisfaction in a little more detail?
Because as others say mirroring between two drives on a NAS doesn't really give you much additional safety. Certainly not as much as backing up to another external drive.
Based on your comment about USB2, is the problem the amount of time it takes to run a backup? How exactly are you doing the backup now, and how long does it take? It's true that gigabit ethernet should be faster than USB2, but there are other bottlenecks (e.g. the laptop drives that you're reading from) which may keep you from getting as much improvement as you're expecting.
posted by bfields at 1:43 PM on January 27, 2015
Maybe you could explain your dissatisfaction in a little more detail?
Because as others say mirroring between two drives on a NAS doesn't really give you much additional safety. Certainly not as much as backing up to another external drive.
Based on your comment about USB2, is the problem the amount of time it takes to run a backup? How exactly are you doing the backup now, and how long does it take? It's true that gigabit ethernet should be faster than USB2, but there are other bottlenecks (e.g. the laptop drives that you're reading from) which may keep you from getting as much improvement as you're expecting.
posted by bfields at 1:43 PM on January 27, 2015
Like a lot of the other responses have noted, you really want more than 1 backup - I would get the NAS in addition to the external hard drive. I backup to an external eSATA drive, a NAS, and I have Crashplan as the online supplement. I use Cobian backup for the dailies, which go to the external drive. I also have a scheduled monthly system image backed up to the external drive. The backups on the external drive are copied to the NAS, so I have 2 local copies. A few months ago I had the NAS and the drive in my PC fail the same week (it was a single bay NAS, since replaced by a dual bay mirrored QNAP) - I ended up restoring from the external drive. I would've been stuck reinstalling and downloading all my datafiles from Crashplan if I didn't have 2 local backups.
posted by Calloused_Foot at 7:45 PM on January 27, 2015
posted by Calloused_Foot at 7:45 PM on January 27, 2015
Response by poster: right now I have a bunch of external drives. let's concentrate on the three WD mystudio FW 800 drives I have. one is to take on location for photoshoots. dump everything on there and if the macbook pro gets trashed I have a copy. two house the archived copies of my keep-worthy shots. they are split as one drive is full. if this drive fails the images are gone. I could connect this drive and daisychain another drive and copy but then I begin running into the "which files did I copy" problem sooner or later again, which takes hours to go through. so what I want is one big harddrive that sits on the network and that automatically copies everything to a second harddrive that's copied to the first. I would like it to show up as a folder or drive on my os x desktop. that's pretty much it.
I like the idea of a 4-bay enclosure that many here have suggested and given that I am prepared to drop $1k without any second thought on this it seems doable (I'm including two drives in that price).
a proprietary layout format. You could say the same about hardware RAID: if, say, an expensive LSI controller managing a RAID5 array breaks on a server, then you need to replace it with another expensive LSI controller.
sorry, I don't understand. you mean a QNAP or Synology system would not be happy if I took out the hard drive and connected it to some other mac as an external drive to copy something off? what does layout system here mean? do they change file and pathnames? (I archive by naming my folders in a certain way, namely date and shoot name or location)
posted by krautland at 4:53 AM on January 28, 2015
I like the idea of a 4-bay enclosure that many here have suggested and given that I am prepared to drop $1k without any second thought on this it seems doable (I'm including two drives in that price).
a proprietary layout format. You could say the same about hardware RAID: if, say, an expensive LSI controller managing a RAID5 array breaks on a server, then you need to replace it with another expensive LSI controller.
sorry, I don't understand. you mean a QNAP or Synology system would not be happy if I took out the hard drive and connected it to some other mac as an external drive to copy something off? what does layout system here mean? do they change file and pathnames? (I archive by naming my folders in a certain way, namely date and shoot name or location)
posted by krautland at 4:53 AM on January 28, 2015
you mean a QNAP or Synology system would not be happy if I took out the hard drive and connected it to some other mac as an external drive to copy something off?
The layout system is low-level: it's the code that decides where to put filesystem data (your files and directories) on specific disk sectors. This used to be done exclusively in hardware, but can now also be done at the OS level. Both QNAP and Synology use 'software RAID', which is more flexible because it relies on a common codebase that isn't tied to a specific manufacturer's hardware.
You can't pull a drive out of a Synology NAS, hook it up via USB to a Mac and have it automatically readable. It's formatted for Linux. You can install drivers on a Mac to make it readable, but not if that drive is set up as part of a RAID5 array, regardless of whether it's hardware or software RAID.
Finally, your latest comment suggests that this isn't so much a backup problem for you as a hard drive capacity problem which means you can't easily back up All Of The Photos. That's a real problem. A NAS will give you expanded drive space for a primary archive; a 6TB external drive gives you expanded drive space as a secondary.
posted by holgate at 8:14 AM on January 28, 2015
The layout system is low-level: it's the code that decides where to put filesystem data (your files and directories) on specific disk sectors. This used to be done exclusively in hardware, but can now also be done at the OS level. Both QNAP and Synology use 'software RAID', which is more flexible because it relies on a common codebase that isn't tied to a specific manufacturer's hardware.
You can't pull a drive out of a Synology NAS, hook it up via USB to a Mac and have it automatically readable. It's formatted for Linux. You can install drivers on a Mac to make it readable, but not if that drive is set up as part of a RAID5 array, regardless of whether it's hardware or software RAID.
Finally, your latest comment suggests that this isn't so much a backup problem for you as a hard drive capacity problem which means you can't easily back up All Of The Photos. That's a real problem. A NAS will give you expanded drive space for a primary archive; a 6TB external drive gives you expanded drive space as a secondary.
posted by holgate at 8:14 AM on January 28, 2015
This thread is closed to new comments.
posted by mkb at 1:48 AM on January 27, 2015