please recommend the right storage solution (mac, photos, not drobo)
March 28, 2013 5:48 AM Subscribe
I'm looking to end my backup mess - I have about ten external hard drives of varying sizes. my rough estimate is that I need 6TB to store my files. I was going to ask about RAID but previous questions make me wonder if it's truly right. so what should I choose? add. info inside.
I am looking to store mostly images and time machine backups. ideally, this solution should behave like just another hard drive showing up on my mac's desktop. drobo seems to use a proprietary solution, which I was told makes my data not easily read should I choose to abandon them at some point. I have no idea if RAID 5 or 6 is better for me or if I need something else. what I want is the reassurance of my data being safe from one drive failing. I have no need for this to be connected to the web but would prefer it working with my in-house wifi.
what (hopefully inexpensive) solution do you recommend, hivemind?
I am looking to store mostly images and time machine backups. ideally, this solution should behave like just another hard drive showing up on my mac's desktop. drobo seems to use a proprietary solution, which I was told makes my data not easily read should I choose to abandon them at some point. I have no idea if RAID 5 or 6 is better for me or if I need something else. what I want is the reassurance of my data being safe from one drive failing. I have no need for this to be connected to the web but would prefer it working with my in-house wifi.
what (hopefully inexpensive) solution do you recommend, hivemind?
I would go with something like this, especially if you can get an eSATA card or Thunderbolt-eSATA adapter for your computer. (USB will be slow as hell.) Or if you have a wired network in your house (especially gigabit), a NAS like this.
Either way, load it down with 3 TB HDs set up as RAID 5. (4 bays gets you 75% space efficiency, so you have room to grow.)
The hard part will be getting your data onto it and organizing. What the different drives store will dictate how they get unified. Do some drives have images, and other Time Machine backups? Is all the non-backup data on the drives also on your computer? If not, would it be possible to copy all the non-backup data to your computer? If so, the easiest thing to do would be to set the RAID up as a new Time Machine drive. I'm not sure how to copy over the Time Machine history as well, though I'm sure it's possible.
posted by supercres at 9:08 AM on March 28, 2013
Either way, load it down with 3 TB HDs set up as RAID 5. (4 bays gets you 75% space efficiency, so you have room to grow.)
The hard part will be getting your data onto it and organizing. What the different drives store will dictate how they get unified. Do some drives have images, and other Time Machine backups? Is all the non-backup data on the drives also on your computer? If not, would it be possible to copy all the non-backup data to your computer? If so, the easiest thing to do would be to set the RAID up as a new Time Machine drive. I'm not sure how to copy over the Time Machine history as well, though I'm sure it's possible.
posted by supercres at 9:08 AM on March 28, 2013
I basically asked the same question last month. I now have a Synology DiskStation DS413 with four 2TB WD Red drives in a RAID configuration. I can lose one drive, and the total capacity is about 5.5 TB. Cost me about $1200 CAD.
From reports on the Synology forums, Time Machine can be weird on non-Apple network drives. I personally can't get it to run, but there are alternatives. I'm also using Crashplan, which has taken a couple of weeks so far to punt the data from the computers and the NAS up to the server. Crashplan is $150/year for an unlimited storage, up to 10 computer setup. Crashplan, with a bit of fiddling, will also run on the NAS.
Wireless might be a bit slow and annoying for this application, but the Synology does support several USB wireless adapters, if you must.
posted by scruss at 9:42 AM on March 28, 2013 [1 favorite]
From reports on the Synology forums, Time Machine can be weird on non-Apple network drives. I personally can't get it to run, but there are alternatives. I'm also using Crashplan, which has taken a couple of weeks so far to punt the data from the computers and the NAS up to the server. Crashplan is $150/year for an unlimited storage, up to 10 computer setup. Crashplan, with a bit of fiddling, will also run on the NAS.
Wireless might be a bit slow and annoying for this application, but the Synology does support several USB wireless adapters, if you must.
posted by scruss at 9:42 AM on March 28, 2013 [1 favorite]
Have you considered a Networked Attached Storage (NAS) solution like the many products available from Synology? (Or look for NAS - there are many makes; Synology is usually recommended as the easiest to set up and use but YMMV.)
(On preview: scruss has specifics above. Ha.)
posted by RedOrGreen at 9:46 AM on March 28, 2013
(On preview: scruss has specifics above. Ha.)
posted by RedOrGreen at 9:46 AM on March 28, 2013
I'm a professional photographer with similar backup needs (several TB of images + multiple computers Time Machine) and I also bought a Synology Diskstation DS413 about 6 months ago and I LOVE it. I run it over wired gigabit to my main retouching machine and wifi to everything else - no issues with performance.
The web interface is amazing, better than anything I've used on any other product. Running other services is dead simple (I run VPN, FTP, torrents, web/sql for a few applications) but if you want to get super nerdy and go to the command line to tinker with whatever you want you can. I had no issues with Time Machine - you just create a separate user for each machine and give them a quota then point TM at it like any other drive.
I considered the Drobo but am really glad I went with the Synology. It does everything and it's easy to use. My friend who's a network admin recommended it to me, he said for small offices he's been replacing his client's aging tower servers with Synology boxes and then running their entire organization's email/backup/etc. off them. They're basically low-power mini servers.
posted by bradbane at 11:50 AM on March 28, 2013 [1 favorite]
The web interface is amazing, better than anything I've used on any other product. Running other services is dead simple (I run VPN, FTP, torrents, web/sql for a few applications) but if you want to get super nerdy and go to the command line to tinker with whatever you want you can. I had no issues with Time Machine - you just create a separate user for each machine and give them a quota then point TM at it like any other drive.
I considered the Drobo but am really glad I went with the Synology. It does everything and it's easy to use. My friend who's a network admin recommended it to me, he said for small offices he's been replacing his client's aging tower servers with Synology boxes and then running their entire organization's email/backup/etc. off them. They're basically low-power mini servers.
posted by bradbane at 11:50 AM on March 28, 2013 [1 favorite]
Buying your own backup solution is problematic unless you're a business or professional with money to throw at it.
I poked around on Newegg.com to find a NAS for you, and the results aren't good. You could go with this one for $100 plus 2 3TB hard drives for $140 apiece to total $380 (plus tax & shipping). But can that NAS even handle 6 TB? Can it handle the inevitable upgrade to 8 or 10TB? It's unclear and would take more research. Probably you're looking at more money for a solution that is likely to work for years.
There's the "Prosumer / home office" step up, but the cheapest 6 TB one I could find is over $1,000. And it 6TB means no room to expand. (Again, this was a cursory search.)
And as others have noted, you also have to properly design your backup system so you can always roll back your data. Can you put together a system that will let you restore a 3-week-old copy because 2 weeks ago your copy was infected with something malicious? I can't, maybe you can.
In contrast, Crashplan gives you unlimited backups for $60/year, and is cheaper if you buy a couple years at once. It handles everything automatically - no need to run backups, make sure your network shares are accessible at the right time, and it warns you if the backup ever fails. Yes, the initial upload will take a lot of time - my back-of-the-napkin calculation says 6 TB will take 117 days (approx 4 months) at 5 Mbps (check my math before you go trusting me). But since Crashplan has a free trial month, you can buy 3 months at $6/month and do the entire upload, then decide whether it's worth keeping.
Basically, any backup solution you roll yourself is likely to be a) expensive for 1 person b) prone to failure c) both. In constrast, you can try Crashplan for $18 for 4 months (with the freebie) and see what you think. The best part is you can still do your own onsite backups if you'd like, and those local copies will be readily available on your home network. But if things go really, really wrong - if your home is flooded, if your equipment is stolen, if you make a mistake and wipe out ALL your data on the computer and the backup - then that cheap, automatic Crashplan backup will be priceless.
I recently restored 0.5 TB from Crashplan. I will admit that the download speed was disappointing and it took awhile. But the fact is that it was available and easy in a way that any backup I could have designed would not be. If anything had gone terribly wrong, I still would have had my data.
posted by Tehhund at 12:55 PM on March 28, 2013
I poked around on Newegg.com to find a NAS for you, and the results aren't good. You could go with this one for $100 plus 2 3TB hard drives for $140 apiece to total $380 (plus tax & shipping). But can that NAS even handle 6 TB? Can it handle the inevitable upgrade to 8 or 10TB? It's unclear and would take more research. Probably you're looking at more money for a solution that is likely to work for years.
There's the "Prosumer / home office" step up, but the cheapest 6 TB one I could find is over $1,000. And it 6TB means no room to expand. (Again, this was a cursory search.)
And as others have noted, you also have to properly design your backup system so you can always roll back your data. Can you put together a system that will let you restore a 3-week-old copy because 2 weeks ago your copy was infected with something malicious? I can't, maybe you can.
In contrast, Crashplan gives you unlimited backups for $60/year, and is cheaper if you buy a couple years at once. It handles everything automatically - no need to run backups, make sure your network shares are accessible at the right time, and it warns you if the backup ever fails. Yes, the initial upload will take a lot of time - my back-of-the-napkin calculation says 6 TB will take 117 days (approx 4 months) at 5 Mbps (check my math before you go trusting me). But since Crashplan has a free trial month, you can buy 3 months at $6/month and do the entire upload, then decide whether it's worth keeping.
Basically, any backup solution you roll yourself is likely to be a) expensive for 1 person b) prone to failure c) both. In constrast, you can try Crashplan for $18 for 4 months (with the freebie) and see what you think. The best part is you can still do your own onsite backups if you'd like, and those local copies will be readily available on your home network. But if things go really, really wrong - if your home is flooded, if your equipment is stolen, if you make a mistake and wipe out ALL your data on the computer and the backup - then that cheap, automatic Crashplan backup will be priceless.
I recently restored 0.5 TB from Crashplan. I will admit that the download speed was disappointing and it took awhile. But the fact is that it was available and easy in a way that any backup I could have designed would not be. If anything had gone terribly wrong, I still would have had my data.
posted by Tehhund at 12:55 PM on March 28, 2013
As others have said RAID isnt backup. It can be a way to pool multiple drives and/or it can maintain availabiltity of disk-based storage in the event of failures.
The best inexpensive for you depends on various factors, including how much or that 6GB estimate is for images, and how much is for time machine backups. Also, of the images, what percentage needs to be easily and immediately accessable vs it being acceptable to have some delay in getting at them?
The cheapest option is to pickup some new, 4 tb hard disks and copy all your image files over to them, making sure that you have two copies of every file. Then, move one of the copies offsite. Maybe a friends house acrosss town, or a safe-deposit box.
For Time Machine, you can repurpose one of your old drives.
As an upgrade, you could add Crashplan to the mix. In addition to their paid offsite cloud backup, you can backup to other people for free over the internet. Even better, you can make a local backup, detatch it, and connect it to a remote machine running crashplan and keep on going, which saves the time it would take to push terrabytes of data over home internet connections.
Another upgrade would be to spend $100-200 on a storage capable router or small NAS that would make your external photo drives and time machine backups avaialble in your home network.
Note, don't confuse a backup with an archive. In other words don't push everything up to Crashplan's cloud service and then feel like you can delete the files to make space on your hard disk thinking that you'll be able to get them back from Crashplan at any time. You probably can but crashplan isn't designed to be your only source for a file.
I don't have as much data as you, but my approach is to try and have at least 3 copies of everything. Two at home, on different disks, and one in Crashplan's cloud backup. So, i have some files on a small server, which I copy at regular intervals to a second disk and backup to Crashplan. For other files, i have the copy on my laptop, a backup of my laptop via TimeMachine, and then my home directory is also backed up to Crashplan.
posted by Good Brain at 7:42 PM on March 28, 2013
The best inexpensive for you depends on various factors, including how much or that 6GB estimate is for images, and how much is for time machine backups. Also, of the images, what percentage needs to be easily and immediately accessable vs it being acceptable to have some delay in getting at them?
The cheapest option is to pickup some new, 4 tb hard disks and copy all your image files over to them, making sure that you have two copies of every file. Then, move one of the copies offsite. Maybe a friends house acrosss town, or a safe-deposit box.
For Time Machine, you can repurpose one of your old drives.
As an upgrade, you could add Crashplan to the mix. In addition to their paid offsite cloud backup, you can backup to other people for free over the internet. Even better, you can make a local backup, detatch it, and connect it to a remote machine running crashplan and keep on going, which saves the time it would take to push terrabytes of data over home internet connections.
Another upgrade would be to spend $100-200 on a storage capable router or small NAS that would make your external photo drives and time machine backups avaialble in your home network.
Note, don't confuse a backup with an archive. In other words don't push everything up to Crashplan's cloud service and then feel like you can delete the files to make space on your hard disk thinking that you'll be able to get them back from Crashplan at any time. You probably can but crashplan isn't designed to be your only source for a file.
I don't have as much data as you, but my approach is to try and have at least 3 copies of everything. Two at home, on different disks, and one in Crashplan's cloud backup. So, i have some files on a small server, which I copy at regular intervals to a second disk and backup to Crashplan. For other files, i have the copy on my laptop, a backup of my laptop via TimeMachine, and then my home directory is also backed up to Crashplan.
posted by Good Brain at 7:42 PM on March 28, 2013
Response by poster: given katrielalex answer I'd say I am looking for a raid system.
cloud storage isn't an option to me - you can't possibly imagine how difficult it sometimes is to even download a podcast from where I am, which is China, much less uploading a single image to flickr.
not sure what esata is but will look into it. if it makes a difference: I am using two mac book pros. one of them has thunderbolt, both have firewire 800, I think at least one only has usb 2.0, which sucks. so that's why I like the idea of them showing up via my home wifi router just like another server hard drive I mount on the desktop.
all my images at present are just the raw files sorted into folders by year and shoot date. so it's literally millions of already organised images but no past time machine backups that I need. I mentioned those because I need to start creating time machine backups and haven't done so because my external drives constantly tell me they are out of space when I try.
I will look into synology. if I ever were to abandon them, the data would be easy to transfer, right? I am asking this because of the drobo issue.
>Buying your own backup solution is problematic unless you're a business or professional with money to throw at it.
agreed but I'm in china and costs here are lower, which makes my life a bit easier. I am asking this question mostly because I need to know what to ask for before I talk to the folks who would set it up. I am trying to avoid being the person who has no clue if what the other one suggests makes any sense for his needs.
>The cheapest option is to pickup some new, 4 tb hard disks and copy all your image files over to them, making sure that you have two copies of every file. Then, move one of the copies offsite. Maybe a friends house acrosss town, or a safe-deposit box.
this is precisely what I am trying to do. I want to have the copies of my files created and administered automatically because I am constantly losing track over whether a file is already duplicated or not. this means that I delete very few files, have to spend more time searching than I would like to and am not 100% sure I am okay if one drive should fail.
thank you for all your replies. this is really nice of you.
posted by krautland at 8:02 PM on March 28, 2013
cloud storage isn't an option to me - you can't possibly imagine how difficult it sometimes is to even download a podcast from where I am, which is China, much less uploading a single image to flickr.
not sure what esata is but will look into it. if it makes a difference: I am using two mac book pros. one of them has thunderbolt, both have firewire 800, I think at least one only has usb 2.0, which sucks. so that's why I like the idea of them showing up via my home wifi router just like another server hard drive I mount on the desktop.
all my images at present are just the raw files sorted into folders by year and shoot date. so it's literally millions of already organised images but no past time machine backups that I need. I mentioned those because I need to start creating time machine backups and haven't done so because my external drives constantly tell me they are out of space when I try.
I will look into synology. if I ever were to abandon them, the data would be easy to transfer, right? I am asking this because of the drobo issue.
>Buying your own backup solution is problematic unless you're a business or professional with money to throw at it.
agreed but I'm in china and costs here are lower, which makes my life a bit easier. I am asking this question mostly because I need to know what to ask for before I talk to the folks who would set it up. I am trying to avoid being the person who has no clue if what the other one suggests makes any sense for his needs.
>The cheapest option is to pickup some new, 4 tb hard disks and copy all your image files over to them, making sure that you have two copies of every file. Then, move one of the copies offsite. Maybe a friends house acrosss town, or a safe-deposit box.
this is precisely what I am trying to do. I want to have the copies of my files created and administered automatically because I am constantly losing track over whether a file is already duplicated or not. this means that I delete very few files, have to spend more time searching than I would like to and am not 100% sure I am okay if one drive should fail.
thank you for all your replies. this is really nice of you.
posted by krautland at 8:02 PM on March 28, 2013
« Older Recover data from iPhone after Restore -- no... | Is "type letters 1 4 and 7 of your password"... Newer »
This thread is closed to new comments.
If you don't want to mess around with low-level drive stuff, there are a lot of consumer-oriented RAID controllers out there that let you basically put data in and have it do all the work. Drobo is one of these; I use one but it's true that if you want to leave Drobo then you'll have to rewrite all the data into a new format. On the other hand, it is a pretty well-made piece of kit wit a good interface.
posted by katrielalex at 6:34 AM on March 28, 2013