Baby's first hard disk?
January 1, 2019 6:09 AM Subscribe
What's the best way to store images and other digital keepsakes for a baby?
We take a lot of pictures, I have some recordings of her heart beat, and there's probably more that I can't think of right now. We would like her to be able to have access to all this when she's older. I guess the three main approaches for storing the data are 1. online 2. on our home LAN 3. on some data storage medium, but we're not sure what the best solution would be given our requirements:
- We want to be able to store arbitrary data in native format
- Data should be available for the foreseeable future. That means we don't want to use technology or a provider that's likely not going to be around in a few years. Offsite backups are also a concern, but I guess that's mostly independent of the solution we choose.
- Easy maintenance. We're both techy people, and are comfortable with doing pretty much any tasks that might be required for initial setup, but we don't want to invest a lot of time afterwards
- If I'm going to store the data on the internet, I want them to be encrypted
- I don't mind paying for a good solution, but probably not much more than 50 Euro a year
My first idea was to store a Veracrypt container in a Dropbox, but I assume there are better solutions. Any ideas are appreciated.
We take a lot of pictures, I have some recordings of her heart beat, and there's probably more that I can't think of right now. We would like her to be able to have access to all this when she's older. I guess the three main approaches for storing the data are 1. online 2. on our home LAN 3. on some data storage medium, but we're not sure what the best solution would be given our requirements:
- We want to be able to store arbitrary data in native format
- Data should be available for the foreseeable future. That means we don't want to use technology or a provider that's likely not going to be around in a few years. Offsite backups are also a concern, but I guess that's mostly independent of the solution we choose.
- Easy maintenance. We're both techy people, and are comfortable with doing pretty much any tasks that might be required for initial setup, but we don't want to invest a lot of time afterwards
- If I'm going to store the data on the internet, I want them to be encrypted
- I don't mind paying for a good solution, but probably not much more than 50 Euro a year
My first idea was to store a Veracrypt container in a Dropbox, but I assume there are better solutions. Any ideas are appreciated.
Within the Amazon cloud offering it's Amazon Glacier that's for long term storage.
posted by jouke at 8:16 AM on January 1, 2019
posted by jouke at 8:16 AM on January 1, 2019
Single write DVD-R is supposedly good for 100 years if stored properly. I don't know exactly how much data you generate but if stability and encryption are important to you it's hard to beat optical media stored on the shelf in your closet, provided that you're OK saving everything off in DVD sized data chunks.
posted by dudemanlives at 9:03 AM on January 1, 2019 [1 favorite]
posted by dudemanlives at 9:03 AM on January 1, 2019 [1 favorite]
Use the LOCKSS principle ("Lots of Copies Keep Stuff Safe"): use multiple drives, keep backups, and move stuff to new media over time as old media degrades and compatibility changes. For example, I keep two separate encrypted USB backup drives for my laptop at home and at work.
posted by migurski at 9:25 AM on January 1, 2019 [5 favorites]
posted by migurski at 9:25 AM on January 1, 2019 [5 favorites]
Best answer: Do all 3: your home computer's HDD/SSD for normal access and long term archives on optical discs, preferably with extra offsite copies, made every year with the new data and verified at predefined time intervals. Online storage is not to be trusted for anything other than a fallback for surprise temporary setbacks like malfunctioning equipment for data that hasn't yet been replicated to a long term archive.
I think the best archival option for "normal" users is optical media: good quality discs easily last more than a decade and don't tend to fail as catastrophically as hard disks, they're read-only so you're covered against accidental deletion or ransomware shenaningans, and the ISO 9660 filesystem has near universal operating system compatibility (some caveats apply), CDs from 20 years ago are still readable with consumer drives you can buy today.
If you opt for BD-R discs, prefer HTL type over LTH.
- Easy maintenance. We're both techy people, and are comfortable with doing pretty much any tasks that might be required for initial setup, but we don't want to invest a lot of time afterwards
Whatever medium you pick, regular verification and probably migration is unavoidable. You could put everything in a 2TB HDD and hope that in 15 years it will still spin up but I wouldn't bet on it.
If you're attempting to preserve a lot of data, it's best to sort it into tiers, with the important things having more strict requirements than secondary stuff. Keep your long term archive minimal.
For example, long term archives would be integrity verified/refreshed every 2 years for tier 1 but only once every decade for tier 2 and not at all for tier 3, tier 1 data is backed up to online storage but tier 2 isn't, and so on.
Always keep at least 2 copies in easily accessible HDD/SSDs at all times (not counting long term archives and offsite backups), when one of the disks fails replace it and copy all relevant data into the new one.
Use checksum files (md5 is fine) for integrity checking and optionally add redundancy with par2 files. If possible, prefer a filesystem with integrity verification like Btrfs or ZFS and do a scrub every 3 months or so, but don't worry too much if that's not an option. IMO total hard drive failure is a much bigger concern than bitrot.
Always doing an integrity check on the source immediately before copying files to a long term archival medium, and again on the destination to ensure everything is properly readable (especially if you're burning optical discs) is more than enough.
Encryption: absolutely use it for online backups but consider if you want to also use it for your local storage and archives. What would be worse: someone stealing one of your backups and being able to access all data, or your kid being unable to access it in the future because the password was lost?
Encryption makes future data accessibilty significantly harder, it's another hoop to jump through and if one of your archive DVDs containing an encrypted tarball goes bad and can't read a few sectors you'll lose everything in it, while if that happens in an unencrypted one you probably lose only the affected files.
I recommend GnuPG for encrypting your uploads: bundle your new, not yet backed-up files into a directory, add a .md5 at the top level as a table of contents, tarball the whole bunch, encrypt it with gpg and send it to Dropbox. As soon as you update your long term archive, remove obsolete online backups. But if you're relying on online storage only for the near future, it's OK to resort to other, less portable solutions. When you need to resort to your backups, the less friction using your software the better.
Full disk encryption is not optimal for use with Dropbox but if it's what you're comfortable with it's much better than none at all and the odds someone would try to cryptographically attack your container are effectively nil.
If you're encrypting only online backups, default GPG is fine but if you also want to encrypt your archives, consider using symmetric encryption with a good password instead. It's easier to write down your password in a paper and keep it secure than needing to maintain a backup of your private OpenPGP key forever.
posted by Bangaioh at 9:59 AM on January 1, 2019
I think the best archival option for "normal" users is optical media: good quality discs easily last more than a decade and don't tend to fail as catastrophically as hard disks, they're read-only so you're covered against accidental deletion or ransomware shenaningans, and the ISO 9660 filesystem has near universal operating system compatibility (some caveats apply), CDs from 20 years ago are still readable with consumer drives you can buy today.
If you opt for BD-R discs, prefer HTL type over LTH.
- Easy maintenance. We're both techy people, and are comfortable with doing pretty much any tasks that might be required for initial setup, but we don't want to invest a lot of time afterwards
Whatever medium you pick, regular verification and probably migration is unavoidable. You could put everything in a 2TB HDD and hope that in 15 years it will still spin up but I wouldn't bet on it.
If you're attempting to preserve a lot of data, it's best to sort it into tiers, with the important things having more strict requirements than secondary stuff. Keep your long term archive minimal.
For example, long term archives would be integrity verified/refreshed every 2 years for tier 1 but only once every decade for tier 2 and not at all for tier 3, tier 1 data is backed up to online storage but tier 2 isn't, and so on.
Always keep at least 2 copies in easily accessible HDD/SSDs at all times (not counting long term archives and offsite backups), when one of the disks fails replace it and copy all relevant data into the new one.
Use checksum files (md5 is fine) for integrity checking and optionally add redundancy with par2 files. If possible, prefer a filesystem with integrity verification like Btrfs or ZFS and do a scrub every 3 months or so, but don't worry too much if that's not an option. IMO total hard drive failure is a much bigger concern than bitrot.
Always doing an integrity check on the source immediately before copying files to a long term archival medium, and again on the destination to ensure everything is properly readable (especially if you're burning optical discs) is more than enough.
Encryption: absolutely use it for online backups but consider if you want to also use it for your local storage and archives. What would be worse: someone stealing one of your backups and being able to access all data, or your kid being unable to access it in the future because the password was lost?
Encryption makes future data accessibilty significantly harder, it's another hoop to jump through and if one of your archive DVDs containing an encrypted tarball goes bad and can't read a few sectors you'll lose everything in it, while if that happens in an unencrypted one you probably lose only the affected files.
I recommend GnuPG for encrypting your uploads: bundle your new, not yet backed-up files into a directory, add a .md5 at the top level as a table of contents, tarball the whole bunch, encrypt it with gpg and send it to Dropbox. As soon as you update your long term archive, remove obsolete online backups. But if you're relying on online storage only for the near future, it's OK to resort to other, less portable solutions. When you need to resort to your backups, the less friction using your software the better.
Full disk encryption is not optimal for use with Dropbox but if it's what you're comfortable with it's much better than none at all and the odds someone would try to cryptographically attack your container are effectively nil.
If you're encrypting only online backups, default GPG is fine but if you also want to encrypt your archives, consider using symmetric encryption with a good password instead. It's easier to write down your password in a paper and keep it secure than needing to maintain a backup of your private OpenPGP key forever.
posted by Bangaioh at 9:59 AM on January 1, 2019
Best answer: Nthing a combination of online backup and local storage on the network. I used to recommend CrashPlan but with the new pricing plans it’s less appealing. These days, I recommend a combination of arq with Backblaze B2 as a backend - I can vouch for that combination as working well. Wasabi is probably also going to be a good backend choice but I have not tried it. Both will be in the neighborhood of $5/mo/TB and Arq is a one time charge. Arq is slightly unfriendly but I’m guessing from your question specifics you won’t have trouble setting it up.
posted by doomsey at 11:52 AM on January 1, 2019
posted by doomsey at 11:52 AM on January 1, 2019
As two other posters have said optical media, preferably archive-grade DVD is the safest. You can get some high-quality JVC ones (made in Japan) on Amazon. This and a Cloud-backup + local disk (such as Toshiba Canvio) and you should be well covered.
posted by jacobean at 3:44 PM on January 1, 2019 [1 favorite]
posted by jacobean at 3:44 PM on January 1, 2019 [1 favorite]
Best answer: Optical media? Hell, it’s hard enough to find a computer now that has an optical drive reader, let alone buying one 100 years from now. Or put another way, let’s say you had punch cards from 50 years ago that you knew to be perfectly preserved. How would you read them? What format would they be in? Perhaps someone with several standard deviations higher of technical aptitude could figure it out, but your by then middle aged kid would just probably think, “Why did Dad leave me all this junk?” and simply toss your intricately prepared but hopelessly outdated backups.
In terms of preserving your digital data, inevitably you are going to have to migrate this data set to some new storage format every 10 years or so. Heck, the format it will be in by the time your kid is in college probably hasn’t been invented yet. Verifying its integrity and readability each time you migrate is key.
On the other hand, I would suggest that you print up photo books of your favorite pics every year, with date and location annotated underneath. Of course this shouldn’t be your main backup, but odds are those photo books will survive far longer than any data contraption you set up.
posted by alidarbac at 6:02 PM on January 1, 2019 [2 favorites]
In terms of preserving your digital data, inevitably you are going to have to migrate this data set to some new storage format every 10 years or so. Heck, the format it will be in by the time your kid is in college probably hasn’t been invented yet. Verifying its integrity and readability each time you migrate is key.
On the other hand, I would suggest that you print up photo books of your favorite pics every year, with date and location annotated underneath. Of course this shouldn’t be your main backup, but odds are those photo books will survive far longer than any data contraption you set up.
posted by alidarbac at 6:02 PM on January 1, 2019 [2 favorites]
Best answer: Download and pay for Arq. Then setup Amazon Galcier and point Arq to the top level baby folder. It will keep everything in sync and cost almost nothing if you don't retrieve the data outside of emergencies. You can restore everything from Arq itself. The backups are encrypted.
In my usecase, my Documents folder gets continuously synced to Glacier. So in the even that I lose my laptop, and the various physical drive clones, I can still get the most irreplaceable stuff back.
posted by special-k at 9:10 PM on January 1, 2019
In my usecase, my Documents folder gets continuously synced to Glacier. So in the even that I lose my laptop, and the various physical drive clones, I can still get the most irreplaceable stuff back.
posted by special-k at 9:10 PM on January 1, 2019
print up photo books of your favorite pics every year, with date and location annotated underneath. Of course this shouldn’t be your main backup, but odds are those photo books will survive far longer than any data contraption you set up.
This is definitely true. The OP states digital right in the first sentence so I assume they already understand this but seconding just in case.
Don't set up any digital picture files preservation plan without already having printed a lot of them. If you need to pick only one, don't pick the digital option.
posted by Bangaioh at 4:10 AM on January 2, 2019 [1 favorite]
This is definitely true. The OP states digital right in the first sentence so I assume they already understand this but seconding just in case.
Don't set up any digital picture files preservation plan without already having printed a lot of them. If you need to pick only one, don't pick the digital option.
posted by Bangaioh at 4:10 AM on January 2, 2019 [1 favorite]
Best answer: DVDs and other optical media have an unfairly poor reputation, largely because of a quantity of poor-quality CD-R media that circulated in the 1990s and caused a lot of people who had bought cheap media to lose data. If you use high-quality media (Japanese manufacture, e.g. Taiyo Yuden, is generally well-regarded; anything Chinese...less so; big-name "archival" media is probably not a bad investment) and store it properly, write-once optical is a very stable format. I would trust it more than I would trust a hard drive, mostly because the media is separate from the read/write mechanism. If the mechanicals on a hard drive stop working, the data might still be fine, but it'll cost a fair bit to have someone rebuild the HDD in a clean room. In comparison, a new optical drive is like $20. I've burned hundreds and hundreds of optical discs in a variety of formats over the years, and across all of them (which I periodically spot-check), I've only had one disc end up totally unreadable... and it was on really crummy, bargain-basement media.
Anyway, I would keep things on (1) a hard drive connected to a computer, probably the one where you run Lightroom or whatever your photo-management software of choice is. This is your "on line" copy and not actually a backup at all, but it's the one you make backups from. I'd also have (2) a copy of the raw camera files on optical disc, probably DVD+R. There are some very minor advantages to DVD+R vs DVD-R. I don't know enough about BD-R to comment. Don't skimp on media, and store them properly away from heat and sunlight, in the original jewel boxes or a high-quality fully enclosed binder. Then (3) I would use a sane regular backup system for the computer, which could be either a Time Machine style local backup, or better yet a cloud service. Make sure you test it to ensure it's backing up what you think it's backing up.
Personally, and this is getting deep into opinion territory, I have mixed feelings about cloud backups. They are safer against a variety of threats than having a local backup. E.g. if your house burns down. However, they are vulnerable to a variety of potentially more-likely threats—like having your credit card get stolen three times in a year, and forgetting to update the account with the new number the third time, getting preoccupied with other stuff in your life, and then having the company delete everything for nonpayment. (This is, uh, not a hypothetical scenario.) Or you die, your spouse has other things to do besides keep making those payments, and the data all gets deleted before they get around to going through old photos. Whatever. It depends on a constant stream of money, in a way that a local backup doesn't. Think about what threats are more common in your life and what makes the most sense to hedge against.
On preview: Really like the photo books suggestion; I might consider taking a disc with the digital versions of the photos and storing it in / near the books.
posted by Kadin2048 at 9:29 AM on January 2, 2019 [2 favorites]
Anyway, I would keep things on (1) a hard drive connected to a computer, probably the one where you run Lightroom or whatever your photo-management software of choice is. This is your "on line" copy and not actually a backup at all, but it's the one you make backups from. I'd also have (2) a copy of the raw camera files on optical disc, probably DVD+R. There are some very minor advantages to DVD+R vs DVD-R. I don't know enough about BD-R to comment. Don't skimp on media, and store them properly away from heat and sunlight, in the original jewel boxes or a high-quality fully enclosed binder. Then (3) I would use a sane regular backup system for the computer, which could be either a Time Machine style local backup, or better yet a cloud service. Make sure you test it to ensure it's backing up what you think it's backing up.
Personally, and this is getting deep into opinion territory, I have mixed feelings about cloud backups. They are safer against a variety of threats than having a local backup. E.g. if your house burns down. However, they are vulnerable to a variety of potentially more-likely threats—like having your credit card get stolen three times in a year, and forgetting to update the account with the new number the third time, getting preoccupied with other stuff in your life, and then having the company delete everything for nonpayment. (This is, uh, not a hypothetical scenario.) Or you die, your spouse has other things to do besides keep making those payments, and the data all gets deleted before they get around to going through old photos. Whatever. It depends on a constant stream of money, in a way that a local backup doesn't. Think about what threats are more common in your life and what makes the most sense to hedge against.
On preview: Really like the photo books suggestion; I might consider taking a disc with the digital versions of the photos and storing it in / near the books.
posted by Kadin2048 at 9:29 AM on January 2, 2019 [2 favorites]
Response by poster: Thank you all for your ideas and suggestions, this is really helpful!
posted by snownoid at 12:52 PM on January 3, 2019
posted by snownoid at 12:52 PM on January 3, 2019
« Older How to do house swap accommodation New York Sydney... | What climate change organization(s) should I... Newer »
This thread is closed to new comments.
posted by peanut_mcgillicuty at 7:03 AM on January 1, 2019 [1 favorite]