Options for secure data storage?
June 8, 2005 7:42 AM Subscribe
What are my best options for safely storing in excess of 700Gb of data?
Background: I'm involved in a project where we have between 700 and 800 Gigabytes of data we need to make sure isn't lost. The data is made up of the source materials for an independent feature film -- including various rough cuts of the film, plus all the source video, audio, and graphic files. Since this is our first feature length project, we've never had a need to secure as much data. Foir shorter projects we've been able to burn a series of CDs or data DVDs to save in a safety deposit box. Since that isn't a good solution for our current project, we need some suggestions. I tend to think some sort of offsite, colocated data storage company would be our best bet, but have no idea where to start or which storage companies are reputable.
At the moment, the data is just sitting on the array of hard disks in our editing studio, so the sooner we can get it secured the better. FWIW, the original rolls of film and color corrected transfer tapes are in a safety deposit box, and we do temporary backups of sections of the project on data DVDs as we go along. I just don't think that's good enough. Do you have recommendations for specific data storage companies or solutions that have worked for you?
Background: I'm involved in a project where we have between 700 and 800 Gigabytes of data we need to make sure isn't lost. The data is made up of the source materials for an independent feature film -- including various rough cuts of the film, plus all the source video, audio, and graphic files. Since this is our first feature length project, we've never had a need to secure as much data. Foir shorter projects we've been able to burn a series of CDs or data DVDs to save in a safety deposit box. Since that isn't a good solution for our current project, we need some suggestions. I tend to think some sort of offsite, colocated data storage company would be our best bet, but have no idea where to start or which storage companies are reputable.
At the moment, the data is just sitting on the array of hard disks in our editing studio, so the sooner we can get it secured the better. FWIW, the original rolls of film and color corrected transfer tapes are in a safety deposit box, and we do temporary backups of sections of the project on data DVDs as we go along. I just don't think that's good enough. Do you have recommendations for specific data storage companies or solutions that have worked for you?
You could get a tape autoloader off of Ebay, dump the data to tape, and then ship it off to an offsite, climate controlled facility run by a company like Iron Mountain.
If the data is that important, you don't want to just rely on hard drives or CDs.
posted by cmonkey at 7:59 AM on June 8, 2005
If the data is that important, you don't want to just rely on hard drives or CDs.
posted by cmonkey at 7:59 AM on June 8, 2005
I second tape backups, harddrives are very senstive, and one little scratch on that DVD and that full frontal nudity scene youve been saving is gone.
posted by Mach5 at 8:19 AM on June 8, 2005
posted by Mach5 at 8:19 AM on June 8, 2005
if you back it up to disk, don't leave the disks online. this may be obvious, but i know someone who did leave them online. they were hacked and *everything* erased.
posted by andrew cooke at 8:20 AM on June 8, 2005
posted by andrew cooke at 8:20 AM on June 8, 2005
thats probably too much data to transfer over the internet. it sounds like you should probably invest in a tape backup system, hard drives would work but long term idle storage isnt what they are made for, since they have moving parts.
when i think storage, i think storagetek or emc, though the more end-user friendly companies like Dell, HP, or IBM might be easier to deal with, and if you find a decent salesperson they could at definitely give you and idea of whats out there. it wouldnt hurt to chat with IBM and Dell sales folks and then just verifying and checking out their ideas on the web.
posted by yeahyeahyeahwhoo at 8:26 AM on June 8, 2005
when i think storage, i think storagetek or emc, though the more end-user friendly companies like Dell, HP, or IBM might be easier to deal with, and if you find a decent salesperson they could at definitely give you and idea of whats out there. it wouldnt hurt to chat with IBM and Dell sales folks and then just verifying and checking out their ideas on the web.
posted by yeahyeahyeahwhoo at 8:26 AM on June 8, 2005
Response by poster: Tape backup might not be a bad idea ... any suggestions for Mac-friendly (OSX) choices?
posted by ScottUltra at 8:54 AM on June 8, 2005
posted by ScottUltra at 8:54 AM on June 8, 2005
Backing up to tape is the most common type of backup for large amounts of data. However, tapes themselves are not a guaranteed solution. Depending on the age of the tapes and the vendor/format, there is risk. If you go with tape, stay away from Exabyte Mammoth 2 units. They are absolute crap. SDLT is pretty good, and currently I have Sony AIT units that work well. Given the amount of data you want to back up, you will need a library unit, unless you plan on being there to change tapes.
Other questions I would ask, are how exactly do you plan on backing the data up? I assume when you burned to DVD/CD it was data in its native format. That will not be the case when backing up to tape. What kind of budget do you have for disaster recovery?
Seeing that the data is on an array is good. You should make sure you have some spare drives for the array on hand in the event you lose a drive. You don't want to lose another drive waiting for a spare to show up.
Good luck with it.
posted by a3matrix at 9:50 AM on June 8, 2005
Other questions I would ask, are how exactly do you plan on backing the data up? I assume when you burned to DVD/CD it was data in its native format. That will not be the case when backing up to tape. What kind of budget do you have for disaster recovery?
Seeing that the data is on an array is good. You should make sure you have some spare drives for the array on hand in the event you lose a drive. You don't want to lose another drive waiting for a spare to show up.
Good luck with it.
posted by a3matrix at 9:50 AM on June 8, 2005
Best answer: Scott,
Realistically I think you can do one of two/three things.
Here's how the industry generally does this:
Back up ONLY the graphics etc.
You have a backup of the footage- the TAPES. That's why timecode is so important.
Especially if it came from telecine tapes.
Need to restore the show? Take the sequence, batch digitize the tapes, add the graphics.
Much smaller than trying to store the video.
-----------
A pair of terabyte firewire drives (lacie has a terabyte for $$800)
Why two? redundancy.
---------------
Buy a firewire Raid 5 (has redundancy built in in case a drive goes down.) Buy two.
That's a start.
DVDs are too small...tapes are too slow.
posted by filmgeek at 10:06 AM on June 8, 2005
Realistically I think you can do one of two/three things.
Here's how the industry generally does this:
Back up ONLY the graphics etc.
You have a backup of the footage- the TAPES. That's why timecode is so important.
Especially if it came from telecine tapes.
Need to restore the show? Take the sequence, batch digitize the tapes, add the graphics.
Much smaller than trying to store the video.
-----------
A pair of terabyte firewire drives (lacie has a terabyte for $$800)
Why two? redundancy.
---------------
Buy a firewire Raid 5 (has redundancy built in in case a drive goes down.) Buy two.
That's a start.
DVDs are too small...tapes are too slow.
posted by filmgeek at 10:06 AM on June 8, 2005
Best answer: Raid 5 is not designed for backup, it's designed for uptime.
My suggestion is two 1TB firewire drives, stored in different physical locations. One of them should be somewhere secure, such as a safe, or a safety deposit box. The other one should be somewhere accessible for an easy restore.
posted by Jairus at 10:14 AM on June 8, 2005
My suggestion is two 1TB firewire drives, stored in different physical locations. One of them should be somewhere secure, such as a safe, or a safety deposit box. The other one should be somewhere accessible for an easy restore.
posted by Jairus at 10:14 AM on June 8, 2005
Response by poster: filmgeek & Jairus - Our tapes are telecine, so we've discussed doing exactly what you propose. Our 2 main concerns about this method revolved around (1) storing the data on drives with moving parts, and the risk of data corruption/loss that comes with transporting the drive to a storage facility, and (2) the time necessary to rebuild the entire edit from timecode in case we lose everything. Redundant backups would ease our worries a bit, and I agree that backing up to tape seems slow and painful. I appreciate your input, and think we'll probably go this route.
posted by ScottUltra at 10:24 AM on June 8, 2005
posted by ScottUltra at 10:24 AM on June 8, 2005
I had heard that apple is actually coming out with an iPod that can store that data for you, AND it will let you view and edit it in real time. Of course, the downside is that it will not support OGG Vorbis...
posted by indiebass at 10:33 AM on June 8, 2005
posted by indiebass at 10:33 AM on June 8, 2005
This thread is closed to new comments.
I remember an article about some Microsoft bigwig who needed to ship like a TB of data and figured out that the cheapest and fastest way was to buy a computer, copy the data and fedex it to where it needed to be, as the cost and speed of transfering that much data over the internet would wbe much worse.
posted by signal at 7:57 AM on June 8, 2005