Automatic encrypted offsite backup

November 30, 2008 5:45 AM

I've two debian machines, one is onsite and one is offsite on a residential connection. I need to backup incrementally to this offsite server over SSH (or something just as secure) on a regular basis, this should require no manual intervention. I would however like a regular email to inform me of the status. The offsite machine isn't secured from burglary etc and therefore the backups need to be fully encrypted. The onsite machine is secure and thus can store encryption keys if need be.

I'm aware of duplicity however that's not stable and hasn't been updated in 5 years unfortuantly and thus isn't a wise idea to use in a commercial enviroment.

Any ideas would be extremely appreciated.
Many thanks,
Whoop

posted by Whoop to computers & internet (16 answers total)

It's true that duplicity isn't terribly stable, but it's not true that it hasn't been updated in five years — the latest release was 12 days ago, for what it's worth.

How about rsync over ssh?
posted by enn at 5:57 AM on November 30, 2008

Postscript: of course using SSH for rsync transit doesn't encrypt things at the remote end. Why not use EncFS on the remote machine? That would allow rsync to transmit deltas against the cleartext filesystem (assuming it is viewing the EncFS filesystem as decrypted).
posted by enn at 6:03 AM on November 30, 2008

Or if you want to be extra-FUSEy you can run EncFS on top of sshfs.
posted by enn at 6:09 AM on November 30, 2008

Sorry, just realised I forgot to mention something. This system also needs to be able to backup used files. I mount windows shares on the linux box, these windows shares often have a load of outlook archives in rsync dies on because they're in use.
posted by Whoop at 6:17 AM on November 30, 2008

Thirding EncFS, though I use it over rsync. Plus the magic of "cp -l" on the ext3 filesystem and some scripts to create daily incrementals that don't take up too much space.

Note that EncFS can be used in two ways. The typical way is to mount a plaintext version of a directory of ciphertext. The other way is to mount a ciphertext version of a directory of plaintext files. You can keep files in plaintext on your local machine, and use this latter sort of mount as your source of ciphertext to send to the remote machine.

Regarding the Windows issues, can you keep the Outlook PST files on the linux box, and then tell Outlook to look for the PSTs on a network share?
posted by chengjih at 6:24 AM on November 30, 2008

The outlook files are unfortuantly on the file server which is running Windows 2003, it wouldn't be practical to change this considering it's also a domain controller. So no, moving the pst's wouldn't be practical unfortuantly :(
posted by Whoop at 7:01 AM on November 30, 2008

How is EncFS unlocked? I assume it's not via a password, as that wouldn't be automatic.
posted by Whoop at 7:02 AM on November 30, 2008

jdmz has the best tutorial for rsync over ssh.
posted by Mach5 at 7:04 AM on November 30, 2008

EncFS is unlocked with a password. You can specify that the password be read from stdin if you like.
posted by enn at 7:18 AM on November 30, 2008

Sorry, just realised I forgot to mention something. This system also needs to be able to backup used files. I mount windows shares on the linux box, these windows shares often have a load of outlook archives in rsync dies on because they're in use.

That's a feature NAB. Imagine needing to recover a file, pulling it from backup, and then discovering that the backup is useless because the file is in an inconsistent state. The worst part of that situation would be that you won't discover the problem until you need to use the backup. Maybe what you want is a backup system that honors whatever file locking mechanism Outlook uses and keep retrying until the file is not in use.
posted by rdr at 8:38 AM on November 30, 2008

Well, last I checked, PST's over a network connection were not supported by Microsoft, but obviously many people operate this way (which was the case when I was doing pc/desktop support). I'd look into putting the PST's local, and using a tool like the Microsoft Outlook Personal Folders Backup Tool to get those PST's on the network share. Then go ahead and do rsync over ssh to an encrypted file system. Dealing with the rsync/encryption is easy work, dealing with Outlook PST's is a nightmare, as users' laptops/behaviors enter into the question.
posted by bxg at 8:48 AM on November 30, 2008

CrashPlan does exactly what you want. It is the best software I have paid for in years. It will do file versioning, though I don't know how it handles PST files. But you can try it for free.
posted by procrastination at 9:06 AM on November 30, 2008

The offsite machine isn't secured from burglary etc and therefore the backups need to be fully encrypted. The onsite machine is secure and thus can store encryption keys if need be.

Needless to say, if you're going to have an encrypted offsite backup of your data, you should also have an offsite backup of the keys needed to decrypt that data.

I mean, you'd feel pretty silly if the onsite computer was destroyed and you went to decrypt the backup and realised the encryption key was lost with the onsite computer.
posted by Mike1024 at 9:13 AM on November 30, 2008

These PST's are a real nightmare, I can't put the pst's local as they're all for different people. These people use the PST's as archives, therefore you would think the file is only read and not written to, there's no point in which these PST's can be expected to be not in use as people leave their PCs logged in 24/7.

Backing up the encryption keys are easy, as they never change and are very small and thus only need to be backed up once to a CD which can be stored in a secure location e.g. a bank vault.
posted by Whoop at 9:35 AM on November 30, 2008

If you think that backing up open PSTs is easier than getting your users to change their behavior, then I don't know what to say. Prioritizing features (encryption, full automation, etc) over functionality (actually copying the files) is not an effective strategy. Move the PSTs local, set up auto-archive to copy stuff to the file server, and back only those up. Lay down the law.
posted by rhizome at 10:05 AM on November 30, 2008

There have been 13 releases of duplicity in the past 12 months.
posted by PueExMachina at 4:20 PM on November 30, 2008