How to create a file library?

August 22, 2007 8:40 AM

[NoseBleedFilter] How to maintain a file library on Big Computer, and a small selection on Little Computer, so Big Computer Looks like Little Computer, but Little Computer doesn't need to carry all the files in file library?

Maybe this is simple, but I can't figure it out - I want to create a sync situation without needing to maintain a duplicate version of whole file system on each machine. : (forgive the metaphors)

1. I have a large computer with extensive backup situation - basically holds everything I own digitally. Lets call this the Library. There are two parts to the Library - Public Stacks, and the 2 weeks of daily backups. We are talking about the Stacks here - they are about 500gbs big.

2. I have a small computer that follows me where ever I go, that I copy a selection of files from the Stacks on to - current projects, music, books, files-that-change-daily.

3. Files checked out on Small Computer's maintains same file structure as Stack's - but not all the files are present.

GOAL:

To have changes made to the files and file structure checked out effect only that part of the Library. So if I check out folder 'A' - only changes to folder 'A' are recorded. 'B-Z' aren't deleted because they don't exist on Small Computer

SUB-GOAL:

To create a COMPLETE copy of the file structure - every file, and every folder - but only have the files I want be filled with data. So I have folders 'A-Z' on my computer, but only folder 'A' holds any actual data.

I hope that was clear - I'm not even sure if this is possible, same way I'm not sure fast then light travel is possible.

I'm on a windows system, but I'm willing to consider any Linux solutions.

posted by mrgreyisyelling to computers & internet (7 comments total)

I'm pretty sure SyncBack can do what you need (I used to do something analogous between my computer and my mp3 player). There's both a freeware and a more fully-featured shareware version.
posted by kickingtheground at 9:09 AM on August 22, 2007

Try microsoft's synctoy. Its free too. I think you should be able to sync on a folder by folder basis. You want to pair up whatever folders you want on the smaller computer.
posted by damn dirty ape at 9:13 AM on August 22, 2007

Both SyncBack and synctoy are sort of meh, buggy, missing simple features.
I asked a related question and got this useful answer.
posted by signal at 9:40 AM on August 22, 2007

Check out Keeping Your Life in Subversion. It requires a Linux system, and it was a little complicated for my needs, but it might be just the ticket for you.

It's free, and infinitely scalable.
posted by nasreddin at 9:51 AM on August 22, 2007 [1 favorite]

Thanks all so far

I'm currently using SyncBack actually - the problem there is the lack of a way to maintain a folder sync without duplicating all the data. So if Folder A has folders 1, 2, 3, 4, 5 - you need all 5 folders to maintain a sync, even if you only want folder 1. Great program otherwise.

Subversion seems to be the answer so far. And it might run on Windows - at first glance. I'm a little nervous though because it always seemed very complicated. Here is O'Reilly Online book on Subversion for the future people.
posted by mrgreyisyelling at 10:51 AM on August 22, 2007

Subversion is a bit of a pain to set up, but it is fantastic for this sort of thing. With text, in particular, but it can store binary diffs as well if you're not using text.

I haven't done any SVN administration in quite a while, but I *think* you'd want to set up one repository ("My Stuff"), with multiple projects (anywhere from one file to a set of files - whatever belongs together). Then, you can check out an individual file, or an entire project, onto your laptop. This is also good for backups, because Stacks is the "trunk", and your backups can be "tagged" (i.e., kept as a record of the 'old version'). My terminology might be a bit off.

The one thing you specified that it doesn't do exactly is file structure. It won't create empty files of the stuff you don't check out, and I don't *think* it'll create parent folders (i.e., checking out file Z that's in folder A/B/C doesn't get you the parent folders). But that might be okay, since when you check Z back in, SVN will automatically put it back in the right place in the tree (A/B/C). So it sort of depends on why that's important.

And it does run on Windows.
posted by spaceman_spiff at 2:03 PM on August 22, 2007

Er, one more thing re: file structure. In my example, if folder/path A/B/C contained 3 files: X, Y and Z. You could check out Z, and all you'd get is Z. But if you checked out, say, B, you'd get B/C/[X,Y,Z].
posted by spaceman_spiff at 2:05 PM on August 22, 2007