Help, UNIX wizards! How to hardlink directories?

January 6, 2015 6:34 PM

How to recursively copy a directory and keeping its tree structure but with the files in the new dir being not copies of but hardlinks to the originals?

ln doesn't allow linking directories so I usually end up making a regular copy and replacing all duplicates with hardlinks but lack of HDD space precludes this.

In Windows there is the Hardlink Clone functionality of Link Shell Extension and I assume it's pretty simple to achieve the same in UNIX despite not being able to figure it out myself.
I'd prefer a command-line solution but if needed I don't mind installing an extra file-manager or other software.

posted by Bangaioh to Computers & Internet (8 answers total)

You can do this by mounting the folder in a line in /etc/fstab:

/home/folderwiththeactualfiles /home/aliasname none bind 0 0

Check the manpage (man fstab) for more information.
posted by fifthrider at 6:39 PM on January 6, 2015

(Don't forget to reboot or run "mount -a" as root to make the change happen.)
posted by fifthrider at 6:40 PM on January 6, 2015

You can't hardlink directories, but you can symlink.

ln -s
posted by erst at 6:50 PM on January 6, 2015 [1 favorite]

To elaborate on the fstab answer above, what you want is a bind mount:
posted by lucasks at 8:14 PM on January 6, 2015

Best answer: There are lots of ways to do just about anything. Which way is best usually depends on your larger aim.

If you could give more details about why you want to make this thing, that would help.

If you want to alias a directory so that the very same directory shows up in two different places in the filesystem - meaning that all changes to files and subdirectories made in either place are reflected in the other - then the usual way to achieve that is with ln -s /path/to/existing/directory /path/to/new/place to make a symlink (symbolic link). Windows makes a distinction between symlinks to files (only available since Windows 7 iirc) and symlinks to directories, which it calls junctions; Unix doesn't.

If a symlink won't work for you because you're using some tool that specifically checks for them and handles them differently, Linux offers mount --bind /path/to/existing/directory /path/to/new/place that lets you make any file (including directories) appear in multiple places; there's also a --rbind option that makes the bind extend recursively to other filesystems mounted inside the original tree. Note that this is a Linux-specific feature, not something inherited from Unix.

If you want to make an initial clone of a tree of files, where the clone starts out occupying almost no extra disk space but where alterations subsequently made to the cloned directory and its subdirectories do not affect the original tree, the usual approach is cp -al to create a new directory full of hardlinks to the same files as in the original tree.

If you're doing that as part of a software development workflow, you're almost certainly better off using version control instead. If you're doing it as part of a backup workflow, you might care to consider LVM or filesystem snapshots as alternatives.
posted by flabdablet at 8:26 PM on January 6, 2015 [1 favorite]

Best answer: cp -al is the answer to the question as-asked. That will create a copy of the directory structure with all files hardlinked.

Others bring up good points when they suggest that you should consider some of the other possible solutions (bind mount, symlinks) based on why you need to do this, because both of those are normally considered preferable solutions to `cp -al`.

LVM snapshots or btrfs might also be worth considering if you have a frequent need to produce snapshot-like copies of data, particularly if you want them to be modifiable independent of the original.
posted by Kadin2048 at 11:01 PM on January 6, 2015

Response by poster: If you could give more details about why you want to make this thing, that would help.

I'm confident cp -al is exactly what I want (and embarassed for not remembering it since I use cp -aL routinely) but I might as well explain in case there's a more clever solution.

I keep all torrents in a tree structure where each tracker has its own parent folder. Files seeding on more than one tracker are renamed and hardlinked to their respective folders, allowing the client to delete and remove each torrent from disk without worrying about others requiring some or all of the files.
The cloned directory would allow me to organise the filesystem hierarchy and rename or delete (not modify, obviously) individual files without affecting active torrents.

Symlinks are not good for this purpose (neither are bind mounts if I understood them correctly) because deleting the original, symlinked file would destroy all "copies" and modifying the tree structure woud also be impossible without breaking things.
btrfs/LVM is out of the question as I prefer to avoid the extra complexity and want to be able to easily access files from OS other than Linux.

Thanks, everyone!
posted by Bangaioh at 7:41 AM on January 7, 2015

want to be able to easily access files from OS other than Linux

Then I agree with you that cp -al is probably your best bet. I believe the currently available ntfs-3g implementation for Linux does support NTFS hardlinks properly, so cp -al should run just fine against an NTFS filesystem which I would expect to be your cross-platform filesystem choice of least grief. FAT filesystems, including exFAT, don't support hard links.
posted by flabdablet at 8:24 AM on January 7, 2015