How to delete all of a file extension from Time Machine backups.
August 19, 2018 7:08 AM Subscribe
I'm having a Time Machine space crisis. How can I delete all the versions of VMs that I've daftly included in the backups?
I'm presuming that TM is saving each version of each *.vdi that I have, so obviously that eats space hugely. I am OK with terminal so, were this a normal directory tree, I could do some find-pipe-delete thing, but I'm scared of confusing TM.
For ongoing stuff, I have now moved all the vdi's to a non-TM location.
I'm presuming that TM is saving each version of each *.vdi that I have, so obviously that eats space hugely. I am OK with terminal so, were this a normal directory tree, I could do some find-pipe-delete thing, but I'm scared of confusing TM.
For ongoing stuff, I have now moved all the vdi's to a non-TM location.
Also, hard disks are cheap now. Even in 2.5" they're only about AU$60/TB. In your position I would just be letting my existing, overstuffed TM drive fill up, then replacing it with a bigger, newer, cheaper one.
As an interesting exercise, it might be amusing to back the entire existing TM drive up to a Borg repository on a new drive, just to see how much space actually did get burnt by duplicating vdi content.
posted by flabdablet at 7:28 AM on August 19, 2018
As an interesting exercise, it might be amusing to back the entire existing TM drive up to a Borg repository on a new drive, just to see how much space actually did get burnt by duplicating vdi content.
posted by flabdablet at 7:28 AM on August 19, 2018
As I said, not a Mac user, but I did look at the way Time Machine works a while ago in the course of researching the tidiest way to build a WAN-based offsite backup facility for the school whose network I ran; and if nothing's changed since then, it should be safe just to run your find-pipe-delete thing against the Time Machine filesystem.
If Apple's iTunes team has not been allowed anywhere near Time Machine development, a Time Machine backup volume doesn't have anything like a database that can get out of sync with the rest of what's on it. Instead, it's just a huge pile of files, most of which are multiply hard-linked so that they can appear inside multiple dated backup subdirectories while actually only eating one lot of disk space.
To make any given file go away completely and reclaim the space it occupies, all you should need to do is delete it from all the subdirectories in which it appears; the space will get reclaimed as soon as its link count drops to zero.
The only feature of a Time Machine filesystem that's at odds with Unix tradition is that TM volumes also allow for arbitrary hard linking of directories; most Unices limit that kind of madness to automatically created .. links to parent directories. But you won't be deleting directories, only vdi files, so the only weird effect you might see is that deleting a vdi file from the first directory you find it in will also instantly delete it from all the other directories that link to the same inode.
If you're using something like a find|xargs pipeline to do your deletions, this might result in some File Not Found errors as xargs attempts to delete files that have unexpectedly disappeared already, but it should be safe to ignore those. Using find's inbuilt -delete primary will delete files as they're found and sidestep this issue.
But seriously, dude, these are your backups. Personally I treat backups as sacred objects, and would never even dream of running
posted by flabdablet at 8:11 AM on August 19, 2018 [2 favorites]
If Apple's iTunes team has not been allowed anywhere near Time Machine development, a Time Machine backup volume doesn't have anything like a database that can get out of sync with the rest of what's on it. Instead, it's just a huge pile of files, most of which are multiply hard-linked so that they can appear inside multiple dated backup subdirectories while actually only eating one lot of disk space.
To make any given file go away completely and reclaim the space it occupies, all you should need to do is delete it from all the subdirectories in which it appears; the space will get reclaimed as soon as its link count drops to zero.
The only feature of a Time Machine filesystem that's at odds with Unix tradition is that TM volumes also allow for arbitrary hard linking of directories; most Unices limit that kind of madness to automatically created .. links to parent directories. But you won't be deleting directories, only vdi files, so the only weird effect you might see is that deleting a vdi file from the first directory you find it in will also instantly delete it from all the other directories that link to the same inode.
If you're using something like a find|xargs pipeline to do your deletions, this might result in some File Not Found errors as xargs attempts to delete files that have unexpectedly disappeared already, but it should be safe to ignore those. Using find's inbuilt -delete primary will delete files as they're found and sidestep this issue.
But seriously, dude, these are your backups. Personally I treat backups as sacred objects, and would never even dream of running
find -delete against a backup volume I didn't have another backup of. So I strongly recommend acquiring more drive space and playing with Borg before you even try to tidy up your Time Machine.posted by flabdablet at 8:11 AM on August 19, 2018 [2 favorites]
If your Time Machine volume is attached across a network then your backup will be in a sparsebundle, which is a weird sort of mountable volume pieced together from "bands" of data stored as binary files. That link mentions that there's a "compact" command in hdiutil that will reclaim the space allocated to data you may have deleted; what it does not mention is how ungodly slow this process will be. Apple pretty much recommends deleting the whole backup and starting over if you need to reclaim space, and while it's possible to go into a sparsebundle, delete files that shouldn't have been in the backup at all, then unmount and compact it, I can tell you from personal experience it is not worth the trouble. Any operation on a sparsebundle is horribly slow; any time you do this sort of operation on a sparsebundle you risk corrupting the whole bundle and having Time Machine refuse to use it going forward. If your Time Machine volume is attached across a network then seriously, just go buy a new drive, label the old one with its effective date, and store it safely for any occasion you need to dig something out of deep storage. Do not attempt to delete files from a Time Machine backup that is attached across a network.
If your Time Machine volume is directly attached to your computer, however, the delete operation will be somewhat safer and the system should recognize the free space. Within the Time Machine interface there is, in fact, a command that will delete all backups of a given file. It's still pretty slow, with a whole lot of disk churning as it goes through all the hard links in old backups, but it's reasonably safe (safer than trying the same operation on a sparsebundle, anyway). Despite the increased level of safety, I have to say personally I wouldn't do this except under extreme duress.
So, to summarize: the best solution is to buy a new drive and put the old one away somewhere. I've got three old TM drives, two current ones in a dual drive dock, and a bunch of other decommissioned hard drives I've kept after various upgrades, and after doing a few heroic recovery or repair operations on corrupt backups I didn't want to lose, I really believe it's better to have more backups than to try to reclaim space.
posted by fedward at 9:38 AM on August 19, 2018 [3 favorites]
If your Time Machine volume is directly attached to your computer, however, the delete operation will be somewhat safer and the system should recognize the free space. Within the Time Machine interface there is, in fact, a command that will delete all backups of a given file. It's still pretty slow, with a whole lot of disk churning as it goes through all the hard links in old backups, but it's reasonably safe (safer than trying the same operation on a sparsebundle, anyway). Despite the increased level of safety, I have to say personally I wouldn't do this except under extreme duress.
So, to summarize: the best solution is to buy a new drive and put the old one away somewhere. I've got three old TM drives, two current ones in a dual drive dock, and a bunch of other decommissioned hard drives I've kept after various upgrades, and after doing a few heroic recovery or repair operations on corrupt backups I didn't want to lose, I really believe it's better to have more backups than to try to reclaim space.
posted by fedward at 9:38 AM on August 19, 2018 [3 favorites]
If you open Time Machine (the lame GUI thing) and right click on a snapshot of a file, you get an option to delete all backups of that file. Whether this is a practical solution for you depends on how many VDI files you backed up, but it does work for individual files.
posted by caek at 2:19 PM on August 19, 2018
posted by caek at 2:19 PM on August 19, 2018
This thread is closed to new comments.
Can't help you with your main issue because not a Mac user, but for ongoing backups of your vdi files you could do a lot worse than use Borg. Content-based de-duplication is the bees' knees.
posted by flabdablet at 7:17 AM on August 19, 2018 [1 favorite]