Sufficiently Advanced Technology, or Magic?
September 3, 2012 10:54 PM Subscribe
How can a virtual machine track the guest OS's use of an emulated disk?
I use Parallels-7 Desktop on a MacBook Pro, and I run various guest OSes (Linux & Windows) using emulated (MacOS file) disks. From time to time, Parallels notifies me that a guest OS has freed a lot of space on one of these disks, and it offers to compress the MacOS file to reduce (real) disk usage.
How can a virtual machine determine whether an emulated disk sector is in-use or free?
I'd guess the simple way to do an emulated disk is just to tell the guest OS how big it is, and accept and store data blocks at whatever address the guest OS says it wants to write. You wouldn't have to allocate the whole file at once (and it seems this is what Parallels does: the "10GB" disk my guest OS sees only takes a few actual gigs in the Mac filesystem, at least until the guest OS disk gets full). You just keep track of what blocks have been written, and return zeros if the guest reads a block that it hasn't written yet.
However, I don't see how Parallels can know when a sector that the guest previously wrote, goes out of use, unless the guest OS somehow notifies it. And if there is such a notification, it seems that it would be a perfect place to write random data to the newly-freed block to prevent recovery of deleted files, etc.
I have some familiarity with filesystem logic & operations, but I've never heard of a "free-data-block" event. Is this something new? Or is there an even cooler trick Parallels uses to detect unused blocks?
I use Parallels-7 Desktop on a MacBook Pro, and I run various guest OSes (Linux & Windows) using emulated (MacOS file) disks. From time to time, Parallels notifies me that a guest OS has freed a lot of space on one of these disks, and it offers to compress the MacOS file to reduce (real) disk usage.
How can a virtual machine determine whether an emulated disk sector is in-use or free?
I'd guess the simple way to do an emulated disk is just to tell the guest OS how big it is, and accept and store data blocks at whatever address the guest OS says it wants to write. You wouldn't have to allocate the whole file at once (and it seems this is what Parallels does: the "10GB" disk my guest OS sees only takes a few actual gigs in the Mac filesystem, at least until the guest OS disk gets full). You just keep track of what blocks have been written, and return zeros if the guest reads a block that it hasn't written yet.
However, I don't see how Parallels can know when a sector that the guest previously wrote, goes out of use, unless the guest OS somehow notifies it. And if there is such a notification, it seems that it would be a perfect place to write random data to the newly-freed block to prevent recovery of deleted files, etc.
I have some familiarity with filesystem logic & operations, but I've never heard of a "free-data-block" event. Is this something new? Or is there an even cooler trick Parallels uses to detect unused blocks?
Response by poster: Thanks, hattifattener! That was exactly the search string I needed to find at Wikipedia. I can't believe I haven't heard about people capitalizing on TRIM to do secure deletion...I have a mild case of tin-foil-hat, and the last thing I heard about secure deletion was about some idiot who sent a kidnapping threat/ransom letter on a flash drive. Turned out, he had saved his drafts on the drive too, and although he had erased the metadata from the final version, wear leveling in the drive caused draft sectors to be available forensically, and the cops found his name and info.
(Frankly, I'm a bit skeptical that this actually happened as the story was related, but based on what I know about Flash operations, it certainly could have happened.)
posted by spacewrench at 11:08 PM on September 3, 2012
(Frankly, I'm a bit skeptical that this actually happened as the story was related, but based on what I know about Flash operations, it certainly could have happened.)
posted by spacewrench at 11:08 PM on September 3, 2012
TRIM may be part of it, but I suspect Parallels uses simpler metrics, especially since it has long been able to compress virtual disks for Windows XP VMs which don't support TRIM (absent third-party software).
Realistically, Parallels doesn't just emulate a completely raw block storage device; it also has some knowledge of how the filesystems it supports function. Note that Parallels can only compress and manipulate certain types of filesystems, which is a good clue to how this works.
So Parallels can get the true in-use size of the filesystem (basically, the same thing you'd see if you check how much free space is available in the guest OS) and compare that to the size of the virtual disk file. If the filesystem thinks it only is keeping track of 10 GB worth of files and the virtual disk file is 20 GB, Parallels can compress the volume so the virtual disk file only takes up about 10 GB as well. The operation would pretty much be akin to traditional defragmentation or partition resizing, though for performance it may take some shortcuts when it comes to compactly stored, but fragmented, files. Parallels can also blow away temporary caches and other garbage before compressing, as it knows the filesystem and OS.
Indeed, even if Parallels relied on TRIM to figure out which blocks are unused, it would be a pain to compress the virtual disk without knowledge of the filesystem. You'd have to excise all the free blocks and maintain a giant mapping table as to where you've put everything. Otherwise, the LBAs that the filesystem refers to won't line up anymore with the locations in the virtual disk file. As I understand it, real disks do a bit of this kind of mapping to deal with bad sectors (and SSDs do a ton of it as part of the block allocation process), but it would be a pain and a big performance drain to do it for a virtual disk. Much easier to move files around in the supported filesystems and keep your virtual block device simple.
A related technology is Apple's "Sparse Image/Sparse Bundle" disk images. These are basically auto-growing disk images that only take up as much space as their contents. Similarly to Parallels, you can use a terminal command to compress them after deleting a bunch of files to reclaim the space. NTFS and other filesystems have sparse files as well.
TRIM was invented because SSDs do behave like raw block storage devices and aren't expected to have any knowledge of the underlying filesystem, just like dumb old hard disks. While VMs like Parallels can "cheat" and use the filesystem's metadata/directory/catalog to figure out what blocks have been abandoned, SSDs had no way of knowing when a block represented free space after a file was deleted. TRIM was invented to help the SSD's firmware know what blocks of data need not be saved during garbage collection and wear leveling, which helps the device maintain performance after a larger number of writes. Useful certainly, but not necessary to implement Parallels's compression scheme.
posted by zachlipton at 12:20 AM on September 4, 2012 [2 favorites]
Realistically, Parallels doesn't just emulate a completely raw block storage device; it also has some knowledge of how the filesystems it supports function. Note that Parallels can only compress and manipulate certain types of filesystems, which is a good clue to how this works.
So Parallels can get the true in-use size of the filesystem (basically, the same thing you'd see if you check how much free space is available in the guest OS) and compare that to the size of the virtual disk file. If the filesystem thinks it only is keeping track of 10 GB worth of files and the virtual disk file is 20 GB, Parallels can compress the volume so the virtual disk file only takes up about 10 GB as well. The operation would pretty much be akin to traditional defragmentation or partition resizing, though for performance it may take some shortcuts when it comes to compactly stored, but fragmented, files. Parallels can also blow away temporary caches and other garbage before compressing, as it knows the filesystem and OS.
Indeed, even if Parallels relied on TRIM to figure out which blocks are unused, it would be a pain to compress the virtual disk without knowledge of the filesystem. You'd have to excise all the free blocks and maintain a giant mapping table as to where you've put everything. Otherwise, the LBAs that the filesystem refers to won't line up anymore with the locations in the virtual disk file. As I understand it, real disks do a bit of this kind of mapping to deal with bad sectors (and SSDs do a ton of it as part of the block allocation process), but it would be a pain and a big performance drain to do it for a virtual disk. Much easier to move files around in the supported filesystems and keep your virtual block device simple.
A related technology is Apple's "Sparse Image/Sparse Bundle" disk images. These are basically auto-growing disk images that only take up as much space as their contents. Similarly to Parallels, you can use a terminal command to compress them after deleting a bunch of files to reclaim the space. NTFS and other filesystems have sparse files as well.
TRIM was invented because SSDs do behave like raw block storage devices and aren't expected to have any knowledge of the underlying filesystem, just like dumb old hard disks. While VMs like Parallels can "cheat" and use the filesystem's metadata/directory/catalog to figure out what blocks have been abandoned, SSDs had no way of knowing when a block represented free space after a file was deleted. TRIM was invented to help the SSD's firmware know what blocks of data need not be saved during garbage collection and wear leveling, which helps the device maintain performance after a larger number of writes. Useful certainly, but not necessary to implement Parallels's compression scheme.
posted by zachlipton at 12:20 AM on September 4, 2012 [2 favorites]
It could be TRIM, but don't you have the virtual machine tools installed in your guest OS?
I don't know about Parallels, but the VMWare tools had a way of letting the host know about free space so that it could safely shrink the virtual disk. It was a batch process, but i dont see any reason why i couldnt be automatically triggered.
posted by Good Brain at 12:20 AM on September 4, 2012
I don't know about Parallels, but the VMWare tools had a way of letting the host know about free space so that it could safely shrink the virtual disk. It was a batch process, but i dont see any reason why i couldnt be automatically triggered.
posted by Good Brain at 12:20 AM on September 4, 2012
Windows XP doesn't support TRIM. Windows 7 does. Not sure about Vista. Linux has had TRIM support since kernel 2.6.28.
Indeed, even if Parallels relied on TRIM to figure out which blocks are unused, it would be a pain to compress the virtual disk without knowledge of the filesystem. You'd have to excise all the free blocks and maintain a giant mapping table as to where you've put everything. Otherwise, the LBAs that the filesystem refers to won't line up anymore with the locations in the virtual disk file. As I understand it, real disks do a bit of this kind of mapping to deal with bad sectors (and SSDs do a ton of it as part of the block allocation process), but it would be a pain and a big performance drain to do it for a virtual disk. Much easier to move files around in the supported filesystems and keep your virtual block device simple.
If you have a look at the on-disk format documentation for almost any of the commonly used VM disk formats, this kind of giant mapping table is pretty much exactly what you'll find. It does indeed cause a performance hit, which is why most VM hosting software gives you the option to use fixed-size image files instead and/or use the underlying host's disk cache.
The performance hit isn't quite as horrible as you might at first think, mainly because the mapped blocks are usually fairly big (1MiB is pretty common) and the VM host software will try to arrange for each such block to occupy a sequential set of underlying disk sectors. This does mean that virtual filesystem "holes" smaller than the mapping block size can't generally be recovered as free real disk space even if the guest OS does support TRIM.
posted by flabdablet at 3:24 AM on September 4, 2012 [1 favorite]
Indeed, even if Parallels relied on TRIM to figure out which blocks are unused, it would be a pain to compress the virtual disk without knowledge of the filesystem. You'd have to excise all the free blocks and maintain a giant mapping table as to where you've put everything. Otherwise, the LBAs that the filesystem refers to won't line up anymore with the locations in the virtual disk file. As I understand it, real disks do a bit of this kind of mapping to deal with bad sectors (and SSDs do a ton of it as part of the block allocation process), but it would be a pain and a big performance drain to do it for a virtual disk. Much easier to move files around in the supported filesystems and keep your virtual block device simple.
If you have a look at the on-disk format documentation for almost any of the commonly used VM disk formats, this kind of giant mapping table is pretty much exactly what you'll find. It does indeed cause a performance hit, which is why most VM hosting software gives you the option to use fixed-size image files instead and/or use the underlying host's disk cache.
The performance hit isn't quite as horrible as you might at first think, mainly because the mapped blocks are usually fairly big (1MiB is pretty common) and the VM host software will try to arrange for each such block to occupy a sequential set of underlying disk sectors. This does mean that virtual filesystem "holes" smaller than the mapping block size can't generally be recovered as free real disk space even if the guest OS does support TRIM.
posted by flabdablet at 3:24 AM on September 4, 2012 [1 favorite]
If you have a look at the on-disk format documentation for almost any of the commonly used VM disk formats, this kind of giant mapping table is pretty much exactly what you'll find. It does indeed cause a performance hit, which is why most VM hosting software gives you the option to use fixed-size image files instead and/or use the underlying host's disk cache.
Yeah you're right. I did some experimentation with Parallels (version 7, using a Windows 7 NTFS expanding volume) after I posted my answer and found that the "compression" operation is not moving files around in the virtual filesystem, at least not to a noticeable extent, but rather is manipulating the virtual disk's allocation table (i.e. "giant mapping table").
Thinking about it, this approach does make sense, because it's the most logical way to support snapshots and other special VM disk features.
posted by zachlipton at 9:34 AM on September 4, 2012
Yeah you're right. I did some experimentation with Parallels (version 7, using a Windows 7 NTFS expanding volume) after I posted my answer and found that the "compression" operation is not moving files around in the virtual filesystem, at least not to a noticeable extent, but rather is manipulating the virtual disk's allocation table (i.e. "giant mapping table").
Thinking about it, this approach does make sense, because it's the most logical way to support snapshots and other special VM disk features.
posted by zachlipton at 9:34 AM on September 4, 2012
Response by poster: It could be TRIM, but don't you have the virtual machine tools installed in your guest OS?
I do, and I understand that if something in the guest was able to report back to the host VM, or if the host VM could parse the guest's filesystem format, this sort of compression would be easy (except for the part about grokking a dozen different filesystems, and making sure that your bugs don't cause corruption in the guest!)
So I guess the real answer is, there are a few ways to do it, and a good VM package will use some or all of them. But the one I was particularly interested in is where something (ideally hardware) can tell that a sector has become unused, because that's a perfect place to erase/secure the sector.
However, after doing a little more reading, people point out:
* You can't trust somebody else to do sector erasing. Apparently several vendors advertise it, but don't actually follow through
* If you have sector erasing, then malware can really hurt you by simply spewing a stream of "unused sector" messages to the disk.
posted by spacewrench at 9:37 AM on September 4, 2012
I do, and I understand that if something in the guest was able to report back to the host VM, or if the host VM could parse the guest's filesystem format, this sort of compression would be easy (except for the part about grokking a dozen different filesystems, and making sure that your bugs don't cause corruption in the guest!)
So I guess the real answer is, there are a few ways to do it, and a good VM package will use some or all of them. But the one I was particularly interested in is where something (ideally hardware) can tell that a sector has become unused, because that's a perfect place to erase/secure the sector.
However, after doing a little more reading, people point out:
* You can't trust somebody else to do sector erasing. Apparently several vendors advertise it, but don't actually follow through
* If you have sector erasing, then malware can really hurt you by simply spewing a stream of "unused sector" messages to the disk.
posted by spacewrench at 9:37 AM on September 4, 2012
Windows XP doesn't support TRIM. Windows 7 does. Not sure about Vista.
Slightly off topic: From what I'm seeing online, it never actually made it to Vista native, but one of the workarounds discussed centered on Intel's SSD Optimizer which detects which files were deleted and communicates that data directly to the Intel SSD.
posted by samsara at 9:41 AM on September 4, 2012
Slightly off topic: From what I'm seeing online, it never actually made it to Vista native, but one of the workarounds discussed centered on Intel's SSD Optimizer which detects which files were deleted and communicates that data directly to the Intel SSD.
posted by samsara at 9:41 AM on September 4, 2012
VM guest additions could easily do the same thing as SSD Optimizer (which I imagine just involves monitoring the filesystem's free blocks map and generating appropriate TRIM-equivalent disk commands).
if the host VM could parse the guest's filesystem format
I don't think even Microsoft would be mad enough to do it that way.
You can't trust somebody else to do sector erasing.
Not on demand, and not while preserving data in adjacent sectors. The closest you could get with a flash EPROM if you had very low-level access to it would be to overprogram all bits in the sector to their non-erased state, since flash can only be erased a block at a time. I'm not aware of a flash drive that allows that level of access.
It's all a bit unnecessary anyway. Keeping your data away from the prying eyes of forensic investigators is a solved problem, and the solution has nothing to do with erasing unused disk blocks.
If you have sector erasing, then malware can really hurt you by simply spewing a stream of "unused sector" messages to the disk.
I don't really see how bogus trims are inherently more damaging than bogus writes. There might be forensic differences depending on the low-level details of what the receiving device does with them, but both are equally capable of causing massive filesystem corruption.
posted by flabdablet at 6:55 PM on September 4, 2012
if the host VM could parse the guest's filesystem format
I don't think even Microsoft would be mad enough to do it that way.
You can't trust somebody else to do sector erasing.
Not on demand, and not while preserving data in adjacent sectors. The closest you could get with a flash EPROM if you had very low-level access to it would be to overprogram all bits in the sector to their non-erased state, since flash can only be erased a block at a time. I'm not aware of a flash drive that allows that level of access.
It's all a bit unnecessary anyway. Keeping your data away from the prying eyes of forensic investigators is a solved problem, and the solution has nothing to do with erasing unused disk blocks.
If you have sector erasing, then malware can really hurt you by simply spewing a stream of "unused sector" messages to the disk.
I don't really see how bogus trims are inherently more damaging than bogus writes. There might be forensic differences depending on the low-level details of what the receiving device does with them, but both are equally capable of causing massive filesystem corruption.
posted by flabdablet at 6:55 PM on September 4, 2012
This thread is closed to new comments.
There is a command called TRIM that was introduced kinda recentlyish to tell flash disks that a sector is no longer needed— this lets the flash disk's internal block allocator so a better job. I believe VMs make use of it too, so maybe that's how Parallels is getting this info.
posted by hattifattener at 10:58 PM on September 3, 2012