Windows, why?

June 4, 2019 11:15 AM

Why is it in that in this the year of our Lord 2019 in Windows I cannot move or rename a file or folder while it is open in an app? I've been able to do this in the Mac OSes for so long I honestly don't remember a time when I couldn't, so clearly this is a task that an OS can routinely be capable of. For Windows not to be able to do this must represent some kind of choice, or some kind of negligence. (E.g., a literal answer like "the system might thereby have conflicting information about where the file/folder is stored" is not what I'm looking for, because this is not 1988 and I know this problem is solvable.) It is so inconvenient not to be able to do this. Perhaps an explanation will help me reconcile myself to it. (Background knowledge level: modest to moderate.)
posted by praemunire to Computers & Internet (15 answers total)

Software person here not to answer directly, but to offer another option: fixing something (whether it's a defect or a surprise annoyance) is usually some outcome of effort required x criticality of problem. It's possible that this was not deliberate, but it's not bad enough that anyone has fixed it.
posted by chesty_a_arthur at 11:23 AM on June 4, 2019 [1 favorite]

Response by poster: Fair enough, but this is something that affects ordinary users on a regular basis in a quite visible way and has done so for decades now, so you'd think it would have a reasonable amount of priority. (But you may well turn out to be right, in which case it goes in the "negligence" bucket.)
posted by praemunire at 11:40 AM on June 4, 2019 [1 favorite]

Basically, Windows and Mac make different assumptions about how important it is for only one process to be editing the file: https://en.wikipedia.org/wiki/File_locking .
posted by sagc at 11:43 AM on June 4, 2019 [9 favorites]

I grew up using Macs before switching to PCs, and may I just say I feel your pain. The worst is when you've already closed or even deleted a file and the folder it was stored in doesn't "realize" it until you do something drastic like restart the whole machine - I've had dozens of instances where I wasn't permitted to delete a completely empty, closed folder (even after checking for hidden files). Totally infuriating.

After skimming the file locking link above, I did a quick search for "disable file locking Windows" and there seemed to be some options - I probably won't look into it further because I'm mostly using a work machine that I can't fiddle with much, but you should have a look!
posted by showbiz_liz at 11:52 AM on June 4, 2019 [3 favorites]

Best answer: The specific culprit in Windows is the dwShareMode parameter of the CreateFile function, the system call for accessing files. Windows allows programs that successfully open a file to unilaterally control whether any other program can read or write on that file at the same time. Unfortunately, this means that any lazy and irresponsible developers can just hog any file they open to themselves, since that's always safer and more convenient from their perspective than giving anyone else access and having to worry about keeping data consistent. And if Microsoft were to try to remove this functionality from one of the most basic functions of the operating system, it would almost certainly break countless programs. So a solution to this problem would be very difficult indeed.
posted by J.K. Seazer at 11:59 AM on June 4, 2019 [12 favorites]

To phrase sagc's statement a little differently, imagine you're working in a file called foo.txt from a network share in a multiuser environment. Someone else, without telling you, changes the name to bar.txt. Is having to try to figure out what the file is now called desired behavior? Microsoft has long viewed business users as the key users to cater to.
posted by Candleman at 12:02 PM on June 4, 2019

@Candleman in macOS you can do that and the first user just has to press save and their changes will save into bar.txt and the application will display the new name.
posted by Lanark at 12:12 PM on June 4, 2019

Response by poster: Not to thread-sit, but since you asked:

Is having to try to figure out what the file is now called desired behavior? Microsoft has long viewed business users as the key users to cater to.

Yes, and I am, in this context, the equivalent of a business/institutional user.
posted by praemunire at 12:13 PM on June 4, 2019

Best answer: And if Microsoft were to try to remove this functionality from one of the most basic functions of the operating system, it would almost certainly break countless programs. So a solution to this problem would be very difficult indeed.

This is basically it. Apple has had no problem in changing fundamentals of their OS multiple times, because people will suck it up. Microsoft has an enormous ecosystem of businesses that run on invisible software that was written by someone's cousin 20 years ago using an undocumented race condition which will take down an entire bank if the order that windows are painted changes. Raymond Chen (a Microsoft developer for decades) has a blog talking about ridiculous efforts to maintain backwards compatibility and to work around bugs in hardware drivers so that windows will run on any hardware instead of building their own machines.
posted by the agents of KAOS at 12:30 PM on June 4, 2019 [18 favorites]

Also I think that
Is having to try to figure out what the file is now called desired behavior?
in this context, "try" means "attempt, and possibly destroy everything if we fail, which is totally a thing that could happen, or maybe nothing happens, who knows?"

So the question really is : gamble with the integrity of your data? and the answer is usually, (correctly imo) "no thankyou"
posted by some loser at 1:20 PM on June 4, 2019 [1 favorite]

Response by poster: in this context, "try" means "attempt, and possibly destroy everything if we fail, which is totally a thing that could happen, or maybe nothing happens, who knows?"

So the question really is : gamble with the integrity of your data? and the answer is usually, (correctly imo) "no thankyou"

I do understand what it means. The thing is, though, this is a solved problem for other systems. That your system has so much shitty legacy code that you can't perform a simple user task without imperiling file integrity doesn't mean it's not embarrassing that you can't perform it when others can.

KAOS and Seazer have gotten at what I wanted to know.
posted by praemunire at 1:51 PM on June 4, 2019 [1 favorite]

I keep around a utility called unlocker for these occasions.
posted by Obscure Reference at 2:34 PM on June 4, 2019 [1 favorite]

To elaborate on the philosophical differences that sagc mentioned:

Windows has mandatory file locking; as J. K. Seazer pointed out, the low-level API for creating files on Windows includes parameters for setting up file locking. The Windows kernel records and enforces locks at an operating system level.

Mac OS X is derived from UNIX. UNIX generally only has discretionary file locking; the kernel does not enforce any file locking scheme upon any program. File locking is instead on an opt-in basis per-program; a separate API must be used to lock a file after calling the API that opens an file, and the locks are only respected by programs that have opted-in.

On a UNIX system, the name of the file only matters when you initially open it. After it is opened, the kernel gives your program a file descriptor number; all subsequent operations on the file reference the file descriptor rather than the name. It doesn't matter if the file gets renamed or even deleted once you have a file descriptor; the descriptor remains open and valid until you close it.
posted by jordemort at 2:40 PM on June 4, 2019 [7 favorites]

Response by poster: "On a UNIX system, the name of the file only matters when you initially open it. After it is opened, the kernel gives your program a file descriptor number; all subsequent operations on the file reference the file descriptor rather than the name. It doesn't matter if the file gets renamed or even deleted once you have a file descriptor; the descriptor remains open and valid until you close it."

I presume this also explains why apps can't find files in recent-files menus once they've been moved.
posted by praemunire at 3:48 PM on June 4, 2019

There's more to it than just lazy developers choosing a poor dwShareMode.

Firstly: a lot of software doesn't use the direct Win32 CreateFile API, it uses the C standard library's fopen() function. The MS C runtime implements this function by calling CreateFile, and its choice of dwShareMode is, of course, only FILE_SHARE_READ. (There are other problems with the C runtime's implementation, notably around Unicode file names.)

So if a developer wants to make a cross-platform tool which can share open files, it needs some platform-specific branching code to call the Win32 API instead of the standard library. And because the handle types you get from each aren't compatible that usually means you're writing a whole file-access layer to abstract the difference between Windows and everywhere-else. This is enough of a pain that many developers can't be bothered.

Secondly: even if you open a file with FILE_SHARE_DELETE, you don't actually get the (‘POSIX’) semantics you want. If you delete a file that some process has a shared-delete handle on, it doesn't really go away, it just gets marked ‘pending deletion’. The process with the open handle can still access the file, although there's not much point since as soon as that last handle closes the file will be properly deleted.

But! In the meantime the directory entry still exists. You can't open it (you'll get a misleading ACCESS_DENIED error just like all the other sharing violations), you can't create a new file with the same name, and you can't delete the directory that contains it. Unix-style inodes give us the ability to have two different files open with the same name, Windows omits this feature, and it's turns out it's necessary to make a bunch of file management tasks work well, most notably run-time code updates.

Windows has built on that original sin of omission ever since, with the layers of workarounds in Windows Installer to get around the inability to replace in-use files, with flaky old PendingFileRenameOperations, endless reboots, and impenetrable Windows Update failure code. Even if you somehow fixed Windows to abide by POSIX semantics now, it would be decades of refactoring to clean up the unfortunate implications of that one little design choice.

It probably wasn't obvious when the Windows NT filesystem architecture was being designed (to replace the horrors of the DOS filesystem), that something like an inode was the Right Thing. I don't know what the designers' goals were, but the idea of replacing system files whilst running probably wasn't something they had in mind — this was long before we took it for granted that any connected PC would need constant patching.

It's not a stupid idea that you shouldn't be able to update a file while an app is using it — there are certainly plenty of scenarios on Mac or Linux where applications made up of multiple files can get deeply confused when the files they open from disc don't match the code version currently running. It just turns out that this problem is, in practice, much smaller than the pessimistic-locking woe that Windows has brought upon itself. I don't think we could have known that in advance, but at this late stage I can't see any easy path for Microsoft to get out of this hole.
posted by BobInce at 11:14 AM on June 5, 2019 [5 favorites]