What happens if the power fails during defrag?
September 10, 2006 12:43 PM Subscribe
I need to de-frag an external drive, but the power where I live is very flaky. Would a power failure cause me to lose my data?
I have a 200gb Maxtor Onetouch II and it's very fragmented and is starting to slow down noticeably. I want to de-frag it, but we average about 1-2 power cuts every day or so here in Kisumu and I'm worried id this happens during the process, it will ruin the drive.
Should I go ahead and de-frag, or am I certain to corrupt my drive in the event of power failure?
I have a 200gb Maxtor Onetouch II and it's very fragmented and is starting to slow down noticeably. I want to de-frag it, but we average about 1-2 power cuts every day or so here in Kisumu and I'm worried id this happens during the process, it will ruin the drive.
Should I go ahead and de-frag, or am I certain to corrupt my drive in the event of power failure?
You should definitely get a UPS (This model has worked out great for me, but shop around). If the power cuts out during normal hard drive operation you can easily lose data and this seems likely with 1-2 outages a day.
posted by null terminated at 12:49 PM on September 10, 2006
posted by null terminated at 12:49 PM on September 10, 2006
If the power cuts out during normal hard drive operation you can easily lose data
Out of interest - how does this happen?
posted by ed\26h at 1:26 PM on September 10, 2006
Out of interest - how does this happen?
posted by ed\26h at 1:26 PM on September 10, 2006
Best answer: Out of interest - how does this happen?
Well, suppose the drive is halfway through writing a sector and the power blips out before it has a chance to complete writing the sector.
Or suppose the OS has written new data to a file but hasn't yet updated the end-of-file pointer or the file allocation table to indicate that the data exists and/or that the blocks it is stored on are used.
Or suppose that an application has written your data to a temp file (which some programs do to make sure you don't lose your original file until the new file has been completely written) but has not yet deleted the original file and renamed the tempfile to the original name.
Or suppose you are using UNIX/Linux, which tends to use all free memory as a disk cache and may not actually write to disk until absolutely necessary, and the power goes out during the time you thought you saved a file and the occasion several minutes later when that data actually gets flushed to disk. (Journaling and application-requested cache flushes can minimize this problem, but there are still times when data hangs out in memory longer than you'd expect.)
Really, I don't understand why every desktop computer power supply doesn't have an integral battery backup. It is so much simpler to do this after the power supply, when everything is already DC, than to do it externally to the computer, which will involve converting power from AC to DC, then back to AC, then back to DC inside the computer. It wouldn't even need to be a big battery, certainly no bigger than a typical laptop battery. A minute is usually more than enough time to flush everything to disk that needs flushing.
posted by kindall at 1:50 PM on September 10, 2006 [1 favorite]
Well, suppose the drive is halfway through writing a sector and the power blips out before it has a chance to complete writing the sector.
Or suppose the OS has written new data to a file but hasn't yet updated the end-of-file pointer or the file allocation table to indicate that the data exists and/or that the blocks it is stored on are used.
Or suppose that an application has written your data to a temp file (which some programs do to make sure you don't lose your original file until the new file has been completely written) but has not yet deleted the original file and renamed the tempfile to the original name.
Or suppose you are using UNIX/Linux, which tends to use all free memory as a disk cache and may not actually write to disk until absolutely necessary, and the power goes out during the time you thought you saved a file and the occasion several minutes later when that data actually gets flushed to disk. (Journaling and application-requested cache flushes can minimize this problem, but there are still times when data hangs out in memory longer than you'd expect.)
Really, I don't understand why every desktop computer power supply doesn't have an integral battery backup. It is so much simpler to do this after the power supply, when everything is already DC, than to do it externally to the computer, which will involve converting power from AC to DC, then back to AC, then back to DC inside the computer. It wouldn't even need to be a big battery, certainly no bigger than a typical laptop battery. A minute is usually more than enough time to flush everything to disk that needs flushing.
posted by kindall at 1:50 PM on September 10, 2006 [1 favorite]
Kindall, I'd imagine heat and space are primary the issues with having a battery inside a power supply. It would be nice though.
Definately second getting a UPS. I would be more concerned if the power cut out while you were doing something important (like..ohh...saving a document). Like chrisamiller said, I'd suspect any good defrag program verifies data got moved to the right place before it deletes it.
posted by crypticgeek at 2:02 PM on September 10, 2006
Definately second getting a UPS. I would be more concerned if the power cut out while you were doing something important (like..ohh...saving a document). Like chrisamiller said, I'd suspect any good defrag program verifies data got moved to the right place before it deletes it.
posted by crypticgeek at 2:02 PM on September 10, 2006
Kindall: Thanks for such a detailed post! I thought that null was refering to "existing data" i.e.: that which is not, at time of power loss, being manipulated by any programs - which alarmed me a bit.
posted by ed\26h at 2:11 PM on September 10, 2006
posted by ed\26h at 2:11 PM on September 10, 2006
back your important data up first, then do the defrag ... if the power cuts out during the defrag, you won't have lost anything
posted by pyramid termite at 2:38 PM on September 10, 2006
posted by pyramid termite at 2:38 PM on September 10, 2006
Best answer: If your power is bad enough that you're scared of running a defrag, you need a battery backup.
Get a used APC Smart-UPS 1500. They're not quite as spiffy as the very newest and coolest units, but they're extremely reliable and very solidly engineered. You can often find them on EBay with brand new batteries for about $200. There is also a run cost involved... you'll need to replace the battery about every three years, which will cost about $120. If you run it down a lot, it will wear out faster; if you don't use it much, it'll last longer. Three years seems to be about the average.
In exchange for your $200 up front and $40/year, you can sit serenely through most power outages. You can usually expect 15 minutes or more of runtime, unless you have a really beastly load. You can run several computers or even a laser printer from this model... one of the few that can drive one. (many UPSes will die a shrieking, gibbering death when a laser printer fires up, because they pull so much power on warmup.)
They used to command premium pricing; these units mostly cost $1500 new, and were well worth it, at least to businesses. They are miles better than any consumer-grade UPS you'll find. To do better, you'd be looking at $800 minimum, even used. Siebert, I think, has some line-interactive (always-on, rather than standby) models that are better, but start at $800 in the used market and go up fast.
For $200, these used APCs are an incredible steal, tremendously underpriced. They're old, but they are GOOD.
posted by Malor at 2:53 PM on September 10, 2006
Get a used APC Smart-UPS 1500. They're not quite as spiffy as the very newest and coolest units, but they're extremely reliable and very solidly engineered. You can often find them on EBay with brand new batteries for about $200. There is also a run cost involved... you'll need to replace the battery about every three years, which will cost about $120. If you run it down a lot, it will wear out faster; if you don't use it much, it'll last longer. Three years seems to be about the average.
In exchange for your $200 up front and $40/year, you can sit serenely through most power outages. You can usually expect 15 minutes or more of runtime, unless you have a really beastly load. You can run several computers or even a laser printer from this model... one of the few that can drive one. (many UPSes will die a shrieking, gibbering death when a laser printer fires up, because they pull so much power on warmup.)
They used to command premium pricing; these units mostly cost $1500 new, and were well worth it, at least to businesses. They are miles better than any consumer-grade UPS you'll find. To do better, you'd be looking at $800 minimum, even used. Siebert, I think, has some line-interactive (always-on, rather than standby) models that are better, but start at $800 in the used market and go up fast.
For $200, these used APCs are an incredible steal, tremendously underpriced. They're old, but they are GOOD.
posted by Malor at 2:53 PM on September 10, 2006
There are, in fact, internal PC power supplies with built-in UPSs, or at least there were last time I looked.
As for how power outages can break, specifically, defragging: you defrag by copying file blocks from one place to another, and then updating the pointers that point to the blocks.
The pinch point is when you rewrite the pointers: if the power goes out during the write, you may scribble on said pointers, crapping up the file (that's a technical term).
posted by baylink at 3:16 PM on September 10, 2006
As for how power outages can break, specifically, defragging: you defrag by copying file blocks from one place to another, and then updating the pointers that point to the blocks.
The pinch point is when you rewrite the pointers: if the power goes out during the write, you may scribble on said pointers, crapping up the file (that's a technical term).
posted by baylink at 3:16 PM on September 10, 2006
Oh one additional note: you'll pay a lot on shipping a UPS. They are extremely heavy. If you see high shipping costs on these on EBay, this is one of the few cases where they're not ripping you off. $50 shipping is not at all unreasonable.
posted by Malor at 3:30 PM on September 10, 2006
posted by Malor at 3:30 PM on September 10, 2006
Response by poster: OK, I went to bed as soon as I posted this and it now looks like I withheld a vital bit of info (yes, I am quite the dumb ass)...
The computer is a laptop, so if the power outs, it continues to run for up to 3 hours - the power usually comes back within that time. However, the drive runs of mains, so if the power outs, it stops about 1 second later.
Is the suggestion that I put the drive on a UPS, the computer on UPS or both? If I put the drive on UPS, might the problem still arise if I don't make it back to the computer in time to pause the defrag?
Thanks for the tips so far. Just as a note, it's impossible to run a desktop without a UPS here. They can be picked up just about anywhere relatively cheaply, so thankfully I won't need to ship something that weighs half a ton all the way around the globe!
posted by davehat at 11:12 PM on September 10, 2006
The computer is a laptop, so if the power outs, it continues to run for up to 3 hours - the power usually comes back within that time. However, the drive runs of mains, so if the power outs, it stops about 1 second later.
Is the suggestion that I put the drive on a UPS, the computer on UPS or both? If I put the drive on UPS, might the problem still arise if I don't make it back to the computer in time to pause the defrag?
Thanks for the tips so far. Just as a note, it's impossible to run a desktop without a UPS here. They can be picked up just about anywhere relatively cheaply, so thankfully I won't need to ship something that weighs half a ton all the way around the globe!
posted by davehat at 11:12 PM on September 10, 2006
Best answer: If it's a laptop a UPS won't gain you much other than more carefully regulating the power (which might not be a bad idea from the sounds of it).
I would do two things:
1) Get a defrag application that lets you do incremental defrags (ie: just folders or individual files). This will let you defrag the thing in smaller segments that should run quickly so you don't have to worry about power running out before its done.
2) If your operating system/computer supports it, set your laptop to hibernate when it gets low on power. (Windows XP: Display Properties -> Screen Saver -> Power) That way it will complete any disk write actions before it goes unconscious, and you shouldn't get anything corrupted.
posted by Ookseer at 2:13 AM on September 11, 2006
I would do two things:
1) Get a defrag application that lets you do incremental defrags (ie: just folders or individual files). This will let you defrag the thing in smaller segments that should run quickly so you don't have to worry about power running out before its done.
2) If your operating system/computer supports it, set your laptop to hibernate when it gets low on power. (Windows XP: Display Properties -> Screen Saver -> Power) That way it will complete any disk write actions before it goes unconscious, and you shouldn't get anything corrupted.
posted by Ookseer at 2:13 AM on September 11, 2006
how do you know that a fragmented hard drive is causing your slow performance?
in my experience its usually a software/os issue or impending hardware death that motivates a person to defrag, wearing out their drive in its last days.
posted by yeahyeahyeahwhoo at 6:27 AM on September 11, 2006
in my experience its usually a software/os issue or impending hardware death that motivates a person to defrag, wearing out their drive in its last days.
posted by yeahyeahyeahwhoo at 6:27 AM on September 11, 2006
Best answer: Well, a UPS will let you run the external drive through power outages, which is a fairly big gain I guess ;)
As far as data loss goes; every vaguely modern OS uses write-behind caching; writes go to buffers in system memory, and those buffers get written out when an application or the OS desires it (which can be based on anything; when the buffer reaches a certain size or age, or just when it happens to be convenient based on other disk activity).
Additionally, hard disks have their own buffers where writes from the OS get delayed and reordered before the drive writes to the actual media. This is the big one; OS's and filesystems tend to go to a lot of effort to order writes in a fairly safe manner; one write might put data on a free point on the disk, and another following it might update a pointer on the filesystem to make that data "active".
This is great, except notice I said "writes get .. reordered" earlier. If the disk decides to do your pointer update before writing the actual data, and then power gets cut, your filesystem now has a pointer to garbage. And that pointer potentially used to point to the data that was being copied, and is now gone. Doh.
Now, in theory, disks (are supposed to) respond to "flush cache" commands that are (supposed to be) inserted by the OS at these crucial points, and there are more advanced systems gaining traction that expand on the idea (this is where SATA's NCQ comes in; Command Queuing where operations are tagged with the order they need to run in). In practice, it's hard to get this stuff right, even when they're a top priority in your expensive enterprise-ready OS/database/RAID controller.
So, what does that mean for you? Well, if you want to be mostly safe without depending on a UPS, you probably want to turn off the on-disk write cache. In Windows 2k/XP, you can use Device Manager -> Disk drives -> (drive context menu, Properties) -> Policies to disable it and the OS write cache. In exchange for your relative safety you can expect an order of magnitude drop in write performance.
posted by Freaky at 7:18 AM on September 11, 2006
As far as data loss goes; every vaguely modern OS uses write-behind caching; writes go to buffers in system memory, and those buffers get written out when an application or the OS desires it (which can be based on anything; when the buffer reaches a certain size or age, or just when it happens to be convenient based on other disk activity).
Additionally, hard disks have their own buffers where writes from the OS get delayed and reordered before the drive writes to the actual media. This is the big one; OS's and filesystems tend to go to a lot of effort to order writes in a fairly safe manner; one write might put data on a free point on the disk, and another following it might update a pointer on the filesystem to make that data "active".
This is great, except notice I said "writes get .. reordered" earlier. If the disk decides to do your pointer update before writing the actual data, and then power gets cut, your filesystem now has a pointer to garbage. And that pointer potentially used to point to the data that was being copied, and is now gone. Doh.
Now, in theory, disks (are supposed to) respond to "flush cache" commands that are (supposed to be) inserted by the OS at these crucial points, and there are more advanced systems gaining traction that expand on the idea (this is where SATA's NCQ comes in; Command Queuing where operations are tagged with the order they need to run in). In practice, it's hard to get this stuff right, even when they're a top priority in your expensive enterprise-ready OS/database/RAID controller.
So, what does that mean for you? Well, if you want to be mostly safe without depending on a UPS, you probably want to turn off the on-disk write cache. In Windows 2k/XP, you can use Device Manager -> Disk drives -> (drive context menu, Properties) -> Policies to disable it and the OS write cache. In exchange for your relative safety you can expect an order of magnitude drop in write performance.
posted by Freaky at 7:18 AM on September 11, 2006
I thought that null was refering to "existing data" i.e.: that which is not, at time of power loss, being manipulated by any programs - which alarmed me a bit.
This could actually happen too, though indirectly, if a program has just finished writing another file in the same directory and the computer happens to be updating the directory block when the power fails.
posted by kindall at 10:17 AM on September 11, 2006
This could actually happen too, though indirectly, if a program has just finished writing another file in the same directory and the computer happens to be updating the directory block when the power fails.
posted by kindall at 10:17 AM on September 11, 2006
This thread is closed to new comments.
Check the documentation for whatever defrag program you're using, though.
posted by chrisamiller at 12:45 PM on September 10, 2006