Formatting external HD
December 13, 2019 3:28 PM Subscribe
I have a My Passport from WD that has suddenly gotten some bad sectors and corrupted directory. Will formatting it fix all that?
Response by poster: It's been utterly sedentary and stable since I got it, but I took it to the library and it came back apparently messed up. I have all backed up. Tried the WD fix tool, then just formatted. Don't know how I will be able to tell bad sectors still exist till ... till they make themselves known.
posted by noelpratt2nd at 3:40 PM on December 13, 2019
posted by noelpratt2nd at 3:40 PM on December 13, 2019
Do you have a Mac? There is a "check disk" utility, probably something similar exists on Windows too.
posted by mekily at 3:45 PM on December 13, 2019
posted by mekily at 3:45 PM on December 13, 2019
It depends on what corrupted those sectors to begin with. If the errors were caused by some transient issue that corrupted the data, then sure, a format should fix those and you can get on with your life, but it could easily be the first signs of the drive's end-of-life. I'd at least run a disk check or CHKDSK on the drive and see if it still reports bad sectors. If after running it a couple of times it keeps finding more bad sectors, I'd consider the drive a lost cause.
As it is, though, I definitely wouldn't trust data to the drive without a reliable backup (though you shouldn't even trust a reliable drive without a backup anyway; drives can fail at any time for a multitude of reasons and don't care if it takes the only copy of your data with it).
posted by Aleyn at 4:27 PM on December 13, 2019 [2 favorites]
As it is, though, I definitely wouldn't trust data to the drive without a reliable backup (though you shouldn't even trust a reliable drive without a backup anyway; drives can fail at any time for a multitude of reasons and don't care if it takes the only copy of your data with it).
posted by Aleyn at 4:27 PM on December 13, 2019 [2 favorites]
If it were me, I'd just get a new drive. They're not that expensive, and you don't want to risk losing data on a possibly-failing unit.
posted by alex1965 at 4:31 PM on December 13, 2019 [6 favorites]
posted by alex1965 at 4:31 PM on December 13, 2019 [6 favorites]
Best answer: Also, Nthing that formatting will erase the disk; if that's a problem, you can use the CHKDSK command to fix issues without a format. If you do decide to format the disk, you need to uncheck the Quick format option for it to find bad sectors, but I suspect that CHKDSK will give you better results anyway.
posted by Aleyn at 4:35 PM on December 13, 2019 [1 favorite]
posted by Aleyn at 4:35 PM on December 13, 2019 [1 favorite]
Back it up. Before you do anything.
Drives can be repaired to bypass faulty sectors, but when one starts to have faulty sectors there is usually a cause (particles in the drive, for example) and things get worse over time. Once backed up, replace the drive. I'd recommend a solid state drive, more reliable than a physically spinning drive and very fast. The prices have dropped greatly since they were first introduced.
But no matter what,
Back it up now.
posted by tmdonahue at 5:08 PM on December 13, 2019 [1 favorite]
Drives can be repaired to bypass faulty sectors, but when one starts to have faulty sectors there is usually a cause (particles in the drive, for example) and things get worse over time. Once backed up, replace the drive. I'd recommend a solid state drive, more reliable than a physically spinning drive and very fast. The prices have dropped greatly since they were first introduced.
But no matter what,
Back it up now.
posted by tmdonahue at 5:08 PM on December 13, 2019 [1 favorite]
I have all backed up.
posted by noelpratt2nd at 3:40 PM on December 13
posted by under_petticoat_rule at 5:13 PM on December 13, 2019
posted by noelpratt2nd at 3:40 PM on December 13
posted by under_petticoat_rule at 5:13 PM on December 13, 2019
Go to the WD website and download WD Utilities. Install it and there are three scans you can do. The first just checks for S.M.A.R.T errors, the second runs a quick scan for bad sectors, the third runs a full scan (can take hours if it's a large drive). Start with the first one and run each sequentially. If any of these throw errors, I would just write off the drive and get a new one--you don't want to be storing data on a ticking time bomb.
posted by reformedjerk at 5:43 PM on December 13, 2019 [1 favorite]
posted by reformedjerk at 5:43 PM on December 13, 2019 [1 favorite]
Seconding a full scan with a S.M.A.R.T.-aware program, or something else that reads (or better yet, non-destructively reads then writes) all sectors on disk.
posted by Bangaioh at 5:53 PM on December 13, 2019
posted by Bangaioh at 5:53 PM on December 13, 2019
Nthing that if what's on it is at all important to you to replace it if there's bad sectors.
posted by Candleman at 7:45 PM on December 13, 2019
posted by Candleman at 7:45 PM on December 13, 2019
Response by poster: I'm running the longer format right now. Have a duplicate device with all on it. I guess it was hooking it up to one of the library's computers to burn some CDs that somehow did it. Never again.
posted by noelpratt2nd at 8:04 PM on December 13, 2019
posted by noelpratt2nd at 8:04 PM on December 13, 2019
Best answer: I guess it was hooking it up to one of the library's computers to burn some CDs that somehow did it.
That is very unlikely. Bad sectors are intrinsic to the drive, not the computer it's plugged into. You should be cautious about using a drive on a public computer but not because of this.
posted by Candleman at 9:52 PM on December 13, 2019 [2 favorites]
That is very unlikely. Bad sectors are intrinsic to the drive, not the computer it's plugged into. You should be cautious about using a drive on a public computer but not because of this.
posted by Candleman at 9:52 PM on December 13, 2019 [2 favorites]
Seconding that it is extremely unlikely for bad sectors to occur because you plugged the computer into an untrusted computer. The timing is almost certainly a coincidence.
posted by Aleyn at 10:30 PM on December 13, 2019
posted by Aleyn at 10:30 PM on December 13, 2019
I’ve had **two** WD mypassport drives (bought at the same time) fail in exactly this way with no warning and no evident cause and light usage. That was a few years ago and I’ll never buy their consumer grade storage products again. Shit.
posted by spitbull at 4:10 AM on December 14, 2019 [1 favorite]
posted by spitbull at 4:10 AM on December 14, 2019 [1 favorite]
it is extremely unlikely for bad sectors to occur because you plugged the computer into an untrusted computer. The timing is almost certainly a coincidence.
Well, there is one plausible mechanism I can think of: if the drive was being written to, and the power supply pins on the USB port it was plugged into were dirty and/or worn, then it could have seen its power supply voltage go out of spec during writes. This might well result in bad writes, and if some of those bad writes happened to occur while writing out modifications to a directory, directory corruption.
If you were only reading files from the drive, this failure mode is less likely; but Windows does sometimes update access times on files as it reads them (search this page for "last access time") and does not have anything like Unix's read-only mount option to prevent it from doing that. The access times are held in Master File Table and directory entries, so those would be what would get corrupted if the power supply brownout hypothesis is correct.
Reformatting a modern spinny drive doesn't actually recreate the sector address marks the way it used to back in the early days when hard disk capacities were still measured in megabytes; all that a long format can generally be relied upon to do is write zeroes out to the data portion of every sector within the partition being formatted. A quick format won't even do that, merely write out a new set of filesystem structures for an empty filesystem. So depending on exactly how a supply brownout has caused a drive's write operations to fail, there's some possibility that the bad write(s) might leave some of the drive's sectors in permanently unusable condition even though there's nothing physically wrong with the surface of the medium.
If that were my drive I'd be keeping an eye on the raw value of the Reallocated Sector Count attribute in its SMART info, and considering the drive untrustworthy if that value was more than a few tens and/or growing by more than one per month. Right after doing a long format is a good time to check this attribute, because the sector writes that the format operation does will prompt the drive to move any sectors that were pending reallocation beforehand to their new homes in the spare zone.
posted by flabdablet at 5:27 AM on December 14, 2019 [3 favorites]
Well, there is one plausible mechanism I can think of: if the drive was being written to, and the power supply pins on the USB port it was plugged into were dirty and/or worn, then it could have seen its power supply voltage go out of spec during writes. This might well result in bad writes, and if some of those bad writes happened to occur while writing out modifications to a directory, directory corruption.
If you were only reading files from the drive, this failure mode is less likely; but Windows does sometimes update access times on files as it reads them (search this page for "last access time") and does not have anything like Unix's read-only mount option to prevent it from doing that. The access times are held in Master File Table and directory entries, so those would be what would get corrupted if the power supply brownout hypothesis is correct.
Reformatting a modern spinny drive doesn't actually recreate the sector address marks the way it used to back in the early days when hard disk capacities were still measured in megabytes; all that a long format can generally be relied upon to do is write zeroes out to the data portion of every sector within the partition being formatted. A quick format won't even do that, merely write out a new set of filesystem structures for an empty filesystem. So depending on exactly how a supply brownout has caused a drive's write operations to fail, there's some possibility that the bad write(s) might leave some of the drive's sectors in permanently unusable condition even though there's nothing physically wrong with the surface of the medium.
If that were my drive I'd be keeping an eye on the raw value of the Reallocated Sector Count attribute in its SMART info, and considering the drive untrustworthy if that value was more than a few tens and/or growing by more than one per month. Right after doing a long format is a good time to check this attribute, because the sector writes that the format operation does will prompt the drive to move any sectors that were pending reallocation beforehand to their new homes in the spare zone.
posted by flabdablet at 5:27 AM on December 14, 2019 [3 favorites]
For what spinning hard drive storage costs now I treat them as consumables. First sign of trouble they just get replaced. Data is precious and drives are cheap.
posted by spitbull at 9:18 AM on December 14, 2019 [1 favorite]
posted by spitbull at 9:18 AM on December 14, 2019 [1 favorite]
The way that modern spinning hard drives work (well, this is at least five years out of date but I doubt it's changed) is that the onboard controller has a pool of spare sectors which transparently replace sectors that have errors. Until these are exhausted, no bad sectors are reported. When you start seeing bad sectors, the disk is pretty much hosed and should be replaced.
I would immediately copy anything you value off the drive.
posted by Gilgamesh's Chauffeur at 11:26 AM on December 14, 2019
I would immediately copy anything you value off the drive.
posted by Gilgamesh's Chauffeur at 11:26 AM on December 14, 2019
Until these are exhausted, no bad sectors are reported. When you start seeing bad sectors, the disk is pretty much hosed and should be replaced.
Not quite right. Sectors that read bad don't immediately get substituted for from the spare pool; that only happens the next time they're rewritten, because until then the drive doesn't know what their contents ought to be.
Instead, the drive will add the unreadable sectors to a list of sectors pending reallocation, and continue to report read errors for them until it's asked to write to them. This is why drives maintain separate Reallocated Sector and Current Pending Sector counts in their SMART tables.
Sectors that make the drive need to retry several times before being able to get a good read will typically also get added to the list of sectors pending reallocation, even though the operating system will not see the reads as bad in this case, merely very slow. If you've got a Windows box that has started running ridiculously slow, and the hard disk access light seems to be staying on longer than usual, and you put your ear on the case while it's running and hear a disk drive going tick... tick... tick... while the light stays on, and something like PassMark DiskCheckup shows that the drive's own Current Pending Sector count is nonzero, that's probably what's going on.
Cloning the drive to a new one is the safest way I know of to fix this, but sometimes the issue resolves spontaneously on a Windows update or reinstallation or even a defrag as Windows rewrites the affected sectors. I'm pretty firmly convinced that the dramatic speed gains that some people have seen some of the time after doing one of these disk-intensive maintenance tasks are due to this underlying mechanism, as well as being the seeds of the prevailing folk wisdom that Windows "just gets slow over time" and needs one or more of these things done to it, generally at some expense, to spruce it up again. The fact that a sector-by-sector clone of an affected Windows installation to a new drive will usually also bring it back to something close to original speed tells me that more failing disk drives remain in service than probably should.
The bad-sector handling facilities built into most operating systems, Windows included, were designed before drives did this kind of automatic spare replacement internally, and they don't allow for it at all. If you've got a drive with unreadable sectors on it and you run a Windows bad-sectors scan over it, Windows will add all the unreadable sectors to the internal $BadClus:Bad file it maintains on that drive, making them unavailable for further use by other files. If you clone the drive to a new one, $BadClus gets cloned along with everything else and you end up with sectors on the new drive that will never be used even though there's nothing wrong with them.
Most drives, when asked to rewrite a sector that's in their Current Pending Sector list, will first attempt to write it to its original physical location and then immediately read it back internally to verify the write; only if that re-read fails will they then go on to rewrite it to a spare sector and mark it as reallocated. This behaviour can occasionally be spotted by keeping an eye on the Current Pending Sector and Reallocated Sector counts in SMART: if you see Current Pending Sector going from zero to nonzero and back but Reallocated Sectors is not increasing, that's what's happening. It also explains how it's possible to start with quite a high Current Pending Sector count, do a long reformat (which writes zeroes to every sector) and end up with Current Pending Sector and Reallocated Sector counts both at zero.
If you see a drive doing this, it's a fair bet that the bad sectors in question were caused by transient conditions like physical shocks or power supply glitches during their original writes and that there's nothing wrong with the drive per se. This could save you a few hundred bucks on unnecessary drive replacements. But it doesn't justify failing to maintain adequate backups.
I would immediately copy anything you value off the drive.
That needs to be done regularly, as a matter of course and not merely as a crisis response. It needs to be done regardless of whether your drives are old or new, reliable or flaky, from brands you like or don't, spinny or SSD. Just make backups. Regularly. Test them regularly, too; backups that never get tested are pointless backup theatre.
Digital information doesn't really exist until you can prove you can put your hands on at least two copies.
posted by flabdablet at 2:44 AM on December 15, 2019 [2 favorites]
Not quite right. Sectors that read bad don't immediately get substituted for from the spare pool; that only happens the next time they're rewritten, because until then the drive doesn't know what their contents ought to be.
Instead, the drive will add the unreadable sectors to a list of sectors pending reallocation, and continue to report read errors for them until it's asked to write to them. This is why drives maintain separate Reallocated Sector and Current Pending Sector counts in their SMART tables.
Sectors that make the drive need to retry several times before being able to get a good read will typically also get added to the list of sectors pending reallocation, even though the operating system will not see the reads as bad in this case, merely very slow. If you've got a Windows box that has started running ridiculously slow, and the hard disk access light seems to be staying on longer than usual, and you put your ear on the case while it's running and hear a disk drive going tick... tick... tick... while the light stays on, and something like PassMark DiskCheckup shows that the drive's own Current Pending Sector count is nonzero, that's probably what's going on.
Cloning the drive to a new one is the safest way I know of to fix this, but sometimes the issue resolves spontaneously on a Windows update or reinstallation or even a defrag as Windows rewrites the affected sectors. I'm pretty firmly convinced that the dramatic speed gains that some people have seen some of the time after doing one of these disk-intensive maintenance tasks are due to this underlying mechanism, as well as being the seeds of the prevailing folk wisdom that Windows "just gets slow over time" and needs one or more of these things done to it, generally at some expense, to spruce it up again. The fact that a sector-by-sector clone of an affected Windows installation to a new drive will usually also bring it back to something close to original speed tells me that more failing disk drives remain in service than probably should.
The bad-sector handling facilities built into most operating systems, Windows included, were designed before drives did this kind of automatic spare replacement internally, and they don't allow for it at all. If you've got a drive with unreadable sectors on it and you run a Windows bad-sectors scan over it, Windows will add all the unreadable sectors to the internal $BadClus:Bad file it maintains on that drive, making them unavailable for further use by other files. If you clone the drive to a new one, $BadClus gets cloned along with everything else and you end up with sectors on the new drive that will never be used even though there's nothing wrong with them.
Most drives, when asked to rewrite a sector that's in their Current Pending Sector list, will first attempt to write it to its original physical location and then immediately read it back internally to verify the write; only if that re-read fails will they then go on to rewrite it to a spare sector and mark it as reallocated. This behaviour can occasionally be spotted by keeping an eye on the Current Pending Sector and Reallocated Sector counts in SMART: if you see Current Pending Sector going from zero to nonzero and back but Reallocated Sectors is not increasing, that's what's happening. It also explains how it's possible to start with quite a high Current Pending Sector count, do a long reformat (which writes zeroes to every sector) and end up with Current Pending Sector and Reallocated Sector counts both at zero.
If you see a drive doing this, it's a fair bet that the bad sectors in question were caused by transient conditions like physical shocks or power supply glitches during their original writes and that there's nothing wrong with the drive per se. This could save you a few hundred bucks on unnecessary drive replacements. But it doesn't justify failing to maintain adequate backups.
I would immediately copy anything you value off the drive.
That needs to be done regularly, as a matter of course and not merely as a crisis response. It needs to be done regardless of whether your drives are old or new, reliable or flaky, from brands you like or don't, spinny or SSD. Just make backups. Regularly. Test them regularly, too; backups that never get tested are pointless backup theatre.
Digital information doesn't really exist until you can prove you can put your hands on at least two copies.
posted by flabdablet at 2:44 AM on December 15, 2019 [2 favorites]
You've gotten increasingly detailed answers on how to decide if the disk is going bad. My simple take on it: reformat, and if you see a failure like this again throw the disk out.
Just a theory for how your disk may have gotten corrupted: did you unplug it without politely unmounting it first? I've corrupted known-good hard drives that way several times in Windows 10, even when I thought the disk had been idle for a long time and nothing could be possibly waiting to be written to it. The file system is just not very resilient to being unplugged. The repair tool usually fixes it but not always, but it has nothing to do with hardware failure.
posted by Nelson at 7:30 AM on December 15, 2019
Just a theory for how your disk may have gotten corrupted: did you unplug it without politely unmounting it first? I've corrupted known-good hard drives that way several times in Windows 10, even when I thought the disk had been idle for a long time and nothing could be possibly waiting to be written to it. The file system is just not very resilient to being unplugged. The repair tool usually fixes it but not always, but it has nothing to do with hardware failure.
posted by Nelson at 7:30 AM on December 15, 2019
This thread is closed to new comments.
But it's also possible that it's a physical error, or wear on the disk. In that case, no.
posted by humboldt32 at 3:34 PM on December 13, 2019 [1 favorite]