Why would a SATA drive work in a SATA>USB dock, but not via SATA cable?
February 11, 2013 4:10 PM Subscribe
I've got a 3tb Seagate SATA drive, formatted NTFS, that I bought along with this Thermaltake SATA docking station. It's the kind where the naked drive just kinda chunks down vertically into the dock, which then connects to the PC via USB. Works great in the dock, but at a certain point, I thought I should just install the drive, not freak out about accidentally bumping it, and I'd get better throughput via sata cable to my mobo. Unfortunately, when I connect via sata, the disk shows up as uninitialized in Windows (Win7 x64)
If I plug the drive back into the dock, it mounts just fine and there's all my files. I am very confused.
Back when 3TB hard drives were first released, there was a schism between the old (less than 2.1TB) drives which used 512k sectors, and the newer 2.1TB or higher drives which, because of their larger size, used 4k sectors. Windows did not have support for these newer drives.
After there was a slew of confusion initially when drives failed to mount, or where folks had to set up the drive with multiple partitions in order to access the full storage capabilities of the drives, manufacturers started selling 3TB drives that could fool the host into thinking they were standard 512k sector drives and could be widely used without special formatting or tools. However, not every drive has that functionality.
If you have one of the earlier 3TB drives that does not have a 512k compatibility mode, it is possible the controller in the Thermaltake dock can support 4k drives, but Windows 7 in conjunction with your SATA controller cannot access the full 3TB "out of the box." (Support is present by default in Windows 8.) If you can provide a model number, we might be able to help further with specifics.
Seagate has more information on this issue with 3TB (and higher) drives here.
posted by eschatfische at 4:28 PM on February 11, 2013 [2 favorites]
After there was a slew of confusion initially when drives failed to mount, or where folks had to set up the drive with multiple partitions in order to access the full storage capabilities of the drives, manufacturers started selling 3TB drives that could fool the host into thinking they were standard 512k sector drives and could be widely used without special formatting or tools. However, not every drive has that functionality.
If you have one of the earlier 3TB drives that does not have a 512k compatibility mode, it is possible the controller in the Thermaltake dock can support 4k drives, but Windows 7 in conjunction with your SATA controller cannot access the full 3TB "out of the box." (Support is present by default in Windows 8.) If you can provide a model number, we might be able to help further with specifics.
Seagate has more information on this issue with 3TB (and higher) drives here.
posted by eschatfische at 4:28 PM on February 11, 2013 [2 favorites]
It's almost certainly an incompatibility between the drive and your mobo's SATA chipset. Without more specific information we can't say what, exactly, is causing the problem.
Standard recommendations are to consider updating your motherboard's firmware or use an add-on PCIe SATA card if all else fails.
posted by Kadin2048 at 6:03 PM on February 11, 2013
Standard recommendations are to consider updating your motherboard's firmware or use an add-on PCIe SATA card if all else fails.
posted by Kadin2048 at 6:03 PM on February 11, 2013
Response by poster: Ok gang, thanks for the feedback!
The motherboard is an ASUS P5G41T-M LX, and the Seagate drive is model: ST3000DM001 SN:Z1F131SH PN: 1CH166-300 FW: CC43
Note - I recently flashed my bios to whatever the most recent iteration is, as I was experiencing dropouts from my audio interface, and that was a suggested possible fix...
posted by stenseng at 7:19 PM on February 11, 2013
The motherboard is an ASUS P5G41T-M LX, and the Seagate drive is model: ST3000DM001 SN:Z1F131SH PN: 1CH166-300 FW: CC43
Note - I recently flashed my bios to whatever the most recent iteration is, as I was experiencing dropouts from my audio interface, and that was a suggested possible fix...
posted by stenseng at 7:19 PM on February 11, 2013
Response by poster: So I just learned about fsutil, and was able to determine that the drive does in fact have the 4k clusters, so I may be sol with my motherboard, and need a controller, since I've tried the drive subsequent to the firmware update...
posted by stenseng at 7:31 PM on February 11, 2013
posted by stenseng at 7:31 PM on February 11, 2013
You might find that all you need to do is install a version of the Intel Rapid Storage Technology disk driver version 9.6 or newer.
posted by flabdablet at 4:44 AM on February 12, 2013
posted by flabdablet at 4:44 AM on February 12, 2013
Also, watch your terminology. Clusters and sectors are different things. NTFS usually uses 4K clusters even on drives with 512-byte (not 512k!) sectors.
In fact, not even sectors are sectors. There are logical sectors, traditionally 512 bytes each, which drive and controller use for addressing and whose size every data transfer between drive and controller need to be a multiple of. Then there are physical sectors, which are the fixed-size chunks that can actually written to the drive platters. Large drives with 4K physical sectors often use 512-byte logical sectors; the ST3000DM001 certainly does this. In fact there is enough legacy infrastructure code with the inbuilt assumption that disk sectors must and shall be 512 bytes long that drives with 4K logical sectoring are still considered rather specialist.
When queried via SATA, the ST3000DM001 reports itself as having 512-byte logical and 4096-byte physical sectors. Early versions of the Intel drivers for the ICH7 SATA controller on your motherboard are naive enough to pass this information on to Windows as-is, and Windows reacts to knowing the 4K physical sector size by screwing things up. Later driver versions just lie about it and tell Windows that the drive has purely traditional 512-byte sectoring, and everything then Just Works. Windows 8 and Windows Server 2012 don't screw things up any more, so drivers for those don't need to tell them white lies.
I expect the reason your drive works properly in a USB dock is that the standard Windows USB mass storage driver doesn't care about physical sector sizes and doesn't report them to Windows; it works purely in logical sectors. There are some special-purpose USB adaptors, notably those supplied with the large Western Digital USB drives, that do actually report a 4K logical sector size to work around partition table and 32-bit Windows limitations but I would be surprised to find that a slot-in SATA to USB docking station is doing this.
I've been surprised before, though, so you might care to plug your drive in via SATA and have a look at it with the Windows disk manager (right-click Computer and chose Manage>Disk Management) to check the partition sizes. If you see a single partition that's 1/8th the size of your drive, then your USB dock probably is doing the large logical sector trick and there is no way you will ever get compatible operation with that directly over SATA.
The traditional partitioning scheme for Windows is MBR, which is a complete dog's breakfast. Partitions are defined using both 32-bit logical block addressing and 24-bit Cylinder/Head/Sector addressing (the CHS addresses are ignored for LBAs they can't represent). The 32-bit variants of pre-Vista versions of Windows also use 32-bit LBA addressing internally. With 512-byte logical sectors, a 32-bit LBA translates to a 2TiB drive size limit; with 4K logical sectors, 32-bit LBAs can be used up to 16TiB.
There is nothing in the MBR partition table itself, though, that says what size sectors it was built for. So if you take a drive that was partitioned for 4K logical sectoring (which is the only way to define a 3TB partition using an MBR partition table) and hook it up in such a way as to make Windows treat it as having 512-byte logical sectoring, all the partitions will appear to be 1/8 their intended size. Not only that but they will start at the wrong disk offsets and Windows will not find the filesystems inside them.
All that aside, the reason that the drive manufacturers are moving to 4K physical sectoring actually has nothing to do with needing bigger sectors to address more drive space; you can address any amount of drive space you like with 512-byte sectors as long as you use big enough addresses. If sector numbers are constrained to fitting in 48 bits as the current SATA interface spec has them, you can address up to 248 × 512 bytes, roughly 144115 TB. The 64-bit variants of Windows pass LBAs around internally as 64-bit numbers. There's plenty of size there.
What they're actually doing is avoiding overhead. Right before each data sectors on a disk track is an address mark that the controller uses to tell which sector is which, and between address marks and data sectors is a gap to ensure that writing out a data sector doesn't screw up the address marks. Together, this stuff adds maybe 10% physical space overhead to each sector. Making the sectors 8 times bigger eliminates 7 out of 8 address marks, cutting the overhead substantially and increasing drive capacity. Using larger sectors also makes it practical to add more error correction bits; turns out that you can get quite substantial improvement in error correction by aggregating the correction bits for 8 times the data.
The downside to using smaller logical sectors in the drive-to-controller interface than physically exist on the disk surface is that if somebody asks the drive to write out 512 bytes to logical block address 1034657, this can no longer be done simply by seeking to the track containing that sector, waiting for its address mark to spin under the head and then writing 512 bytes. Instead, you have to seek to the track containing sector int(1034657 / 8) = 129332, wait for that one's address mark to spin under the head, read 4096 bytes into a buffer, replace a 512-byte range of those with the data to be written, wait for the address mark for sector 129332 to spin under the head again, and write the 4096 bytes back out. This read/modify/write sequence is much slower than a normal disk write.
Disk reads can still run at full speed, as can writes provided that (a) they're all done in multiples of 8 logical sectors, i.e. multiples of 4096 bytes and (b) they all begin at logical block addresses exactly divisible by 8. If either of those conditions isn't met, the drive must do at least one read/modify/write sequence for each write command issued by the controller.
In practice, both conditions are pretty easy to meet. Lots of popular disk filesystems are or can be built around 4K clusters and do all their reads and writes 4K at a time. Provided those filesystems are laid out on disk starting on 4K boundaries, everything works smoothly.
Windows versions from Vista onwards set up their disk partitions on 1MiB boundaries, which is fine. The installers for XP and older versions, though, start their disk partitions on track (32256-byte, 63 logical sectors) boundaries and end them on cylinder (8225280-byte, 255 × 63 logical sectors) boundaries. Apart from the assumed head and sector counts having no relationship at all to anything physical in the drive, neither of these numbers is a multiple of 4096 bytes, which means that naive installation of one of these Windows versions on a drive with 4K physical sectors will screw up every disk write the filesystem ever needs.
Seagate has allegedly done something clever inside their 4K formatted drives to work around this issue. I have never been able to find out exactly how that's supposed to work, so I'm not inclined to trust it; every time I partition a large drive, I put my partitions on 1MiB boundaries like Vista does.
Sorry this was a bit of a ramble, but I hope you've now got enough information to work through your confiusion.
posted by flabdablet at 7:12 AM on February 12, 2013 [2 favorites]
In fact, not even sectors are sectors. There are logical sectors, traditionally 512 bytes each, which drive and controller use for addressing and whose size every data transfer between drive and controller need to be a multiple of. Then there are physical sectors, which are the fixed-size chunks that can actually written to the drive platters. Large drives with 4K physical sectors often use 512-byte logical sectors; the ST3000DM001 certainly does this. In fact there is enough legacy infrastructure code with the inbuilt assumption that disk sectors must and shall be 512 bytes long that drives with 4K logical sectoring are still considered rather specialist.
When queried via SATA, the ST3000DM001 reports itself as having 512-byte logical and 4096-byte physical sectors. Early versions of the Intel drivers for the ICH7 SATA controller on your motherboard are naive enough to pass this information on to Windows as-is, and Windows reacts to knowing the 4K physical sector size by screwing things up. Later driver versions just lie about it and tell Windows that the drive has purely traditional 512-byte sectoring, and everything then Just Works. Windows 8 and Windows Server 2012 don't screw things up any more, so drivers for those don't need to tell them white lies.
I expect the reason your drive works properly in a USB dock is that the standard Windows USB mass storage driver doesn't care about physical sector sizes and doesn't report them to Windows; it works purely in logical sectors. There are some special-purpose USB adaptors, notably those supplied with the large Western Digital USB drives, that do actually report a 4K logical sector size to work around partition table and 32-bit Windows limitations but I would be surprised to find that a slot-in SATA to USB docking station is doing this.
I've been surprised before, though, so you might care to plug your drive in via SATA and have a look at it with the Windows disk manager (right-click Computer and chose Manage>Disk Management) to check the partition sizes. If you see a single partition that's 1/8th the size of your drive, then your USB dock probably is doing the large logical sector trick and there is no way you will ever get compatible operation with that directly over SATA.
The traditional partitioning scheme for Windows is MBR, which is a complete dog's breakfast. Partitions are defined using both 32-bit logical block addressing and 24-bit Cylinder/Head/Sector addressing (the CHS addresses are ignored for LBAs they can't represent). The 32-bit variants of pre-Vista versions of Windows also use 32-bit LBA addressing internally. With 512-byte logical sectors, a 32-bit LBA translates to a 2TiB drive size limit; with 4K logical sectors, 32-bit LBAs can be used up to 16TiB.
There is nothing in the MBR partition table itself, though, that says what size sectors it was built for. So if you take a drive that was partitioned for 4K logical sectoring (which is the only way to define a 3TB partition using an MBR partition table) and hook it up in such a way as to make Windows treat it as having 512-byte logical sectoring, all the partitions will appear to be 1/8 their intended size. Not only that but they will start at the wrong disk offsets and Windows will not find the filesystems inside them.
All that aside, the reason that the drive manufacturers are moving to 4K physical sectoring actually has nothing to do with needing bigger sectors to address more drive space; you can address any amount of drive space you like with 512-byte sectors as long as you use big enough addresses. If sector numbers are constrained to fitting in 48 bits as the current SATA interface spec has them, you can address up to 248 × 512 bytes, roughly 144115 TB. The 64-bit variants of Windows pass LBAs around internally as 64-bit numbers. There's plenty of size there.
What they're actually doing is avoiding overhead. Right before each data sectors on a disk track is an address mark that the controller uses to tell which sector is which, and between address marks and data sectors is a gap to ensure that writing out a data sector doesn't screw up the address marks. Together, this stuff adds maybe 10% physical space overhead to each sector. Making the sectors 8 times bigger eliminates 7 out of 8 address marks, cutting the overhead substantially and increasing drive capacity. Using larger sectors also makes it practical to add more error correction bits; turns out that you can get quite substantial improvement in error correction by aggregating the correction bits for 8 times the data.
The downside to using smaller logical sectors in the drive-to-controller interface than physically exist on the disk surface is that if somebody asks the drive to write out 512 bytes to logical block address 1034657, this can no longer be done simply by seeking to the track containing that sector, waiting for its address mark to spin under the head and then writing 512 bytes. Instead, you have to seek to the track containing sector int(1034657 / 8) = 129332, wait for that one's address mark to spin under the head, read 4096 bytes into a buffer, replace a 512-byte range of those with the data to be written, wait for the address mark for sector 129332 to spin under the head again, and write the 4096 bytes back out. This read/modify/write sequence is much slower than a normal disk write.
Disk reads can still run at full speed, as can writes provided that (a) they're all done in multiples of 8 logical sectors, i.e. multiples of 4096 bytes and (b) they all begin at logical block addresses exactly divisible by 8. If either of those conditions isn't met, the drive must do at least one read/modify/write sequence for each write command issued by the controller.
In practice, both conditions are pretty easy to meet. Lots of popular disk filesystems are or can be built around 4K clusters and do all their reads and writes 4K at a time. Provided those filesystems are laid out on disk starting on 4K boundaries, everything works smoothly.
Windows versions from Vista onwards set up their disk partitions on 1MiB boundaries, which is fine. The installers for XP and older versions, though, start their disk partitions on track (32256-byte, 63 logical sectors) boundaries and end them on cylinder (8225280-byte, 255 × 63 logical sectors) boundaries. Apart from the assumed head and sector counts having no relationship at all to anything physical in the drive, neither of these numbers is a multiple of 4096 bytes, which means that naive installation of one of these Windows versions on a drive with 4K physical sectors will screw up every disk write the filesystem ever needs.
Seagate has allegedly done something clever inside their 4K formatted drives to work around this issue. I have never been able to find out exactly how that's supposed to work, so I'm not inclined to trust it; every time I partition a large drive, I put my partitions on 1MiB boundaries like Vista does.
Sorry this was a bit of a ramble, but I hope you've now got enough information to work through your confiusion.
posted by flabdablet at 7:12 AM on February 12, 2013 [2 favorites]
Oh, sorry - flabdabet caught it, I said 512k sectors when I meant 512 byte sectors. Sorry about that, and I hope it doesn't add to the confusion. That said, flabdabet's explanation is fantastic, and this is almost certainly an issue that involves the format of your drive, the partitioning of your drive, Windows level of support for 4k drives, and your motherboard's support of that drive.
From a practical standpoint, I would recommend:
1) Making sure Windows 7 has been upgraded to Windows 7 SP1. I would also recommend installing this hotfix that increases Windows compatibility with drives like yours, even if you're running SP1.
2) Installing the latest Windows chipset driver for the Intel controller on your motherboard. You can get it from Asus' download page.
If that doesn't resolve the issue, I suspect getting things to work would be likely to require some sort of destructive action (ie, re-formatting the drive). If you don't have a backup of this data, you might want to consider buying another 3TB drive, upgrading Windows and your motherboard as above, installing the new drive in your PC case and partitioning/formatting the new drive there, transferring the data from the old drive to the new drive, then using the external drive as a backup.
posted by eschatfische at 8:55 AM on February 12, 2013
From a practical standpoint, I would recommend:
1) Making sure Windows 7 has been upgraded to Windows 7 SP1. I would also recommend installing this hotfix that increases Windows compatibility with drives like yours, even if you're running SP1.
2) Installing the latest Windows chipset driver for the Intel controller on your motherboard. You can get it from Asus' download page.
If that doesn't resolve the issue, I suspect getting things to work would be likely to require some sort of destructive action (ie, re-formatting the drive). If you don't have a backup of this data, you might want to consider buying another 3TB drive, upgrading Windows and your motherboard as above, installing the new drive in your PC case and partitioning/formatting the new drive there, transferring the data from the old drive to the new drive, then using the external drive as a backup.
posted by eschatfische at 8:55 AM on February 12, 2013
Response by poster: I'll give these assorted chipset patches / drivers / win7 patches a go and report back this evening. My hope is to be able to install internally with the hardware I've got. If not, I'll look into either a sata controller and second drive, or mobo compatible second drive...
posted by stenseng at 11:16 AM on February 12, 2013
posted by stenseng at 11:16 AM on February 12, 2013
Response by poster: PS - Thank you all for your excellent feedback!
posted by stenseng at 11:16 AM on February 12, 2013
posted by stenseng at 11:16 AM on February 12, 2013
Response by poster: So, after some testing and forum digging, it appears that my motherboard is incompatible with the Rapid Storage Technology disk driver, and the Win 7 hotfix doesn't solve my problems, so I'm thinking I'm out of luck for now. Samsung has a "drive unlocker" tool, but I don't know if it would do me any good, considering that my drive already has a file system and data on it, and I couldn't get it to work on my pc anyway. *sigh*
posted by stenseng at 1:18 PM on February 13, 2013
posted by stenseng at 1:18 PM on February 13, 2013
Might be worth your while trying Intel's driver update utility before giving up hope* completely.
I wouldn't bother with "disk unlockers" - as far as I know, all these do is install a shim that makes extra "phantom" disk drives available to Windows by adding an offset to the Windows LBAs before handing them off to the disk driver. So instead of your 3TB drive showing up in Windows as a single 3TB drive, it would show up as separate 2TB and 1TB drives. Effectively they just implement a non-standard kinda-sorta-partitioning scheme and, as you correctly suspect, are no use at all for getting access to an existing filesystem on a large drive.
*i.e. spending fifty bucks on an add-on PCIe SATA controller card.
posted by flabdablet at 4:06 PM on February 13, 2013
I wouldn't bother with "disk unlockers" - as far as I know, all these do is install a shim that makes extra "phantom" disk drives available to Windows by adding an offset to the Windows LBAs before handing them off to the disk driver. So instead of your 3TB drive showing up in Windows as a single 3TB drive, it would show up as separate 2TB and 1TB drives. Effectively they just implement a non-standard kinda-sorta-partitioning scheme and, as you correctly suspect, are no use at all for getting access to an existing filesystem on a large drive.
*i.e. spending fifty bucks on an add-on PCIe SATA controller card.
posted by flabdablet at 4:06 PM on February 13, 2013
Response by poster: Flabdablet - I'll try the driver updater tonight! Beyond that, yes - it'll be continuing to use the thermaltake dock until next payday, then to Newegg for a sata controller... =)
posted by stenseng at 4:36 PM on February 13, 2013
posted by stenseng at 4:36 PM on February 13, 2013
« Older I seem to ask a lot of questions about photos... | YANMD, but you *are* affordable. Is mild... Newer »
This thread is closed to new comments.
posted by scose at 4:22 PM on February 11, 2013