Choosing a filesystem for an external hdd
May 2, 2009 3:10 PM Subscribe
What filesystem should I use on my new USB 2.0 external hard drive? It'll mostly stay attached to an Ubuntu box and be used to store large video files for streaming to playback devices on my network, but I'd also like to be able to take it to a friend's house and plug it into whatever computer they happen to have.
The files will be large and replaceable, so I'm not even going to attempt to keep backups. I'm guessing it should be a Microsoft filesystem, since afaik Windows isn't happy with anything else and every other major OS has some way of dealing with them. Should it be NTFS? Some version of FAT? What tool should I use to do the formatting? I also have Windows and OS X machines available for setup.
The files will be large and replaceable, so I'm not even going to attempt to keep backups. I'm guessing it should be a Microsoft filesystem, since afaik Windows isn't happy with anything else and every other major OS has some way of dealing with them. Should it be NTFS? Some version of FAT? What tool should I use to do the formatting? I also have Windows and OS X machines available for setup.
FWIW, FAT32 support may leave linux at some point in the future- this was the primary thing that Microsoft just sued TomTom for. It's fine for your application, but I wouldn't rely on FAT32 support being available in Linux (and the more anti-patent distributions such as Debian) for the long term.
posted by jenkinsEar at 3:28 PM on May 2, 2009
posted by jenkinsEar at 3:28 PM on May 2, 2009
Best answer: I was debating this myself just the other day.
Ext3, Reiser etc: Supported on Windows only with the installation of additional software.
FAT32: Popular for portable hard drives etc; supported by Windows, Linux, and OS X. Maximum file size of 4GB, (i.e. unsuitable for a some video work).
NTFS: Supported on Windows. Supported on Linux, but with lots of warning labels about advanced features and data loss. Read-only on OS X except with extra software.
In summary: I decided to try NTFS (formatted using a windows computer as it's a windows filesystem) because support for files above 4GB was a requirement for me. Then I discovered my new portable hard disk was dead on arrival, so I'm afraid I haven't yet tried an NTFS portable hard disk in practice.
posted by Mike1024 at 3:30 PM on May 2, 2009
Ext3, Reiser etc: Supported on Windows only with the installation of additional software.
FAT32: Popular for portable hard drives etc; supported by Windows, Linux, and OS X. Maximum file size of 4GB, (i.e. unsuitable for a some video work).
NTFS: Supported on Windows. Supported on Linux, but with lots of warning labels about advanced features and data loss. Read-only on OS X except with extra software.
In summary: I decided to try NTFS (formatted using a windows computer as it's a windows filesystem) because support for files above 4GB was a requirement for me. Then I discovered my new portable hard disk was dead on arrival, so I'm afraid I haven't yet tried an NTFS portable hard disk in practice.
posted by Mike1024 at 3:30 PM on May 2, 2009
"if you're going for large static files NTFS features won't add much"
Unless some of those files are >=4G in size.
I've read and written NTFS from both Linux and MacOS. It hasn't been particularly demanding use, but I haven't had a problem.
Also, I just really don't like FAT32
posted by Good Brain at 3:40 PM on May 2, 2009
Unless some of those files are >=4G in size.
I've read and written NTFS from both Linux and MacOS. It hasn't been particularly demanding use, but I haven't had a problem.
Also, I just really don't like FAT32
posted by Good Brain at 3:40 PM on May 2, 2009
Nthing FAT32. It sucks, yeah, but it will work best for you given the circumstances.
posted by aheckler at 5:28 PM on May 2, 2009
posted by aheckler at 5:28 PM on May 2, 2009
Best answer: Alternative: Make a small FAT32 partition with which you can install the tools necessary to
I generally prefer XFS for large volumes these days; anyone who's sat through a mandatory "it's been X days since fsck was run" on a 2TB ext3 volume will probably agree with me.
posted by SpecialK at 5:36 PM on May 2, 2009
I generally prefer XFS for large volumes these days; anyone who's sat through a mandatory "it's been X days since fsck was run" on a 2TB ext3 volume will probably agree with me.
posted by SpecialK at 5:36 PM on May 2, 2009
Ack. That first line was supposed to be "make a small fat32 partition with which you can install the tools necessary to read the other more modern filesystem you choose."
posted by SpecialK at 5:37 PM on May 2, 2009
posted by SpecialK at 5:37 PM on May 2, 2009
Yeah, SpecialK's got it if you have > 4G files (if you don't, just use FAT32). Make a small FAT16 or FAT32 partition to hold e.g. Linux & OS X versions of FUSE, then make the rest an NTFS partition. Accessing that through FUSE works OK on OS X, and I'm assuming it's the same on Linux (that may be a mistake ;-).
As you say, the optimum way would be to have it formatted to suit your system (ext3?), but I don't think there's been much work on porting FUSE to Windows so you're kinda stuck with NTFS if you want to carry files > 4G. I seem to remember a ext2 plugin for WinNT/Win2k/XP that would read (and write?) ext3 volumes, but I don't think it was free, and I don't know if it was updated for Vista.
posted by Pinback at 6:21 PM on May 2, 2009
As you say, the optimum way would be to have it formatted to suit your system (ext3?), but I don't think there's been much work on porting FUSE to Windows so you're kinda stuck with NTFS if you want to carry files > 4G. I seem to remember a ext2 plugin for WinNT/Win2k/XP that would read (and write?) ext3 volumes, but I don't think it was free, and I don't know if it was updated for Vista.
posted by Pinback at 6:21 PM on May 2, 2009
Response by poster: Thanks for the help, everyone. I'm going to take my chances with NTFS, since 4G+ files seem like they'll be getting more common as HD rips go mainstream. While SpecialK's solution is appealoing and would seem to give the best of both worlds, I don't want to have to tell a group of my buddies "Hey gang, let's watch this great video! It'll just be a few minutes while I get on your computer and install some weird software you're never heard of first. Oh shit, you're running Windows 7 SP13? Hang on, lemme see if there's an update..."
posted by contraption at 9:54 PM on May 2, 2009
posted by contraption at 9:54 PM on May 2, 2009
NTFS-3g has gone a looong way towards making read/write access of NTFS filesystems on linux work well. I regularly transfer large media files/isos etc via my 500GB external storage drive between windows machines and my home linux file server, and FAT32 was a no-go, so I use NTFS. I just plug it in via firewire or usb, automount, and off I go.
Ironically enough, I've had windows bork the drive a couple of times when it was unplugged improperly and not be able to read it any more, yet linux was able to mount it every time (with the --force option due it being an unclean unmount on windows), access the files, and clean it up when unmounted so windows could read it again.
NTFS should work just fine, with ntfs-3g on the linux client.
posted by ArkhanJG at 1:54 AM on May 3, 2009
Ironically enough, I've had windows bork the drive a couple of times when it was unplugged improperly and not be able to read it any more, yet linux was able to mount it every time (with the --force option due it being an unclean unmount on windows), access the files, and clean it up when unmounted so windows could read it again.
NTFS should work just fine, with ntfs-3g on the linux client.
posted by ArkhanJG at 1:54 AM on May 3, 2009
ntfs-3g, which comes standard with Ubuntu and various other popular distros, has never caused me any file system integrity problems. However, it doesn't support any form of NTFS access control: all existing NTFS files are readable and writable regardless of their Windows-set NTFS permissions, and files created by ntfs-3g get Full Control for Everyone permissions.
In fact, for your intended application, this lack of NTFS access control support is more of a feature than a bug, and you'd be best advised to make Windows emulate it by setting the permissions on the drive's root folder and all subfolders and files to Full Control for Everyone; otherwise you'll strike Access Denied problems when you use the drive to transport files from one Windows box to another.
As this blog entry notes, you might also want to put a FAT32 partition on there if you want to make some of your collection accessible to e.g. DVD/USB players or in-car MP3 players.
posted by flabdablet at 7:46 AM on May 3, 2009
In fact, for your intended application, this lack of NTFS access control support is more of a feature than a bug, and you'd be best advised to make Windows emulate it by setting the permissions on the drive's root folder and all subfolders and files to Full Control for Everyone; otherwise you'll strike Access Denied problems when you use the drive to transport files from one Windows box to another.
As this blog entry notes, you might also want to put a FAT32 partition on there if you want to make some of your collection accessible to e.g. DVD/USB players or in-car MP3 players.
posted by flabdablet at 7:46 AM on May 3, 2009
Just FYI, out-of-the-box Macs will read NTFS fine - it's only if you want to write to them that you need MacFUSE + NTFS-3G.
posted by agentmunroe at 7:47 AM on May 3, 2009
posted by agentmunroe at 7:47 AM on May 3, 2009
« Older Why do golfers sometimes place a token or coin on... | Can a car be towed under abandoned car law if it's... Newer »
This thread is closed to new comments.
posted by orthogonality at 3:13 PM on May 2, 2009