Best large File inter-operable file system?
December 22, 2010 10:10 AM Subscribe
What is the most inter-operable file system that can handled files larger then 4GB? (i.e. DVD iso files at 4.7GB or uncompressed AVI that are larger then 4GB.) Best would be something that Windows XP/Vista/7, Ubuntu/other linux flavors, and Mac could all read and write.
I store a variety of files and back up other peoples stuff often (as I am there defacto IT guy I hate it when people don't back up important stuff, so I do it for them). I think I know the answer to the question above, but I'm trying to find new solutions to an old problem. I rehash this question in my own head about every 6 months (about the time I buy a new drive).
Criteria:
*Able to hand files larger then 4GB
*Read & Write on the three OSes (natively would be best, but unlikely)
Useful:
*have a relatively fast write speed that doesn't require large processing power to translate.
I store a variety of files and back up other peoples stuff often (as I am there defacto IT guy I hate it when people don't back up important stuff, so I do it for them). I think I know the answer to the question above, but I'm trying to find new solutions to an old problem. I rehash this question in my own head about every 6 months (about the time I buy a new drive).
Criteria:
*Able to hand files larger then 4GB
*Read & Write on the three OSes (natively would be best, but unlikely)
Useful:
*have a relatively fast write speed that doesn't require large processing power to translate.
ntfs-3g is your best bet, as far as I know. EXT drivers exist for windows, but all have problems.
posted by a robot made out of meat at 10:20 AM on December 22, 2010
posted by a robot made out of meat at 10:20 AM on December 22, 2010
Yeah, writing NTFS is no problem for Linux or OS X with NTFS-3g, which has been stable since 2007.
posted by Rhomboid at 10:20 AM on December 22, 2010
posted by Rhomboid at 10:20 AM on December 22, 2010
Oh, and there is an HFS+ driver for windows, it's just commercial. If you're willing to pay, macdrive works. Access via linux is ok, but has to ignore the journaling.
posted by a robot made out of meat at 10:24 AM on December 22, 2010
posted by a robot made out of meat at 10:24 AM on December 22, 2010
If you need local access to a hard drive, NTFS (through ntfs-3g on Ubuntu, other Linux distros, or Mac OS, or perhaps MacFUSE on Mac OS) is indeed likely to be the right answer.
However, Windows, Ubuntu and Mac OS all provide built-in read/write support for SMB-based network file systems -- natively in Windows and MacOS, and through Samba in Ubuntu. If you put the files on a NAS box of some sort -- a Windows, Linux or Mac box with shared folders, or a physical NAS connected to your network -- any of the machines on that network would be able to read and write to it.
So, if you were thinking of an external hard drive that would work with all three natively, you might want to think about a network hard drive instead of a USB, Firewire or SATA-2 based drive.
posted by eschatfische at 10:25 AM on December 22, 2010
However, Windows, Ubuntu and Mac OS all provide built-in read/write support for SMB-based network file systems -- natively in Windows and MacOS, and through Samba in Ubuntu. If you put the files on a NAS box of some sort -- a Windows, Linux or Mac box with shared folders, or a physical NAS connected to your network -- any of the machines on that network would be able to read and write to it.
So, if you were thinking of an external hard drive that would work with all three natively, you might want to think about a network hard drive instead of a USB, Firewire or SATA-2 based drive.
posted by eschatfische at 10:25 AM on December 22, 2010
exFAT is available for free from Microsoft. It's supported in OS X 10.6.5, the latest version. And, uh, Wikipedia says it's supported on Linux?
I've been using Paragon's NTFS solution on OS X Leopard—only $40—and every now and then I get some bugs when I delete something from NTFS partition on my Mac. And it's nice and fast on an old dual G5 with Leopard and on my Santa Rosa MacBook Pro.
posted by infinitewindow at 10:37 AM on December 22, 2010
I've been using Paragon's NTFS solution on OS X Leopard—only $40—and every now and then I get some bugs when I delete something from NTFS partition on my Mac. And it's nice and fast on an old dual G5 with Leopard and on my Santa Rosa MacBook Pro.
posted by infinitewindow at 10:37 AM on December 22, 2010
Yeah, native exFAT is available on Windows and Macs now and supports very large files & filesystems. I think there's a read-only version for linux but that will probably change.
posted by chairface at 12:02 PM on December 22, 2010
posted by chairface at 12:02 PM on December 22, 2010
I've had excellent results reading/writing to NTFS with ntfs-3g in Linux.
I've also had excellent results reading/writing to ext3 with ext2 IFS in windows.
I'd go with the former, all else being equal.
posted by astrochimp at 1:25 PM on December 22, 2010
I've also had excellent results reading/writing to ext3 with ext2 IFS in windows.
I'd go with the former, all else being equal.
posted by astrochimp at 1:25 PM on December 22, 2010
What about something like this? it removes any question of drivers or software support etc. The current editions go up to 2TB and you can daisy chain extra storage on if you need it. Also, if you wish to set it up through the control panel, you can also add things like private storage per user etc.
I don't work for WD, I just have one of those drives and it's for the win.
posted by dougrayrankin at 4:25 PM on December 22, 2010
I don't work for WD, I just have one of those drives and it's for the win.
posted by dougrayrankin at 4:25 PM on December 22, 2010
I would use FAT or exFat. NTFS 3g had really slow writes for me on Linux.
posted by gjc at 4:57 PM on December 22, 2010 [1 favorite]
posted by gjc at 4:57 PM on December 22, 2010 [1 favorite]
Response by poster: Thanks everyone. You all more or less mirrored my thought process, but I thank you all as I learned a few new bits.
*exFAT sound like a good option, and I now see a FUSE development of it for ubuntu currently in beta (read and write). I'm a bit leary as I'm concerned that the FUSE part of this could end up causing similar issues to the ntfs-3g drivers mentioned below. Ubuntu is my primary OS so it would be annoying if it ate my processing up every time I did a large transfer. Worth a shot though.
*NTFS is a good idea, but I have the same problem gjc does with really slow writes (doing more then a 1tb of non-delta transfer using rsync takes nearly 24 hours which is a pain in the ass). I've also had some serious errors using ntfs-3g which caused loss of data. I don't really trust it for non-windows machines.
*HFS+ is an interesting idea, but I've had my share of problems working with MacDrive. It never seems to be quite as easy as one hopes. Plus the processing overhead is a bit high for my tastes.
*ext2/ext3 might be the answer, though I'm not entirely sure on the Windows end. ext2 IFS for writes seems like it'll work (and certainly Ext2Read, Explore2fs, DiskInternals Linux Reader, or Ext2fsd should do reads) I'll have to test the processing overhead on a windows machine and see if it's reliable enough. This misses the native criterian, but it might work.
*dougrayrankin idea about the WD My Books is interesting, and it's a revelation that they use ext3 (damned cool really), but not quite what I was looking for. A NAS or daisy chained RAID using My Books is impractical for regular (safe) transport.
posted by wonderfullyrich at 2:30 AM on December 23, 2010
*exFAT sound like a good option, and I now see a FUSE development of it for ubuntu currently in beta (read and write). I'm a bit leary as I'm concerned that the FUSE part of this could end up causing similar issues to the ntfs-3g drivers mentioned below. Ubuntu is my primary OS so it would be annoying if it ate my processing up every time I did a large transfer. Worth a shot though.
*NTFS is a good idea, but I have the same problem gjc does with really slow writes (doing more then a 1tb of non-delta transfer using rsync takes nearly 24 hours which is a pain in the ass). I've also had some serious errors using ntfs-3g which caused loss of data. I don't really trust it for non-windows machines.
*HFS+ is an interesting idea, but I've had my share of problems working with MacDrive. It never seems to be quite as easy as one hopes. Plus the processing overhead is a bit high for my tastes.
*ext2/ext3 might be the answer, though I'm not entirely sure on the Windows end. ext2 IFS for writes seems like it'll work (and certainly Ext2Read, Explore2fs, DiskInternals Linux Reader, or Ext2fsd should do reads) I'll have to test the processing overhead on a windows machine and see if it's reliable enough. This misses the native criterian, but it might work.
*dougrayrankin idea about the WD My Books is interesting, and it's a revelation that they use ext3 (damned cool really), but not quite what I was looking for. A NAS or daisy chained RAID using My Books is impractical for regular (safe) transport.
posted by wonderfullyrich at 2:30 AM on December 23, 2010
This thread is closed to new comments.
posted by Chocolate Pickle at 10:13 AM on December 22, 2010