How to copy one large file from Linux to Mac OS X
April 4, 2008 2:52 PM Subscribe
I want to copy a 6.5Gb file from my Linux machine to my Macbook running Tiger as quickly as possible. I have a USB drive. What is the best way to do this?
I have tried: Copying over ethernet (very slow), creating an hfsplus partition (couldn't get it working in Ubuntu), creating a FAT32 partition (won't write files > 4Gb), creating an ext3 partition (can't read it as extfsx never got updated for Tiger). I don't need a shareable filesystem in general, I just want to move this one file. I can't help thinking there is some brilliant method involving dd, but I'm not sure exactly what.
I have tried: Copying over ethernet (very slow), creating an hfsplus partition (couldn't get it working in Ubuntu), creating a FAT32 partition (won't write files > 4Gb), creating an ext3 partition (can't read it as extfsx never got updated for Tiger). I don't need a shareable filesystem in general, I just want to move this one file. I can't help thinking there is some brilliant method involving dd, but I'm not sure exactly what.
Use rar tools to split it in to a multi file rar archive and then re-assemble ?
posted by iamabot at 2:58 PM on April 4, 2008
posted by iamabot at 2:58 PM on April 4, 2008
Huh. How slow was the ethernet copy? At a megabyte/sec, it could take a couple of hours, but that's not bad, & faster than looking around for very long.
What comes to mind: run split on the files to split them up into, say, CD-sized pieces, put them on the FAT32 partition, & join them on the Mac side. You could just "cat split* > join.img" there.
posted by Pronoiac at 3:02 PM on April 4, 2008
What comes to mind: run split on the files to split them up into, say, CD-sized pieces, put them on the FAT32 partition, & join them on the Mac side. You could just "cat split* > join.img" there.
posted by Pronoiac at 3:02 PM on April 4, 2008
If you connect the two with a Firewire cable, and boot the mac with 'T' held down, you miiight be able to mount its disk on the Ubuntu computer.
This was written for live CDs, but it might help with the HFS+ business:
http://jclark.org/weblog/2005/05/24/ubuntumount/
posted by Plug Dub In at 3:04 PM on April 4, 2008
This was written for live CDs, but it might help with the HFS+ business:
http://jclark.org/weblog/2005/05/24/ubuntumount/
posted by Plug Dub In at 3:04 PM on April 4, 2008
scp will have some overhead (since they removed the null cipher in the stock client a long time ago) but it should get you 2-10MB/s+. I can get 4MB/s without strain on an 802.11n wireless network. Your drive, even an old one, will push 20-50MB/s+ so it's not a bottleneck.
How slow was the network copy? It will by far be the fastest.. even with a 10/100 switch slowing things down I'd estimate two minutes at most (6500 Mb @ 10MB/s).
posted by kcm at 3:07 PM on April 4, 2008
How slow was the network copy? It will by far be the fastest.. even with a 10/100 switch slowing things down I'd estimate two minutes at most (6500 Mb @ 10MB/s).
posted by kcm at 3:07 PM on April 4, 2008
Yes, ethernet will be a little slow, but will likely be faster than the time all the time you're spending trying to figure out some other solution. Just start the copy before bed, and let it run overnight.
posted by chrisamiller at 3:09 PM on April 4, 2008
posted by chrisamiller at 3:09 PM on April 4, 2008
Oh. You meant 6.5GB (you say FAT32 won't write >4.5Gb, but the limit is actually 4GB (technically 4GB-1B). Multiply my estimate by 8, maybe 10 minutes total.
posted by kcm at 3:10 PM on April 4, 2008
posted by kcm at 3:10 PM on April 4, 2008
use the split command to split the file up, then tar each one, stick as many on a stick and then untar and recombine on the other end. Or any permutation of this.
posted by furtive at 3:24 PM on April 4, 2008
posted by furtive at 3:24 PM on April 4, 2008
Best answer: How were you copying it over the network?
one$ ssh two.local "netcat -l -p 5000 |zcat > newfile" #listen on the far end and write to a file
one$ gzip -c oldfile |netcat two.local 5000 #send this (compressed) file over the network
That should avoid the encryption overhead.
Or, just "scp" it, and it would take... ~80Mb/s ... 10 MB/s ... 0.6GB/min ... about 10 minutes at best, but maybe as much as 100 if it's only 10Mb/s Ethernet.
You know that macs automatically cross-over enet cables, yes? Plug them right together.
posted by cmiller at 3:25 PM on April 4, 2008 [1 favorite]
one$ ssh two.local "netcat -l -p 5000 |zcat > newfile" #listen on the far end and write to a file
one$ gzip -c oldfile |netcat two.local 5000 #send this (compressed) file over the network
That should avoid the encryption overhead.
Or, just "scp" it, and it would take... ~80Mb/s ... 10 MB/s ... 0.6GB/min ... about 10 minutes at best, but maybe as much as 100 if it's only 10Mb/s Ethernet.
You know that macs automatically cross-over enet cables, yes? Plug them right together.
posted by cmiller at 3:25 PM on April 4, 2008 [1 favorite]
1. If your linux machine has firewire, put your Mac in Target Disk Mode and copy.
2. Network transfer should be the 2nd fastest. Enable FTP sharing on your mac and upload away. I've gotten upwards of 10MB/s over 10/100 ethernet. I've found that FTP has much less overhead than SMB and AFS.
posted by mphuie at 3:34 PM on April 4, 2008
2. Network transfer should be the 2nd fastest. Enable FTP sharing on your mac and upload away. I've gotten upwards of 10MB/s over 10/100 ethernet. I've found that FTP has much less overhead than SMB and AFS.
posted by mphuie at 3:34 PM on April 4, 2008
Response by poster: Network transfer people -- you're right, it's not that slow (15 mins). I just did my sums wrong... an order of magnitude wrong.
posted by beniamino at 3:39 PM on April 4, 2008
posted by beniamino at 3:39 PM on April 4, 2008
Further to cMiller's point, if you have a Cat5e fully wired cable then they should go up to gigabit speeds, provided the linux box supports it.
posted by gaby at 8:16 PM on April 4, 2008
posted by gaby at 8:16 PM on April 4, 2008
« Older A piano, a viola and a jungle book. Music... | First Shot at Full-Time Librarianship Newer »
This thread is closed to new comments.
posted by iamabot at 2:57 PM on April 4, 2008