Enormous Download from Dropbox.com

January 18, 2018 11:40 AM

I have a seldom-used 500g dmg file in DropBox that's not mirrored on my drive. I need to grab it for a sec, but don't have room on my hard drive, so I'm downloading to an external drive. But I haven't been able to complete the d/l in Safari or Chrome (for reasons explained below).

I'm afflicted with the common and longstanding Mac issue where external drives frequently spontaneously eject (you get lots of “Disk Not Ejected Properly” notifications). That's likely what screws up the downloads, since I'm downloading to an external drive.

The workaround for the problem is an app called Mountain which automatically re-mounts ejected drives. But it's not quick enough to recover the download, apparently.

I don't expect to solve the random disk ejection issue here, but I do need to get the damned file downloaded. Any suggestions? Maybe a download manager app capable of auto-restarting stalled downloads? Or does DropBox block that sort of thing?

posted by Quisp Lover to Computers & Internet (21 answers total)

Best answer: Try mounting the Dropbox account with Cyberduck, and get it that way?

(I have only used this product on Windows and with Google Drive, but I have used this product to do something similar)
posted by deezil at 11:50 AM on January 18, 2018

Could you move a less important file from your OS drive to the external and download the Dropbox file to the OS drive that's not going to spontaneously disconnect?

For example, monolingual is an app which helps you remove unused language files. Or clean out your downloads folder, etc.
posted by bluecore at 11:57 AM on January 18, 2018

I'm gonna guess that it's using your full drive for the temp space it downloads the file to before renaming it and saving it to your external drive. Perhaps setting TMP/TEMP to /external_drive would help.
posted by rhizome at 12:00 PM on January 18, 2018

Response by poster: deezil, interesting app. Looks like an old-time FTP app (I already use Fetch), but it somehow cleverly patches in DropBox, etc. I'm trying the download, but I don't have high hopes it'll be any gnarlier. Perhaps it will allow restarted downloads once it stalls, however.

bluecore, the transfer process as I back up to the external will be subject to the same dismounting issue.

rhizome, that's certainly possible (and I'll try it if everything else rules out), but I suspect Occam probably prefers the drive ejection bug as a culprit.
posted by Quisp Lover at 12:02 PM on January 18, 2018

Response by poster: CyberDuck just reported "connection failed" after downloading 2.7gb, but at least did so with some info (as opposed to my browser download attempts, which silently peter out):

Premature end of Content-Length delimited message body (expected: 512000155136; received: 2123676466.

Can anyone interpret? is this the issue rhizome warned me of?

Good news however....Cyberduck did let me continue, rather than start from scratch. So this may work, as long as I keep checking in (awfully bumpy, though).
posted by Quisp Lover at 12:04 PM on January 18, 2018

I imagine you've been down this path, but just for completeness' sake could it be sleep related?
posted by rhizome at 12:08 PM on January 18, 2018

Response by poster: This citi never sleeps.
posted by Quisp Lover at 12:09 PM on January 18, 2018 [1 favorite]

Premature end of Content-Length delimited message body (expected: 512000155136; received: 2123676466.

That sounds like a server-side error to me rather than a storage connection issue on your end. Is it possible there was some sort of time out on Dropbox's side?
posted by zarq at 12:26 PM on January 18, 2018

If CyberDuck doesn't work, can you borrow a different computer and hook the portable drive up to that? Even a Windows machine might work because there are drivers for them to access drives formatted for a Mac.
posted by exogenous at 12:26 PM on January 18, 2018

Response by poster: Damn. CyberDuck has locked up after reporting 20gb downloaded (though the file on my drive is still 2gb). I forced quit, and it will only restart download from scratch.

I have no way of determining problems on DropBox' end.

I paid $$$$ for this MacBook Pro, and I'd really prefer not to have to borrow a computer to do computer stuff. But it may come to that....
posted by Quisp Lover at 12:47 PM on January 18, 2018

Response by poster: DropBox rep says only way to do this is to change my DB folder location to the external drive, config it so the huge file mirrors locally, and wait for all 2TB to transfer. Then I can re-re-locate DB folder back to my internal drive, config the huge file not to mirror, and wait for all 1.5TB to transfer.

There is a downside to user-friendly software.

OTOH, CyberDuck's third attempt is now up to 30gb and running strong.....
posted by Quisp Lover at 1:41 PM on January 18, 2018

Ah, I don't use Dropbox so I didn't know about their local db repository, which is likely quite close to my tempdrive idea: If the drive the db directory is on is out of space, it can't be updated and reasonably would crash out.
posted by rhizome at 2:05 PM on January 18, 2018

Instead of changing the local Dropbox folder location, I wonder if it would work to disable Dropbox sync on the laptop while you try and download.
posted by exogenous at 2:09 PM on January 18, 2018

Best answer: You could try mounting your Dropbox as a remote file system. From the point of view of macOS, your DMG will then appear to be like any other file, but one on a (very slow) external disk. This might allow you to open it and get whatever file you need out. I've never used this tool, but it claims to do it.
posted by caek at 2:17 PM on January 18, 2018

Best answer: For future reference, if you're storing very large disk images somewhere remote, it may make sense to make them sparseimages rather than monolithic regular DMG files. A sparseimage appears to be a single file in the GUI, but it's actually a directory containing lots of files. This makes it easier to sync over flaky connections because you can use command line tools like rsync, which can recover downloads of directories from where they left off.
posted by caek at 2:21 PM on January 18, 2018

Response by poster: caek, cool, did not know you could do this! FWIW, CloudMounter is similar, it seems.

It's a password-encoded DMG, which may/may not work, and both these apps are >$40, and CyberDuck is up to 60gb with only one restart (not sure why I'm only getting 12 mb/sec on my premium FIOS service, normally >100mb/sec), but this will definitely be my fallback!
posted by Quisp Lover at 2:23 PM on January 18, 2018 [1 favorite]

Could you move a less important file from your OS drive to the external and download the Dropbox file to the OS drive that's not going to spontaneously disconnect?

bluecore, the transfer process as I back up to the external will be subject to the same dismounting issue.

One option might be you could choose a bunch of smaller files and get through that backup process incrementally?
posted by juv3nal at 4:41 PM on January 18, 2018

Response by poster: Not a bad idea, but I think I'd rather exfoliate with barbed wire than clear out space by moving 500GB worth of small files!

OTOH: CyberDuck is at 160GB, and has only shut down eight times - and, thank god, resumed each time without a hitch. I realize multitudes eagerly await these updates. In other news, my humidifier just ran out of water.
posted by Quisp Lover at 4:59 PM on January 18, 2018

Best answer: I hope you got your file. FWIW, I sometimes do the following :

- Create a remote machine (on AWS, or MacinCloud)
- Install dropbox and let it sync on the cloud machine
- Login through remote desktop, and do whatever I need to do remotely (or put the file on an ftp server)
- Use an ftp client for download
- Make sure I uninstall dropbox, clean the machine of credentials etc before I trash it.
posted by motdiem2 at 12:23 AM on January 19, 2018 [1 favorite]

Response by poster: As an eager Internet user from well before the WWW, this suggestion holds gleeful appeal. This is how we used to do ALL this stuff, from the good ol' command line....
posted by Quisp Lover at 2:26 PM on January 19, 2018 [1 favorite]

Response by poster: Update: 346GB complete. I'm living the dream.
posted by Quisp Lover at 2:32 PM on January 19, 2018 [2 favorites]