FTP to FTP Directly?
March 6, 2012 9:12 PM Subscribe
I need to transfer 350GB (around 70GB x 5) from 5 servers to my server. Must I download all the information to my computer (not that bad time-wise) and re-upload to my server (very bad time-wise). Is there some way I can do an FTP to FTP transfer.
Hopefully free? I currently use Filezilla, but am not married to it.
Hopefully free? I currently use Filezilla, but am not married to it.
Can you access a shell on your server through SSH? If so, you can run command line FTP, or curl, or wget to retrieve the files from the other servers. If not, it may be possible to do so with a web interface shell or something, but I don't know what already exists in that area.
posted by WasabiFlux at 9:22 PM on March 6, 2012
posted by WasabiFlux at 9:22 PM on March 6, 2012
Basically you're going to hop on the server you own, or the five remotes and use ftp or scp to either push or pull the data. It's easier if the data is in a tar volume or similar so you're not messing about with dealing with a ton of individual files.
posted by iamabot at 9:24 PM on March 6, 2012
posted by iamabot at 9:24 PM on March 6, 2012
Can you describe your situation in a bit more detail? I don't understand why you can't FTP data directly from the clients (the five servers) to the server ("my server"). Run an FTP service (or preferably, SFTP or SSH for strong encryption) on "my server" and FTP/SFTP/SSH clients on on the five clients.
posted by Blazecock Pileon at 9:33 PM on March 6, 2012
posted by Blazecock Pileon at 9:33 PM on March 6, 2012
Or you could do this in the other direction: run FTP/SFTP/SSH services on the five servers, and a client on your server ("my server").
posted by Blazecock Pileon at 9:34 PM on March 6, 2012
posted by Blazecock Pileon at 9:34 PM on March 6, 2012
There's a variant of FTP designed for your very situation: FXP. However, I can't say I've tried it; I always do what everyone here is suggesting, and use ssh to connect to one of the remote machines, and then have them transfer between each other using FTP, HTTP, rsync, or netcat combined with tar/untar. If I had ssh access to both, I'd probably use rsync over ssh.
posted by Hither at 9:41 PM on March 6, 2012 [1 favorite]
posted by Hither at 9:41 PM on March 6, 2012 [1 favorite]
This should be trivially easy if you have SSH access to your servers. Just log onto each one at a time and use 'scp -r SOURCE your_username@remotehost.edu:/some/remote/directory/TARGET'
posted by KeSetAffinityThread at 9:44 PM on March 6, 2012 [1 favorite]
posted by KeSetAffinityThread at 9:44 PM on March 6, 2012 [1 favorite]
If you're looking for a windows app with a graphical interface that will do this, try FlashFXP. (Shareware)
Some servers can be configured to prevent this being enabled.
posted by Simon_ at 10:16 PM on March 6, 2012 [1 favorite]
Some servers can be configured to prevent this being enabled.
posted by Simon_ at 10:16 PM on March 6, 2012 [1 favorite]
There's a variant of FTP designed for your very situation:
Actually, original FTP allows for exactly that (one of the original motivations for PASV, before the curse of NAT, was so that you could open control connections to two servers and tell them to transfer files). I don't think it works any more in the modern Internet though.
posted by hattifattener at 10:21 PM on March 6, 2012
Actually, original FTP allows for exactly that (one of the original motivations for PASV, before the curse of NAT, was so that you could open control connections to two servers and tell them to transfer files). I don't think it works any more in the modern Internet though.
posted by hattifattener at 10:21 PM on March 6, 2012
If the data could change during the time when you are transmitting I would look into rsync. You can run it a second time to catch anything that was modified.
posted by dgran at 5:55 AM on March 7, 2012
posted by dgran at 5:55 AM on March 7, 2012
FXP for the win, both sides have to support it but it works great. You can basically orchestrate ftp transfer from host a to host b from host c, and the data goes directly between a and b. c is only involved in setting up the transfer between the two.
posted by some loser at 6:13 AM on March 7, 2012
posted by some loser at 6:13 AM on March 7, 2012
FTP is deprecated, consider SCP/SFTP by using rsync. Depending on what type of data you're transporting, it can even save time through the compression of the SSL.
posted by Threeway Handshake at 6:25 AM on March 7, 2012
posted by Threeway Handshake at 6:25 AM on March 7, 2012
Yeah; if FXP is out, consider using SCP. Despite the encryption overhead, it often tends to be faster.
posted by schmod at 6:52 AM on March 7, 2012
posted by schmod at 6:52 AM on March 7, 2012
Worse comes to worse, work out with the hosting providers to do a sneakernet arrangement, where you ship a drive, they load them or or attach them to the servers, then you ship it to the next place.
posted by iamabot at 8:37 AM on March 7, 2012
posted by iamabot at 8:37 AM on March 7, 2012
This thread is closed to new comments.
posted by iamabot at 9:22 PM on March 6, 2012