What's the easiest way to transfer a website (including hidden files) without shell access?

April 18, 2007 4:25 PM

I am trying to transfer a couple of websites from 1and1 beginner accounts to dreamhost. I usually do this by zipping or tarring the source site and then ftp. The rub is that the 1and beginner package does not allow shell access (although the target does). I'm looking at wget but I don't see a way to make it copy hidden files, which could bite me later if there are some config files buried in there somewhere. Probably missing something real obvious here. Any ideas?
posted by Manjusri to Computers & Internet (16 answers total)

I should have mentioned that, as well as copying hidden files and preserving permissions, it needs to recurse directories. Thanks!
posted by Manjusri at 4:26 PM on April 18, 2007

Ask 1and1 for a backup of your directory. You should be able to get that through the control panel, but if you need to, just email their tech support and ask for it. You don't have to explain why, just ask.
posted by cschneid at 4:36 PM on April 18, 2007

scp -r -p user@1and1:filez/ .

Unless that's not allowed either.
posted by sfenders at 5:00 PM on April 18, 2007

In which case one of those gui ftp programs with all the fancy options is your best bet.
posted by sfenders at 5:02 PM on April 18, 2007

You don't mention which methods *are* allowed.
posted by rhizome at 5:11 PM on April 18, 2007

In general, I'd probably work from the command line of the new Dreamhost account, and try a direct FTP (or other) connection from there. It should be faster than an FTP connection with your desktop machine to your old 1and1 account. It should also get all the files. As for preserving permissions, I'm not sure whether plain old FTP will do this (the scp command given by sfenders would be a better bet if it works), but you could always do 'man ftp' at the command line to learn more.

Dreamhost's tech support may be able to help you, too.
posted by amtho at 5:15 PM on April 18, 2007

I don't see any way to do this through the control panel, I just fired off a request to tech support, but I would like to know a way to accomplish this myself, esp. if they are uncooperative.

scp yields the following error message:
This account is restricted by rssh.
Allowed commands: sftp

sfenders: is there a free ftp gui that can handle the transfer between hosts. I'm a little leery about bringing it to a desktop both for the time and the potential of mucking something up in the transition.

rhizome: what should I be checking to see is allowed?
posted by Manjusri at 5:18 PM on April 18, 2007

amtho: If it is possible to make ftp or sftp recurse and get hidden files, I've not been able to glean it from my googling so far. :\
posted by Manjusri at 5:25 PM on April 18, 2007

FTP is fine, just connect and drag and drop. I'm pretty sure FTP sees dotfiles and other invisibles by default. If you're connecting via command line, "mget -R *" should get you everything.

If they actually do allow SFTP, you can connect with WinSCP for GUI transfers.
posted by rhizome at 5:44 PM on April 18, 2007

WinSCP will recurse, if you ask it to do so. I'm pretty sure that it will preserve permissions upon request, as well.
posted by wierdo at 6:31 PM on April 18, 2007

mget -R does get hidden files, and the command prompt shuts off the prompting for each file. However I still do not see how to recurse subdirectories with command line ftp, which is essential for this method to work. The 1and1 ftp server does not appear to support the command recursive mget.

At first glance it looks like winSCP is a client for downloading to a windows box. I'm trying to go directly between hosts, does WinSCP support this? The main reason I want to avoid downloading to my desktop is that something will get mucked up in translation. For instance I suspect that permissions would not be preserved during the process, and that the unix/windows file formats might get interchanged and not changed back.
posted by Manjusri at 6:42 PM on April 18, 2007

WinSCP is okay for a few files, but IIRC it's kind of lousy for mirroring. Also: It's hella slow.

Assuming you're allowed FTP access to the old site, you may be able to use FXP to transfer the files directly. Depends on whether both hosts allow FXP to work or not.
posted by neckro23 at 7:32 PM on April 18, 2007

I'm trying to go directly between hosts

Ah, I see. You might check to see if "lftp" is installed at the other host. It has a "mirror" command.

If not, and you really want to avoid copying it to your desktop for some reason, you could always make a CGI to run tar for you.
posted by sfenders at 8:44 PM on April 18, 2007

(ok, CGI maybe not, since if you don't have shell access it wouldn't make much sense to allow CGI scripts)
posted by sfenders at 8:48 PM on April 18, 2007

I'd look at PHP classes that allow you to create gzip files.
Here's a potential source
posted by seanyboy at 12:12 AM on April 19, 2007

Fxp and lftp were not allowed. Creating an archive with cgi and php are excellent ideas, but, thanks to the neckro23 mentioning the search term I should have been using, I found this Python script for mirroring sites via ftp and it seems to have done the trick. I did have to run it multiple times for each site as it timed out. Also, it did not transfer files in the root directory so a manual ftp with mget -R handled that.

Thanks for the help all.
posted by Manjusri at 12:12 PM on April 20, 2007