FTP moves in mysterious ways?
February 20, 2012 2:32 AM   Subscribe

Since about a week my work as a web developer has become incredibly frustrating due to intermittent but very frequent FTP problems. Making small changes in a remote stylesheet has become agony (7 out of 10 times the uploading fails, have to restart software and try again). I cannot find the root cause. It's not the software (tried several different applications), it's not the server (problems occur on all the websites I manage) and it happens on both uploading and downloading. I have turned off firewalls, antivirus software and things like DropBox but all to no avail. I don't have any other connection issues (I'm not on WiFi and web browsing and bittorrent work fine for example). I'm on a 64-bit Windows 7 Professional (Service Pack 1 with all the latest updates installed) laptop. That is the only variable I cannot change, I don't have another computer at hand. Does anyone have an idea what could be causing these problems? Is there something else I could try to fix it?
posted by dinkyday to Computers & Internet (15 answers total)
You say "it's not the server" because it happens for all the web-sites you manage but do you know for sure that those websites are in different data centres ? Just a thought. Using tracert on Windows would at least give you an indication of where you're going to.

If it were me one thing I'd try is : start a virtualized Amazon Web Service Windows instance; install some FTP software (Filezilla is free and good) and try doing some uploads from the virtualized instance to the web server - see how often that fails. I would expect you will have business as usual (ie no errors) but it's just possible you won't in which case it's likely it's the data centre the sites are hosted at.

Do you have any sort of shell/remote desktop access to the servers ? If you do you could upload the files to a virtual directory (utilising HTTP and not FTP) and then download the files onto the server using wget or (if you've got Remote Desktop) just a plain old browser. Sounds a bit wild west (and doesn't actually solve the problem) but if you're really stuck...
posted by southof40 at 2:52 AM on February 20, 2012

I should have said I would expect this is your ISP which in some way is stuffing up the transit of FTP traffic but the things I suggested would narrow it down (albeit at the expense of some time and work).
posted by southof40 at 2:53 AM on February 20, 2012

Do you have any sort of shell/remote desktop access to the servers ? If you do you could upload the files to a virtual directory (utilising HTTP and not FTP) and then download the files onto the server using wget or (if you've got Remote Desktop) just a plain old browser.

Just realised this isn't very clear. I mean if you can logon to shell or a desktop on the web-host you could then initiate a HTTP request to wherever you have mounted the files and so suck them onto the web-host.
posted by southof40 at 2:55 AM on February 20, 2012

Can you try using SFTP instead of FTP?
posted by reptile at 2:56 AM on February 20, 2012 [1 favorite]

reptile has my suggestion: Install Putty and see if scp works for you.

NB. Another possibility is that you're behind a NATing router that can't keep up with all your other connections. IIRC bittorrent can be especially hard on some consumer-grade firewalls (and older commercial ones). The same problem might be happening upstream if your ISP is putting your connection on a non-public IP address.
posted by pharm at 3:15 AM on February 20, 2012 [1 favorite]

To add to what has been said, FTP is a somewhat antiquated protocol. For the most part it works in the modern world of WiFi and NATs and so forth, but some features make it weird. It has no encryption and is insecure in other ways. It has an active mode which may not work well behind firewalls because it requires the server to set up a connection to your computer. It sometimes uses exotic port numbers. Many of its features make sense only in the world of 1980s unix.

People usually try to avoid FTP and use another protocol instead where possible. Essentially two have taken its role: HTTP upload and SCP/SFTP. In your case, SCP or SFTP is the best choice. It only uses a standard port number, all connections are established from your computer, everything is encrypted. Try WinSCP, which is a good client

That said, the fact that FTP connections don't work suggest a misconfiguration somewhere; it may be your FTP client, your firewall software, your computer OS, the router or the server. To rule out the first, you could try a Linux bootable stick (which is something useful to have at hand anyway). It's hard to tell without further information which it is. Could you try explicitly specifying Passive Mode? In fact I suspect that the server, or the servers, are at fault.
posted by faustdick at 3:34 AM on February 20, 2012 [1 favorite]

You could try a linux livecd to rule out problems on your machine -- if ftp fails using linux it must be the router or your provider.
posted by 3mendo at 3:51 AM on February 20, 2012

Sounds like your internet connection is dropping packets?
posted by gjc at 4:51 AM on February 20, 2012

If you use a router, try connecting directly to your Internet source (cable box for eg.).
posted by backwards guitar at 4:59 AM on February 20, 2012

Response by poster: Wow, thanks for all the suggestions! To answer a few of your questions: yes, I am sure that at least some of my websites are on different datacenters, none of them support SCP, SFTP or shell/remote desktop access, using a Mac OS X install on VirtualBox I have encountered the same problems and my laptop is connected directly to the cable modemrouter. Now I am going to try the Linux Live CD suggestion and report back after that...
posted by dinkyday at 5:15 AM on February 20, 2012

I have similar issues on a Comcast cable modem/router setup at my SO's house, and ZERO issues on my AT&T DSL at my house. I would try a different ISP connection (a Starbucks or something). A stylesheet upload shouldn't even be a blip on any decent free wifi connection.
posted by chrisfromthelc at 7:38 AM on February 20, 2012 [1 favorite]

It woud help a lot to know what FTP client you were using, what kind of FTP servers you're connecting to, and what kind of router you have. Are you using the FTP mode built into the Windows shell? Or some other client?

Have you tried the FileZilla FTP client? That's the most capable Windows FTP client I know. You may not like the UI, but you have a better chance making it reliable than most things.

As faustdick says, FTP is a weird protocol. In particular the default active mode causes problems with NAT based routers; by default the FTP server opens a socket back to the client to actually effect the file transfer, which is backwards. Passive mode is the solution to this weirdness, here's an explanation.

So try enabling passive mode. Here's docs on passive mode in FileZilla. Here's how in the Windows graphical shell / MSIE. If you're using a command line FTP client try typing "passive" at the start of the session and see if it fixes things. In Windows ftp.exe, it may be quot pasv.
posted by Nelson at 8:38 AM on February 20, 2012

Response by poster: Nelson, thanks for you reply. Normally I work in EditPad Pro, editing the CSS or PHP files directly on the server (it has a built in FTP client and the passive mode was already turned on). I have been doing this for the last two years without a problem until this week, even though nothing has changed (I was in the middle of a project). Often the connection already fails at trying to display the next directory I want to access (I'm clicking through the directories and suddenly it gets stuck and stays stuck). After that I have also tried regular FTP clients like FileZilla and CuteFTP, even though they don't really fit into my preferred workflow. FileZilla indeed seems to be the best of the bunch. Where the others just give up or hang FileZilla keeps on trying and usually succeeds (even though the waiting is a bit ridiculous for the tiny files it is trying to download). My modem is the Thomson TWG870 (cable modemrouter). I feel a bit dumb but I don't know what kind of FTP server I am connecting to, do you mean whether it runs on Windows or Unix (I'm pretty sure it runs on Unix). If my ISP is the problem how can I prove that to them? They will try and tell me that there's no problem so I need to come prepared...
posted by dinkyday at 8:57 AM on February 20, 2012

Can you take your laptop to a coffee shop or somewhere else with free wifi and try the uploads from there?

If they work, it indicates the problem is somewhere at your end (but not on your laptop) -- look to your cable router or ISP (my money would be on the router, those things don't last forever).

If they don't work, it indicates the problem is either within your laptop or at the remote end. Since you've tried multiple FTP clients, it's probably not that, so either your laptop hardware is having an issue or the problem exists at the remote (server) end. If you can get your laptop to FTP elsewhere (e.g. an amazon-hosted filezilla server as southof40 suggests) that will let you tell for sure whether the problem is your laptop or the remote end.
posted by axiom at 2:07 PM on February 20, 2012

Also, have you tried shutting down/restarting your modem/router?
posted by exphysicist345 at 7:47 PM on February 20, 2012

« Older a poorly timed rebellious phase   |   Embedded Video Hosting - What's a good option for... Newer »
This thread is closed to new comments.