How can I FTP only the recently modified files from within a deep file structure?
September 15, 2009 8:54 AM Subscribe
How can I FTP only the recently modified files from within a deep file structure?
I work for an organization with a fairly large website (~20,000 discrete items - .asp, .txt, and .pdf). Recently it was decreed that we would change the way some of our links work (inserting intermediary warning page in new targeted window). I was able to write a find+replace script to switch out the old link for the new one, without messing up anything else. It works a treat, but after running it on my local development version of the site I don't particularly want to FTP the whole thing over to the live site, all 20,000 files, when only maybe 3,000 have been changed. Particularly when some of the .pdf's (none of which are affected by this) are giant - 1,000's of MB. Doing it manually would be nuts as these files are sprinkled throughout a fairly complex and deep file structure.
Is there a way to FTP only the recently changed files, while maintaining the file structure? I normally use Flash FXP, but would be fine with some fancy command line FTP expressions, or any other cheap/free solution.
I work for an organization with a fairly large website (~20,000 discrete items - .asp, .txt, and .pdf). Recently it was decreed that we would change the way some of our links work (inserting intermediary warning page in new targeted window). I was able to write a find+replace script to switch out the old link for the new one, without messing up anything else. It works a treat, but after running it on my local development version of the site I don't particularly want to FTP the whole thing over to the live site, all 20,000 files, when only maybe 3,000 have been changed. Particularly when some of the .pdf's (none of which are affected by this) are giant - 1,000's of MB. Doing it manually would be nuts as these files are sprinkled throughout a fairly complex and deep file structure.
Is there a way to FTP only the recently changed files, while maintaining the file structure? I normally use Flash FXP, but would be fine with some fancy command line FTP expressions, or any other cheap/free solution.
Seconding rsync, but if that's not an option, FileZilla and many/most other FTP clients will allow you to only overwrite files whose modified date varies, or whose size varies...
You'll still have to let the FTP client scan through every directory and queue up the files, only to decide not to upload them, but it'll get the job done.
posted by twiggy at 8:57 AM on September 15, 2009
You'll still have to let the FTP client scan through every directory and queue up the files, only to decide not to upload them, but it'll get the job done.
posted by twiggy at 8:57 AM on September 15, 2009
You could use an
posted by adipocere at 9:21 AM on September 15, 2009
xcopy with the /d flag to copy over to some other tree just those files which are recent, then upload that new tree, if rsync isn't your thing.posted by adipocere at 9:21 AM on September 15, 2009
If you had the files on a *nix system, and you wanted to find things that had changed in the last 10 days, you could do: find . -type f -mtime -10
The "-10" can be changed to "-5" for more recently than 5 days, etc. Then you could use the -exec switch in find to run your ftp transfer.
posted by beerbajay at 9:48 AM on September 15, 2009
The "-10" can be changed to "-5" for more recently than 5 days, etc. Then you could use the -exec switch in find to run your ftp transfer.
posted by beerbajay at 9:48 AM on September 15, 2009
Transmit (Mac FTP app) can synchronize only changed files in subdirectories (upload or download).
posted by kirkaracha at 10:02 AM on September 15, 2009
posted by kirkaracha at 10:02 AM on September 15, 2009
<CowboyCoder>Just run the script on production!</CowboyCoder>
posted by nomad at 10:07 AM on September 15, 2009
posted by nomad at 10:07 AM on September 15, 2009
Response by poster: <CowboyCoder>Just run the script on production!</CowboyCoder>
Oh, man, I know. The best part of that is then I could just overwrite the dev site with the live, and not have to worry about accidentally publishing stuff that should only be on dev to the live site. What could POSSIBLY go wrong?
Forgot to mention that I'm a Windows guy - rsync looks cool but scary(in that way that EVERYTHING *nix looks scary), and the windows wrapper version (DeltaCopy) looks a bit iffy, and I'm not doing any cygwin stuff.
Right now I'm leaning towards biting the bullet and doing it manually, just avoiding directories that I know are full of stuff that won't be changed.
posted by dirtdirt at 10:27 AM on September 15, 2009
Oh, man, I know. The best part of that is then I could just overwrite the dev site with the live, and not have to worry about accidentally publishing stuff that should only be on dev to the live site. What could POSSIBLY go wrong?
Forgot to mention that I'm a Windows guy - rsync looks cool but scary(in that way that EVERYTHING *nix looks scary), and the windows wrapper version (DeltaCopy) looks a bit iffy, and I'm not doing any cygwin stuff.
Right now I'm leaning towards biting the bullet and doing it manually, just avoiding directories that I know are full of stuff that won't be changed.
posted by dirtdirt at 10:27 AM on September 15, 2009
seconding Filezilla. Just configure the account and drag the modified (local) directory tree on the remote tree. It'll take a while to scan your disk and then as it encounters the first duplicate file, it'll ask you what to do. You want to overwrite all files that are newer or different (and check the box that says "repeat for all files") and that's it. You can also leave it running overnight and set it up so that it shuts down your computer when it's done.
notice: I assume that everything that's on the production server has been already backed up.
posted by _dario at 1:42 PM on September 15, 2009
notice: I assume that everything that's on the production server has been already backed up.
posted by _dario at 1:42 PM on September 15, 2009
WinSCP has an 'only transfer changed files' flag. (And it's free.)
posted by bhance at 1:02 PM on September 16, 2009
posted by bhance at 1:02 PM on September 16, 2009
Are the files' timestamps a reliable indicator of whether they should be updated or not? If so, you can use Total Commander to search for files newer than a certain timestamp and then copy only those. It has a built-in FTP client.
posted by Nameless at 5:21 PM on September 16, 2009
posted by Nameless at 5:21 PM on September 16, 2009
This thread is closed to new comments.
posted by Oktober at 8:55 AM on September 15, 2009