Version control with automated deployment?
December 18, 2011 12:28 PM Subscribe
I'd like to start using source control for a website, with automated, one-click (or close to it) deployment.
Is there a reason to prefer subversion, mercurial, or git?
Are there tutorials on how to do this? Best practices? Etc?
Is there a reason to prefer subversion, mercurial, or git?
Are there tutorials on how to do this? Best practices? Etc?
Heroku will let you deploy with a git push and nothing else.
posted by ConstantineXVI at 12:42 PM on December 18, 2011 [1 favorite]
posted by ConstantineXVI at 12:42 PM on December 18, 2011 [1 favorite]
Best answer: The VCS isn't that important... most build tools have plugins for most version control systems.
Best build tool to use depends on the complexity of your build. Are we talking about static HTML or a dynamic, database-backed site? If dynamic, what language is it written in? Are you using an off-the-shelf system like, say, Wordpress? Is site configuration stored in the database, or on disc? How many servers?
(Assuming we're talking static HTML, a Linux host and your team isn't distributed, I'd look at Capistrano/Fabric/Gradle and SVN. Briefly you want each release to be tagged with a name in the VCS, then you want to push that release into a new folder on the server, with the name of the new folder base on the tag. Then you symlink the webroot to that folder, and restart the web server. This means the flip between versions happens almost instantaneously, you're never serving a partially-uploaded site, and you have old versions of the site hanging around that you can flip back to in an emergency. A more complex site might have a more complex build process... eg creating configuration files or flushing a CDN).
posted by Leon at 1:11 PM on December 18, 2011
Best build tool to use depends on the complexity of your build. Are we talking about static HTML or a dynamic, database-backed site? If dynamic, what language is it written in? Are you using an off-the-shelf system like, say, Wordpress? Is site configuration stored in the database, or on disc? How many servers?
(Assuming we're talking static HTML, a Linux host and your team isn't distributed, I'd look at Capistrano/Fabric/Gradle and SVN. Briefly you want each release to be tagged with a name in the VCS, then you want to push that release into a new folder on the server, with the name of the new folder base on the tag. Then you symlink the webroot to that folder, and restart the web server. This means the flip between versions happens almost instantaneously, you're never serving a partially-uploaded site, and you have old versions of the site hanging around that you can flip back to in an emergency. A more complex site might have a more complex build process... eg creating configuration files or flushing a CDN).
posted by Leon at 1:11 PM on December 18, 2011
Most SCMs come with hook functions (e.g. Git, SVN). So if the content is all static (HTML and images), then you can have it automatically execute a script after each commit.
As far as which one you choose, there's no easy answer. You'll have to compare and contrast them to see which one best suits your application.
posted by spiderskull at 1:13 PM on December 18, 2011
As far as which one you choose, there's no easy answer. You'll have to compare and contrast them to see which one best suits your application.
posted by spiderskull at 1:13 PM on December 18, 2011
In increasing order of complexity:
posted by djb at 2:02 PM on December 18, 2011
- Push static HTML to GitHub Pages
- Push content to GitHub Pages and use their Jekyll install
- Use a different CMS and push to your own server
posted by djb at 2:02 PM on December 18, 2011
Prefer Mercurial or Git - they are (roughly equivalent) better solutions to the problem of revision control. Git seems to be the winner in terms of usage, so I would go with Git, as it and Mercurial are basically equivalent.
As far as deployment, there isn't a standard, and everything depends on what your website needs and how your website works. You'll have to write your won "post-commit hooks" or "commit hooks" to make sure that when content is changed, it gets changed on your webserver just right.
posted by pmb at 2:09 PM on December 18, 2011
As far as deployment, there isn't a standard, and everything depends on what your website needs and how your website works. You'll have to write your won "post-commit hooks" or "commit hooks" to make sure that when content is changed, it gets changed on your webserver just right.
posted by pmb at 2:09 PM on December 18, 2011
Git, if you're on the Microsoft stack. Use Appharbor.
posted by blue_beetle at 2:12 PM on December 18, 2011
posted by blue_beetle at 2:12 PM on December 18, 2011
Best answer: Beanstalkapp provides both hosted version control (svn and git) and deployment. Take a look at their guides for tutorials.
posted by hooray at 2:20 PM on December 18, 2011
posted by hooray at 2:20 PM on December 18, 2011
You'll have to write your won "post-commit hooks" or "commit hooks" to make sure that when content is changed, it gets changed on your webserver just right.
Blindly copying every commit to a live server with a commit hook is little better than editing directly on live. Use tagged releases, preferably to a staging server then to live.
We really need to know more about your current setup before we can help you automate it.
(Please be wary of people pushing you towards dVCS without understanding your situation.)
posted by Leon at 3:27 PM on December 18, 2011
Blindly copying every commit to a live server with a commit hook is little better than editing directly on live. Use tagged releases, preferably to a staging server then to live.
We really need to know more about your current setup before we can help you automate it.
(Please be wary of people pushing you towards dVCS without understanding your situation.)
posted by Leon at 3:27 PM on December 18, 2011
Response by poster: Thanks for everything so far.
The current setup is... SFTP, backups in folders named according to date, SQL databases backed up and sent via SCP to a different server nightly.
The software is WordPress & similar php scripts (phpBB, FluxBB, osCommerce, phplist) on one domain, plus WP alone on a few other domains, each with customized, from-scratch themes (the main thing being changed).
I also am working on a side project that isn't live in Python with CherryPy, plus various plugins for WP I've written. For them, deployment isn't much of an issue.
I would like to be able to check in revisions for the live sites and then hit one button to make them go live. (Intelligently, so that only the changed files are altered, for example.)
I'd like to be able to roll back to any previous version with one click as well, I suppose, since eventually I'm sure something will work on the dev environment but not on the live site.
I am a huge fan of developing in XAMPP, and if it could generate a complete, preconfigured XAMPP folder with one click that simply needs to be downloaded to work, I would like that. It seems like that would be a good way to simplify concurrent development with collaborators, too. But if that's just stupid, tell me.
I can whip up scripts in PHP, Python, and BASH to help make any of this happen. (JavaScript and Visual Basic, too, but I don't think they are relevant.)
Anything else that would be good to know?
posted by jsturgill at 4:27 PM on December 18, 2011
The current setup is... SFTP, backups in folders named according to date, SQL databases backed up and sent via SCP to a different server nightly.
The software is WordPress & similar php scripts (phpBB, FluxBB, osCommerce, phplist) on one domain, plus WP alone on a few other domains, each with customized, from-scratch themes (the main thing being changed).
I also am working on a side project that isn't live in Python with CherryPy, plus various plugins for WP I've written. For them, deployment isn't much of an issue.
I would like to be able to check in revisions for the live sites and then hit one button to make them go live. (Intelligently, so that only the changed files are altered, for example.)
I'd like to be able to roll back to any previous version with one click as well, I suppose, since eventually I'm sure something will work on the dev environment but not on the live site.
I am a huge fan of developing in XAMPP, and if it could generate a complete, preconfigured XAMPP folder with one click that simply needs to be downloaded to work, I would like that. It seems like that would be a good way to simplify concurrent development with collaborators, too. But if that's just stupid, tell me.
I can whip up scripts in PHP, Python, and BASH to help make any of this happen. (JavaScript and Visual Basic, too, but I don't think they are relevant.)
Anything else that would be good to know?
posted by jsturgill at 4:27 PM on December 18, 2011
Best answer: Commit hooks are the way to go here. They're generally shell scripts that are triggered during the commit. For example, some people use pre-commit hooks to reject commits with bad changelog spelling, or are too short. Post commit hooks have historically been used to notify all developers of the change. It's entirely possible to have a commit hook upload a new version and reload your web servers though.
You'd set up a post commit hook. Leon suggests using a tag in case you want to commit and share code without deploying it, and it's not a bad idea. If you had a lot of automated QA, you might be tempted to run the test suite ina precommit hook, but they're generally way too slow for that. Instead, you can use dVCS to commit to a QA repo, which has a post commit hook to test and push to the tree other developers pull from if it passes. Very complicated though.
Rolling back is a bit tricky; if you had a working version v1, and deployed a broken version v2, lthe simplest method might be to commit a version v3 with the code from v1. Most VCS tools have some command to do this.
One advantage I find that SVN and Hg have over git is external repositories. SVN lets you define a mountpoint for code. This is especially useful if your customizations to Wordpress are entirely contained within a single directory. I'm told mercurial's tools allow you to cross version control systems, whereas I know that SVN only allows externals to other SVN systems. A prolific UNIX hacker who prefers git did implement a secondary tool mr (multi-repository) to get this feature, but I haven't tried it out yet.
posted by pwnguin at 9:46 PM on December 18, 2011
You'd set up a post commit hook. Leon suggests using a tag in case you want to commit and share code without deploying it, and it's not a bad idea. If you had a lot of automated QA, you might be tempted to run the test suite ina precommit hook, but they're generally way too slow for that. Instead, you can use dVCS to commit to a QA repo, which has a post commit hook to test and push to the tree other developers pull from if it passes. Very complicated though.
Rolling back is a bit tricky; if you had a working version v1, and deployed a broken version v2, lthe simplest method might be to commit a version v3 with the code from v1. Most VCS tools have some command to do this.
One advantage I find that SVN and Hg have over git is external repositories. SVN lets you define a mountpoint for code. This is especially useful if your customizations to Wordpress are entirely contained within a single directory. I'm told mercurial's tools allow you to cross version control systems, whereas I know that SVN only allows externals to other SVN systems. A prolific UNIX hacker who prefers git did implement a secondary tool mr (multi-repository) to get this feature, but I haven't tried it out yet.
posted by pwnguin at 9:46 PM on December 18, 2011
Oh yea, one major caveat: databases tend to fuck everything up. Anything that changes schema will be problematic because you can't roll back without risking data loss. This isn't any different than the standard quo however. Just a stumbling block to the roll-back philosophy.
posted by pwnguin at 9:48 PM on December 18, 2011
posted by pwnguin at 9:48 PM on December 18, 2011
Response by poster: Looking at the thread, it looks like I should just read through the Git tutorials and figure things out, or pay Beanstalk to do it for me.
These questions came to mind:
Is there a common way to decouple commits and deployment, so that deployment must be manually, seperately invoked?
What else should I look at?
Have you used Git for live web projects and have any pearls of hard-earned wisdom to drop on me?
posted by jsturgill at 7:53 AM on December 19, 2011
These questions came to mind:
Is there a common way to decouple commits and deployment, so that deployment must be manually, seperately invoked?
What else should I look at?
Have you used Git for live web projects and have any pearls of hard-earned wisdom to drop on me?
posted by jsturgill at 7:53 AM on December 19, 2011
Best answer: Take a look at fabric. It's the Python tool to do what you're trying to do.
(Intelligently, so that only the changed files are altered, for example.)
There are lots of reasons not to do this. It's better to start from scratch with each new build - otherwise a change to a live file can get "carried along" by the build process and you won't know it. Changes are no longer atomic - if you've changed 50 files there will be a point somewhere during the release where you're serving 25 new files and 25 old files. etc etc.
Is there a common way to decouple commits and deployment, so that deployment must be manually, seperately invoked?
Yes; don't use post-commit hooks. I'd do something like this:
Have you used Git for live web projects and have any pearls of hard-earned wisdom to drop on me?
I've used RCS, sourcesafe, SVN, git and I'm currently using hg at work and git at home. SVN's graphical tools are the most mature; if you're on Windows start with TortoiseSVN and WinMerge. Git is where the world's going, but honestly, you don't need the advanced features it offers (it is faster, though).
posted by Leon at 8:08 AM on December 19, 2011
(Intelligently, so that only the changed files are altered, for example.)
There are lots of reasons not to do this. It's better to start from scratch with each new build - otherwise a change to a live file can get "carried along" by the build process and you won't know it. Changes are no longer atomic - if you've changed 50 files there will be a point somewhere during the release where you're serving 25 new files and 25 old files. etc etc.
Is there a common way to decouple commits and deployment, so that deployment must be manually, seperately invoked?
Yes; don't use post-commit hooks. I'd do something like this:
- Tag a release in source control (it doesn't have to be a tag it could be a revision number, but don't use HEAD/tip. It has to be a fixed point.)
- Run a fabric script that does the following:
- Check the named tag/revision out of source control onto the local machine
- Generate config files, replace placeholders in the code (you don't want your db password stored in source control)
- Wrap the whole thing up as a .tar.gz
- scp the tar to the live machine
- untar into a new directory
- symlink any static asset folders into your new directory
- point your webroot at the new directory
- restart server, flush caches, etc
- tidy-up. remove the tars and the local checkout
Have you used Git for live web projects and have any pearls of hard-earned wisdom to drop on me?
I've used RCS, sourcesafe, SVN, git and I'm currently using hg at work and git at home. SVN's graphical tools are the most mature; if you're on Windows start with TortoiseSVN and WinMerge. Git is where the world's going, but honestly, you don't need the advanced features it offers (it is faster, though).
posted by Leon at 8:08 AM on December 19, 2011
Response by poster: Fabric seems about perfect. Thanks for the step-by-step. Seems like I might be able to take a weekend and get some sort of workable process down.
I'll do my googling and reading up, but any specific tutorials, videos, talks, or other resources about Fabric and Git would be helpful. (I'd like to start with Git and then go back to learn SVN as life/employers/projects dictate.)
posted by jsturgill at 8:30 AM on December 19, 2011
I'll do my googling and reading up, but any specific tutorials, videos, talks, or other resources about Fabric and Git would be helpful. (I'd like to start with Git and then go back to learn SVN as life/employers/projects dictate.)
posted by jsturgill at 8:30 AM on December 19, 2011
Best answer: It sounds like you're feeling your way towards Continuous Integration.
StackOverflow is often a better forum for programming questions. Try http://stackoverflow.com/questions/tagged/continuous-integration?sort=votes&pagesize=15
posted by richb at 10:04 AM on December 19, 2011
StackOverflow is often a better forum for programming questions. Try http://stackoverflow.com/questions/tagged/continuous-integration?sort=votes&pagesize=15
posted by richb at 10:04 AM on December 19, 2011
Response by poster: Thanks for the terminology. It is a huge help to know what things are called.
posted by jsturgill at 11:08 AM on December 19, 2011
posted by jsturgill at 11:08 AM on December 19, 2011
Best answer: For what it's worth the purpose of Continuous Integration is to run tests on (almost) every commit. If you don't have any tests, you don't need it yet. If you automatically release to live any build that passes all its tests, that's Continuous Deployment.
posted by Leon at 12:14 PM on December 19, 2011
posted by Leon at 12:14 PM on December 19, 2011
This thread is closed to new comments.
Deployment is a wildly different question, and pretty much decoupled with scm. It's be useful to know what you're deploying. If you're just doing HTML, you can use GitHub Pages and have your version control and deployment all in the same cake.
posted by tmcw at 12:36 PM on December 18, 2011