Version Control for Writers
July 31, 2013 3:04 PM Subscribe
Surely there are some more sophisticated programmers' tools for storing and comparing multiple drafts, or versions, of a written document. Might we use those tools, to study written work, or to keep track of versions both linear and non-linear? Authors, editors, literary scholars, archivists: all eventually have to do work with multiple versions of a text. Can the existing tools be made more readily available to them?
I'm interested in using version control software to help writers, editors, publishers, scholars and archivists. In my efforts to figure this out, I've been introduced to Git, and to Flashbake, and to Git-Annex Assistant. These powerful tools are great! Because I'm a web developer in my day job, I'm sure I'll be able to learn them quickly enough and put them to work. The trouble is, most of the people I work on writing with are not so skilled with computers. They may not want to use the command line, and they probably prefer a word processor over a text editor. Since most of this stuff is open source, I'm looking for ways to make these tools a little more user friendly, for the average writer, editor, etc.
Here is a link to a blog post where I detailed some of my early thoughts on the subject: http://nocategories.net/ephemera/writing/version-control-for-writers/
I would be grateful for any thoughts, comments, advice, etc. that any of you might be able to provide.
I'm interested in using version control software to help writers, editors, publishers, scholars and archivists. In my efforts to figure this out, I've been introduced to Git, and to Flashbake, and to Git-Annex Assistant. These powerful tools are great! Because I'm a web developer in my day job, I'm sure I'll be able to learn them quickly enough and put them to work. The trouble is, most of the people I work on writing with are not so skilled with computers. They may not want to use the command line, and they probably prefer a word processor over a text editor. Since most of this stuff is open source, I'm looking for ways to make these tools a little more user friendly, for the average writer, editor, etc.
Here is a link to a blog post where I detailed some of my early thoughts on the subject: http://nocategories.net/ephemera/writing/version-control-for-writers/
I would be grateful for any thoughts, comments, advice, etc. that any of you might be able to provide.
There are GUI clients for all the popular revision control systems, and there are a variety of visual diff tools available as well. But what seems most popular for writers is MS Word's "track changes" features, largely it seems, because it lets you work in MS Word. You are never going to get good version control software for MS Word documents unless it comes from Microsoft, or Microsoft open-sources it's .doc format, as determining what has changed in a document requires knowing how that document is constructed.
posted by tylerkaraszewski at 3:21 PM on July 31, 2013
posted by tylerkaraszewski at 3:21 PM on July 31, 2013
I think the big thing you're going to have to think about is not just what will help an individual writer do their writing, but what can actually make it up and down the food chain of publishing (while such a formal system exists, which we will have for a while now I think) intact. I used Scrivener to write my book and just made incremental backups of the files I was working on. However my editor would only accept a Word doc file and she returned it to me with track changes on. And later when I had the galleys it was Adobe Reader with track changes on (or whatever their equivalent is) This is what most writers I know use: Word with track changes. So I think a few things about your proposal
- If it's not going to become something that is standard operating procedure at publishing houses, it's going to be tough to get buy-in
- Cloud-based options where people can interact with the same document and not email around attachments solves a lot of problems including this one. Google docs does a good job of this
- It's very easy to make a new tool that does a thing and then say "Why don't people do it the more sensible way?" I work in libraries and we have literally thousands of geeks telling us that they can improve our systems. And I'm sure they can. But our systems are interoperable, and work within a much larger system. And if you can't change that entire system just showing up with your whizbang idea doesn't help.
They may not want to use the command line
Be serious, people who are not already using a command line won't start using one for version control. My suggestion would be to get a thing that works with Open Office and works well. It can take a track-changes Word doc and import it and value add to that something that people would really like to have. It can compare documents and do the diff itself (without you knowing what a diff is) and it can return things that can be exported back into a doc or docx file so you can send it to your publisher who is still using Word 2008.
Unless you have been tasked to do this by one of the Big Six publishers, I would try to find a way to make your idea seamlessly segue into things that people are already using. You'd have more luck getting a version control word processor to take off on facebook than you would to build your own tool and just hope it caught on because it's so sensible.
posted by jessamyn at 3:21 PM on July 31, 2013 [4 favorites]
- If it's not going to become something that is standard operating procedure at publishing houses, it's going to be tough to get buy-in
- Cloud-based options where people can interact with the same document and not email around attachments solves a lot of problems including this one. Google docs does a good job of this
- It's very easy to make a new tool that does a thing and then say "Why don't people do it the more sensible way?" I work in libraries and we have literally thousands of geeks telling us that they can improve our systems. And I'm sure they can. But our systems are interoperable, and work within a much larger system. And if you can't change that entire system just showing up with your whizbang idea doesn't help.
They may not want to use the command line
Be serious, people who are not already using a command line won't start using one for version control. My suggestion would be to get a thing that works with Open Office and works well. It can take a track-changes Word doc and import it and value add to that something that people would really like to have. It can compare documents and do the diff itself (without you knowing what a diff is) and it can return things that can be exported back into a doc or docx file so you can send it to your publisher who is still using Word 2008.
Unless you have been tasked to do this by one of the Big Six publishers, I would try to find a way to make your idea seamlessly segue into things that people are already using. You'd have more luck getting a version control word processor to take off on facebook than you would to build your own tool and just hope it caught on because it's so sensible.
posted by jessamyn at 3:21 PM on July 31, 2013 [4 favorites]
Response by poster: thanks for the comment so far. some quick responses...
GUI clients do exist for most version control systems, but they're designed with coders in mind. Writers might want a slightly different GUI and/or featureset.
I'm thinking non-cloud-based would provide greater flexibility for things like the inevitable need to convert to .doc format, etc.
For what it's worth, I am perhaps least interested in the publishers.
As for tracked changes in word, those are quite useful, to a point, but they don't handle forked versions, or merging changes from one version to another.
posted by dylan_k at 3:29 PM on July 31, 2013
GUI clients do exist for most version control systems, but they're designed with coders in mind. Writers might want a slightly different GUI and/or featureset.
I'm thinking non-cloud-based would provide greater flexibility for things like the inevitable need to convert to .doc format, etc.
For what it's worth, I am perhaps least interested in the publishers.
As for tracked changes in word, those are quite useful, to a point, but they don't handle forked versions, or merging changes from one version to another.
posted by dylan_k at 3:29 PM on July 31, 2013
Draft sounds like something that may help.
posted by Blandanomics at 3:40 PM on July 31, 2013 [1 favorite]
posted by Blandanomics at 3:40 PM on July 31, 2013 [1 favorite]
Best answer: It is an interesting topic, and a challenging one. I'd suggest that while your motivations are admirable, your approach is already off on the wrong foot.
Making these developer-oriented tools a little easier for writers and editors who are strongly biased to working with word processors, you are unlikely to get very far. Your intended audience (ie. those who are tied to MS Word) are inherently conservative. They have a tool they know that works well enough for them that they probably aren't even interested in exploring other features already in the tool that might make their life easier.
As an alternative, I'd suggest that you shift your initial focus to people who are already exploring alternatives beyond the mainstream (ie MS Word). Perhaps look to the people who are embracing minimalist writing tools, like various markdown-based editors. Start with them, talk to a bunch of them about their writing process, and then go from there. Not only are they more likely to be open to new ways of working, they're more likely to forgive tools that are a bit rough around the edges.
If you can satisfy these people they can help evangelize to their more conservative colleagues and if it makes them noticeably more productive, it also creates pressure for "competitors" to change too.
A few related thoughts:
First, git, etc are probably overkill, rather than trying to adapt an existing version control tool, you might be better off starting from scratch.
Second, If you are intrigued by flashbake, you could try taking things to the next level by making it easier to install on Mac and Windows. The easier you make the first step, the better.
Third, writer/editor James Fallows actually did a stint as a program manager working on Microsoft Word about a decade and a half-ago. His focus was revamping the revision tracking and editing tools, which, last I looked, haven't had a big change since. He also has a long history of seeking out, learning, using and writing about writing software. It wouldn't hurt to try to engage with him.
Fourth, recent versions of OS X actually automatically store previous versions of files, and Windows has a similar feature (Shadow Copies), though I'm not sure it is turned on automatically. Building on these features might be the easiest way to reach people in the tools they already use (see above though for why that might not be the best way to get started).
posted by Good Brain at 4:31 PM on July 31, 2013 [2 favorites]
Making these developer-oriented tools a little easier for writers and editors who are strongly biased to working with word processors, you are unlikely to get very far. Your intended audience (ie. those who are tied to MS Word) are inherently conservative. They have a tool they know that works well enough for them that they probably aren't even interested in exploring other features already in the tool that might make their life easier.
As an alternative, I'd suggest that you shift your initial focus to people who are already exploring alternatives beyond the mainstream (ie MS Word). Perhaps look to the people who are embracing minimalist writing tools, like various markdown-based editors. Start with them, talk to a bunch of them about their writing process, and then go from there. Not only are they more likely to be open to new ways of working, they're more likely to forgive tools that are a bit rough around the edges.
If you can satisfy these people they can help evangelize to their more conservative colleagues and if it makes them noticeably more productive, it also creates pressure for "competitors" to change too.
A few related thoughts:
First, git, etc are probably overkill, rather than trying to adapt an existing version control tool, you might be better off starting from scratch.
Second, If you are intrigued by flashbake, you could try taking things to the next level by making it easier to install on Mac and Windows. The easier you make the first step, the better.
Third, writer/editor James Fallows actually did a stint as a program manager working on Microsoft Word about a decade and a half-ago. His focus was revamping the revision tracking and editing tools, which, last I looked, haven't had a big change since. He also has a long history of seeking out, learning, using and writing about writing software. It wouldn't hurt to try to engage with him.
Fourth, recent versions of OS X actually automatically store previous versions of files, and Windows has a similar feature (Shadow Copies), though I'm not sure it is turned on automatically. Building on these features might be the easiest way to reach people in the tools they already use (see above though for why that might not be the best way to get started).
posted by Good Brain at 4:31 PM on July 31, 2013 [2 favorites]
You might look and see if there are any version control systems hooked into the LaTeX typesetting language.
I also was going to say that this might be something better left to the filesystem in all but the largest of shops. Either it automatically saves old versions (via Dropbox maybe?), or users are trained to save their revisions with unique filenames.
Also, perhaps a wiki can be modified to work the way you want it to. Set up correctly, it probably wouldn't be all that difficult to set it up such that a final draft of version 1 of the document could be "locked" in a way, and then programatically printed to PDF or LaTeX or .doc or whatever the final format would be, and then any subsequent changes would automatically fork to version 2 of the document.
posted by gjc at 4:44 PM on July 31, 2013
I also was going to say that this might be something better left to the filesystem in all but the largest of shops. Either it automatically saves old versions (via Dropbox maybe?), or users are trained to save their revisions with unique filenames.
Also, perhaps a wiki can be modified to work the way you want it to. Set up correctly, it probably wouldn't be all that difficult to set it up such that a final draft of version 1 of the document could be "locked" in a way, and then programatically printed to PDF or LaTeX or .doc or whatever the final format would be, and then any subsequent changes would automatically fork to version 2 of the document.
posted by gjc at 4:44 PM on July 31, 2013
For what it's worth, I am perhaps least interested in the publishers.
But writers are very interested in publishers; it was when I got my agent and she sent me multiple manuscripts with changes tracked in MS Word that I realized openoffice.org was no longer going to cut it. Because this is the industry standard, and the way that professionals all communicate, it's how I do things, too.
(Like Jessamyn, I do version control in Scrivener, and then export to MS Word.)
Also, as someone who has to stare at some type of writing software all day, I definitely want it to be relatively intuitive, as well as easy on the eyes. Command line software is a no-go for me, and this is speaking as someone who has mucked around in linux previously.
posted by PhoBWanKenobi at 5:14 PM on July 31, 2013 [1 favorite]
But writers are very interested in publishers; it was when I got my agent and she sent me multiple manuscripts with changes tracked in MS Word that I realized openoffice.org was no longer going to cut it. Because this is the industry standard, and the way that professionals all communicate, it's how I do things, too.
(Like Jessamyn, I do version control in Scrivener, and then export to MS Word.)
Also, as someone who has to stare at some type of writing software all day, I definitely want it to be relatively intuitive, as well as easy on the eyes. Command line software is a no-go for me, and this is speaking as someone who has mucked around in linux previously.
posted by PhoBWanKenobi at 5:14 PM on July 31, 2013 [1 favorite]
Not all writing is the same, nor all version control situations are alike. I believe writing for the screen is much more collaborative than other fiction writing, so it may benefit more from version control between different authors. Screenwriter John August has designed Fountain, a markup language specific for scripts, and Highland, an app that uses Fountain as its format. (Note: I'm not a user, I just know these exist). The advantage of text formats like Markdown is that you can use regular source control and diff programs, but you probably know that already. And the advantage of having a well-known writer as its designer is that you have some traction.
Blandanomics: holy moly Draft looks great. And it's a web application, so usable by Mac OS, Windows and Linux users (they don't say it anywhere obvious on their About and Read More pages, I thought it would be a Mac OS-only app).
posted by kandinski at 5:17 PM on July 31, 2013 [1 favorite]
Blandanomics: holy moly Draft looks great. And it's a web application, so usable by Mac OS, Windows and Linux users (they don't say it anywhere obvious on their About and Read More pages, I thought it would be a Mac OS-only app).
posted by kandinski at 5:17 PM on July 31, 2013 [1 favorite]
I wouldn't commit the sin of self-promotion unless I thought it was absolutely relevant. Which I guess means I'll be committing that sin.
About a year ago, my friend and I wanted to write a novel collaboratively, but we didn't much care for any of the tools available at the time. My friend is decent with a computer, but I'm a software developer, which means our tolerance for toolsets varies wildly. I decided to build us a tool to work in, centered around Git, Markdown, and wiki-style web-based document editing.
I call it Giterary.
Currently, it's a pain to set up (which is why I don't have anything available for download, yet), and a lot of the features I've been working on haven't quite made it to the public facing sites. However, the docs I have on there give some idea of my philosophy in building it, and some of the things one can do with it. It doesn't quite compete with Draft or Google Docs for UI stuff, but it makes the right choices in maintaining version control and reasonable document organization, and is informed by some of my own paranoid, vague technological elitism.
I'm still working on it, and hopefully, someday, I'll get a public release out there. Until then, hope this gives you an idea of how we're using source control to manage our novel.
posted by kaiden11 at 5:53 PM on July 31, 2013 [4 favorites]
About a year ago, my friend and I wanted to write a novel collaboratively, but we didn't much care for any of the tools available at the time. My friend is decent with a computer, but I'm a software developer, which means our tolerance for toolsets varies wildly. I decided to build us a tool to work in, centered around Git, Markdown, and wiki-style web-based document editing.
I call it Giterary.
Currently, it's a pain to set up (which is why I don't have anything available for download, yet), and a lot of the features I've been working on haven't quite made it to the public facing sites. However, the docs I have on there give some idea of my philosophy in building it, and some of the things one can do with it. It doesn't quite compete with Draft or Google Docs for UI stuff, but it makes the right choices in maintaining version control and reasonable document organization, and is informed by some of my own paranoid, vague technological elitism.
I'm still working on it, and hopefully, someday, I'll get a public release out there. Until then, hope this gives you an idea of how we're using source control to manage our novel.
posted by kaiden11 at 5:53 PM on July 31, 2013 [4 favorites]
Response by poster: @Blandanomics Draft would be good, except it only supports linear progression of the writing (so far as I can tell) and it's a webapp, so what if it goes away?
posted by dylan_k at 6:30 PM on July 31, 2013
posted by dylan_k at 6:30 PM on July 31, 2013
Response by poster: Regarding Microsoft Word, publishers and the industry standard: I think that as long as the final (or currently final) version of the writing can be easily converted into an industry-standard format, the users should be ok.
They might be even happier though, if all they had to do was "save as" to an open, text-based format from their word processor of choice, and then take advantage of more powerful version control features, from within a separate application.
Is it possible for writers to have their cake (version control) and eat it too (a word processor)?
posted by dylan_k at 6:56 PM on July 31, 2013
They might be even happier though, if all they had to do was "save as" to an open, text-based format from their word processor of choice, and then take advantage of more powerful version control features, from within a separate application.
Is it possible for writers to have their cake (version control) and eat it too (a word processor)?
posted by dylan_k at 6:56 PM on July 31, 2013
The problem is that we need version control tracking just as much on pieces that we're working with our editors on as we do the "final" product. My books average 5 revisions after I send them to my editor. Each of those revisions happen via track changes in microsoft word.
They might be even happier though, if all they had to do was "save as" to an open, text-based format from their word processor of choice, and then take advantage of more powerful version control features, from within a separate application.
This sounds convoluted and wouldn't work for me with my back-up habits (cloud storage, autosave).
posted by PhoBWanKenobi at 7:38 PM on July 31, 2013 [1 favorite]
They might be even happier though, if all they had to do was "save as" to an open, text-based format from their word processor of choice, and then take advantage of more powerful version control features, from within a separate application.
This sounds convoluted and wouldn't work for me with my back-up habits (cloud storage, autosave).
posted by PhoBWanKenobi at 7:38 PM on July 31, 2013 [1 favorite]
You might look and see if there are any version control systems hooked into the LaTeX typesetting language.
There are some people out there using git/svn/mercurial with LaTeX, but they're, well, the sort of people who'd do such a thing. There's some discussion of it here. However, I suspect the sort of people who do this (and I've used git with LaTeX) aren't the sort of people dylan_k is thinking about.
posted by hoyland at 8:24 PM on July 31, 2013 [1 favorite]
There are some people out there using git/svn/mercurial with LaTeX, but they're, well, the sort of people who'd do such a thing. There's some discussion of it here. However, I suspect the sort of people who do this (and I've used git with LaTeX) aren't the sort of people dylan_k is thinking about.
posted by hoyland at 8:24 PM on July 31, 2013 [1 favorite]
Component content management systems like Flare and Author-it allow for version control of document components like topics, chapters, graphics, etc. They're commonly used for technical documentation but could be adapted for any type of document. They normally include WYSIWYG editors that look like a stripped-down version of MS Word, although advanced formatting usually requires a bit of technical knowledge.
posted by neushoorn at 12:37 AM on August 1, 2013
posted by neushoorn at 12:37 AM on August 1, 2013
or Microsoft open-sources it's .doc format
That will probably never happen, but the .docx format is already (amid some controversy) part of a published ISO standard. You can change the extension to .zip and open it up with standard tools to look at all the XML files.
But writers are very interested in publishers
The publisher I've worked with has a DocBook based authoring system, authors who don't simply write their own tools (this is a tech book publisher, after all) are encouraged to use a WYSIWYG XML authoring tool. The system comes with a tool which will generate a PDF from the XML source if authors want to see what it will look like in print. Authors can and do keep all their DocBook sources in version control systems. This all then plugs into the same typesetting system as the Word-driven author uses.
Word was always a bad fit for this system anyway, the only reason it works at all is because they've devoted a lot of time writing Word templates to enforce styles and macros to do things like tag words for indexing. All of this also has the effect of making it hard to use any other word processor than Word. Certainly as an author primarily using Linux I found the Word-based workflow a complete PITA, files frequently got corrupted as the went between Open/LibreOffice and Word.
posted by robertc at 3:00 PM on August 1, 2013 [1 favorite]
That will probably never happen, but the .docx format is already (amid some controversy) part of a published ISO standard. You can change the extension to .zip and open it up with standard tools to look at all the XML files.
But writers are very interested in publishers
The publisher I've worked with has a DocBook based authoring system, authors who don't simply write their own tools (this is a tech book publisher, after all) are encouraged to use a WYSIWYG XML authoring tool. The system comes with a tool which will generate a PDF from the XML source if authors want to see what it will look like in print. Authors can and do keep all their DocBook sources in version control systems. This all then plugs into the same typesetting system as the Word-driven author uses.
Word was always a bad fit for this system anyway, the only reason it works at all is because they've devoted a lot of time writing Word templates to enforce styles and macros to do things like tag words for indexing. All of this also has the effect of making it hard to use any other word processor than Word. Certainly as an author primarily using Linux I found the Word-based workflow a complete PITA, files frequently got corrupted as the went between Open/LibreOffice and Word.
posted by robertc at 3:00 PM on August 1, 2013 [1 favorite]
This thread is closed to new comments.
posted by jacalata at 3:17 PM on July 31, 2013 [2 favorites]