Successful collaboration on large, technical Word documents?

December 28, 2009 10:22 PM

Scientists and other academics: Please advise on strategies for successful collaboration on large, technical Word documents.

I work at a research institute, and half my job is an administrative/accounting position supporting a team of scientists and engineers. Basically, I'm their Girl Friday. Unfortunately, ever since they figured out that I have a decent grasp of grammar, punctuation, and usage and an eye for nit-picky details, I have also been their copyeditor. ohgodpleasemakethepainstop.

I've come to the realization that roughly 25% of the time we spend on document creation, collaboration, revision, formatting, etc. is unnecessarily wasted time caused by the fact that many (most?) people here either a) don't how to use Word very well and/or b) don't plan their collaborative writing projects very well.

My coworkers' Word skills run the gamut from treating Word like a typewriter (i.e. hitting Enter several times when they want to start a new page instead of inserting a page break) to a decent comfort level with styles, captions/cross-references, citation management, etc. No one (myself included) knows how to use Word's advanced features for managing long documents. I plan to address our Word deficiencies by mastering the program myself, then I will assemble links to existing online videos, documentation, etc., into a little self-study course customized around the type of documents we produce. (So, heavy coverage on things like captions/cross-referencing for figures and tables, and no coverage of never-used features like mail merge.)

I don't know how to teach them to plan their collaborative writing projects better, though, because I don't know the answers myself. I just know that there has to be a better way than how we're doing it now! For example, it's only AFTER they've already written most of a 100+ page document that they will confer and then come to me, the copyeditor, to tell me about all the different variations of terms the four authors of the document have used and which particular consistent term and spelling these should all be replaced with. I ask them, "Shouldn't you have agreed on this sort of thing BEFORE you all started writing?" Their response is a sheepish, "Probably," but these large collaborative writing projects are infrequent enough that they don't seem to ever learn from their mistakes.

I need to fix them! Or I will lose my mind! (Most or all should be receptive to being "fixed" -- everyone agrees that the way we're doing things now is painfully inefficient.)

So, any advice? Resources? Books, articles, checklists on how to do the collaboration thing more efficiently? Anything specific to scientists, engineers, or academics would be especially helpful.

(Unfortunately, new software better suited for this type of work is not an option -- half my coworkers have resisted upgrading from Word 2003, so there's no way I'm going to convince them to learn a new program even if it is a much better tool.)

Thank you!

posted by Jacqueline to writing & language (18 answers total)

Really? What science/engineering field is this that Word is prevalent over LaTeX? The great part about LaTeX is you can stick it in source control - even if the profs/researchers don't want to bother with it, they can email stuff to you and you can maintain the repo yourself. No recommendations for Word itself, unfortunately, especially if people aren't into learning it enough to stick to a coherent set of standards - you're going to probably end up doing a lot of cleanup and management no matter how you slice it.
posted by devilsbrigade at 10:44 PM on December 28, 2009 [5 favorites]

Could you perhaps come up with a template or a handful that would be the standardized format for handing in each step of the process? Switching over to forms might prove more efficient.
posted by cmgonzalez at 10:48 PM on December 28, 2009

Could you use Google Docs and then export the result to Word later? The interface is very similar to Word (minus a lot of tools they won't need for the composing stage). Plus you won't need to worry about merging documents and keeping track of the "master" copy.

Later, when you're close to finished, you can turn revision control on in Word and email out a draft for their final comments.
posted by Popular Ethics at 11:06 PM on December 28, 2009

I hear your pain. I'm an academic who worked on such a collaborative writing project with 10 co-authors for a textbook (yes, we did finish it, and no-one died). We used Word because it was the only think everyone could cope with and we had to downgrade it one version so everyone's computer could read the chapters. No advanced options were used. We couldn't even do 'track changes'. No one used bibliographic software.

If your crew is anything like mine, then no one will bother with any tutorials or learning modules or whatever. Captions/cross referencing etc.? Are you kidding me? None of my crew would go near this stuff no matter what you did. Remember, they're happy with their ignorance and don't want to waste their time with these things.

You might have luck with introducing bibliographic software, but not necessarily.

I know this doesn't sound hopeful, but I recommend that once a week (or pick a regular time) you email the group and ask questions like:
-Have you been writing with terms or ideas this week which might need to be standardized with the group?
-Did you enter any sources referred to into the bibliographic software?
-Label your tables/cross reference (etc.)?

We did it by always assigning one person the guardian of the primary draft (of the whole manuscript or just the section or chapter) and any changes which were made (in another text colour, or if they were able with track changes) had to be given to that person. This person in charge shifted around.

We had physical meetings every 2 weeks and subgroup/task meetings in between. Things like the glossary were divided up and compiled to the 'person in charge' who flagged back consistency issues and retasked everybody to comply to the standard (which was decided by the group at our regular meeting).

One feature was a bit unusual about our project. No one section or task could ever go to one person, it had to be more. Every chapter was written by 2 or more people, and then proofed by 2 other people. That way, consistency errors were picked up and continuity was good across chapters. Even spelling, grammar, tables, glossary sections, everything was done by a buddy system and then checked by 2 other people.

I have several points from these examples:
1. we did our project completely low tech and despite perhaps what may be perceived (accurately) as an even lower level than you describe (email and basic Word), we completed the project (300+ pgs) without the hell you describe.

2. They are relying on you because they can. We'd have abused you too if we'd have had someone like you. But we didn't.

3. Just because it's your job to support them, doesn't mean that you shouldn't just task most of these things back to your group ("I'm seeing discrepancies about how x is being used as a term, please go back and fix it in all of your sections since this may require context changes"). Obviously you can do the SEARCH/REPLACE but that's not what's making you crazy.

4. Just because they see that the work flow is poor, does not mean in any way that they will accept process or technological changes you suggest. In fact, many of these are likely to make things worse.

The bottom line is that they are lazy in terms of their commitment to the social processes of the collaborative nature of the project. Until they accept that, they will continue to compensate for this failure by relying on you, and the technical solutions will not be sufficient to address the problem.

Your role is to put everything back on them and to suggest that they work together to pick a solution (you would be willing to provide them with options for them to work through together....). This is not a technical problem, it's a social problem and they have to fix it.

And about the 'ohgodpleasemakethepainstop': you can not 'fix them'. You can change how you respond to their requests. And it is your choice about whether or not you choose to 'lose your mind' because they refuse to see the value in educational modules on picky esoteric word functions like paragraph formatting.

Good luck!
posted by kch at 11:14 PM on December 28, 2009 [9 favorites]

How are files shared between your coworkers? Do you have a central repository for collaborative projects, where everybody can see what everybody else is up to? Try to come up with a simple and easy way for everybody to push their changes to the shared space as early and as often as possible, to help everybody stay on the same page and minimize the coordination work you have to do afterwards.

Like devilsbrigade says, LaTeX would help you do this in a very efficient manner, but your real problem is one of policy, not method: you can still do a lot with the tools that you have if you come up with a good scheme for using them.

The technical aspect doesn't have to be fancy: Some shared network storage, where everybody copies their files with a sequential naming convention is still much better for a small group than ad hoc e-mailing of files on the last day of a project deadline.
posted by Dr Dracator at 11:18 PM on December 28, 2009

The comment by kch above perfectly captures what I'm talking about: If your group is not technicaly minded, you need to simplify the tools used and rely less on technology, and spend your time enforcing good workflow.
posted by Dr Dracator at 11:23 PM on December 28, 2009

I sometimes have to deal with those kind of issues, and I have to deal with them in a severely restricted environment, IT-wise.

I find that completely ignoring the formatting (crtl-a, ctrl-c, ctrl-v into notepad, ctrl-a, ctrl-c, ctrl-c in a blank doc) is often the best policy. I sometimes use Word MVPS and have bought Word Annoyances, which was fairly useful. Learn to use metacharacters in search and replace (., *, +, []). For some stuff, just dropping all the text in a proper text editor (I use Emacs, which has a pretty steep learning curve; I hear notepad++ is pretty good) may be faster than trying to coax Word into doing what you want. It can however mean that you have to re-do the formatting.

You'll probably want to work with the most recent Word version you can get (it's probably worth it to learn 2007 now since you don't have that much prior knowledge): later versions tend to include some very useful features. For instance, the document comparison in 2003 is worlds better than 2000's, and 2003 includes a "select all" checkbox in the search dialog that you can use to put a certain styling on every occurence of a certain word or phrase.
posted by Monday, stony Monday at 11:25 PM on December 28, 2009

On the other end of things, you may find learning skills such as regular expressions helpful for tracking down and modifying all the weird and inconsistent spellings, acronyms, etc. Many text editors support them, and they have saved me quite a bit of hassle.
posted by beepbeepboopboop at 12:23 AM on December 29, 2009

What science/engineering field is this that Word is prevalent over LaTeX?

Even a decade ago, I would have been (pleasantly) surprised to see LaTeX instead of Word in sciences outside of CS, Math, and Physics (including the CS- and Physics- oriented areas of Biology). Good luck getting Word users to switch to it!

I am not a Word guru, but I have used Word for creating documents with small groups of people, and anytime the number of editors has exceeded "one", it's been a huge pain except when there was a single competent person in control of the master document.

I imagine distributed editing can work painlessly if everyone is proficient in Word, but I've never personally seen this.

I think you should sit down with the group and ask them to find one person to own each section of the document, have that person handle any edits at the text and formula level, and then have you do the final assembly of the sections and final formatting (headings, pagination, citation cleanup).

Once a week, as everyone is writing, you should get the in-progress drafts from each section owner, assemble them into a rough draft, and send it back to the section owners for review. They should each read the other sections to check for vocabulary issues, and talk directly with the relevant section owners to work these out.

This would be far more efficient than having you discover these issues three days before the document goes to press, and then having a fire drill. I think that the contributors may also find this a more efficient model (not in terms of your time - they may not care about this - but in terms of the time it takes them to get to a final draft of high quality).

Good luck.
posted by zippy at 12:27 AM on December 29, 2009

I'm in finance and have (had the misfortune to have) worked on collaborative documents with MS Word as the primary tool in the past.

If you don't plan adequately, you will end up with precisely the situation, perhaps worse, than you've described.

We had a particularly well organised admin who was adept at assigning an "owner" to a specific chapter (you could structure you work differently). The owner was the only team member who had an MS Word version of that chapter, everyone else had PDFs, effectively giving them read only access. Note that while some team members had no MS Word documents (read only across the entire book), no single team member had an MS Word document for all chapters.

Periodically she'd collect all the MS Word versions out there and reintegrate, creating another baselined version of the master document. According to our planned schedule, this was done once a fortnight.

When someone wanted to work on or modify a particular chapter, if it was already checked out you'd have to confer with the current "owner". If she or he agreed your changes could be directly integrated.

If nobody had the chapter checked out, you'd get the MS Word version and would be noted as the current owner. You'd be responsible for any changes to that chapter while it was under your stewardship. Since each chapter had a distinct version number / history, earlier versions could easily be reintegrated should someone object to changes.

The advantages were only a single person would modify a single chapter at any point in time. The admin was effectively a gatekeeper, letting contention for content be decided by the subject matter experts.

Disadvantages? Legion, but no system is perfect. But it reduced chaos, confusion and largely worked for us.
posted by Mutant at 1:45 AM on December 29, 2009

re: Mutant's example - that is old school version control software implemented by humans instead of computers. Modern vc software aids in integrating multiple edits on a single document, but this only works with plain text.
posted by idiopath at 6:48 AM on December 29, 2009

Of course the first comment says "use LaTeX" because people who use it can't fathom not doing so. Except there is a learning curve, good luck trying to get people outside of fields that use complex equations to switch! I have a PhD in zoology/behavioral bio/evolution/ecology and NOBODY I knew in grad school or beyond used anything other than Word or perhaps WordPerfect. Neuroscience, biology, psychology departments - nobody. If everyone you work with does not already use LaTeX you are not going to be successful getting them to switch.

What I do when collaborating - step 1, make sure everyone knows about and uses Reviewing Mode to make edits. Step 2, make them turn off the option to automatically define styles based on their input. It greatly limits the inevitable style mess you get when sharing (once had a collaborator send me a file with "normal" style defined as 18 point boldface!) Step 3, use the merge and compare feature. Step 4, try to determine a mutually acceptable reference manager software package like EndNote. Give people the basics on how to insert references. Step 5, aside from the basics (bold, italic, super or subscript) leave the formatting to the journal editors! They get paid to do this and are used to doing it with Word documents. (For the people asking you to use LaTeX again - most if not all of the journals in my field ask for a Word doc for submission, some accept plain text or TeX files but many actually accept only Word. Even if you did convert everyone to using a text editor you would probably have to move it back to Word in the end anyway!)
posted by caution live frogs at 7:24 AM on December 29, 2009

OH MY GOD MY LUDDITE BOSS HAS ACTUALLY AGREED TO TRY LATEX.

You have no idea how huge of a breakthrough this is! The man barely knows how to use his cell phone, and I have to tell him to "learn2internet, old dude" almost weekly. :)

So thank you to devilsbrigade and everyone else who recommended that. I looked it up, sent him a few links, and now he seems persuaded that it would be a better tool for our group and has put it on the agenda for our next staff meeting.

From all the advice received so far it seems like LaTeX + some sort of revision control system (whether technological* or human-enforced) + a house style manual would solve many of our problems.

(* I have repeatedly begged IS for SharePoint, but apparently our CIO has already unsuccessfully tried for years to generate interest in it amongst our PIs. And since we're soft money, if the PIs don't want to do something it isn't going to happen because they raise their own salaries and thus no one can make them do anything.)

Responses to a few other questions/suggestions:

"Could you use Google Docs and then export the result to Word later?"

No, our documents have tons and tons of figures, tables, citations, etc. and thus we really need to use the caption, cross-reference, and citation features of Word to keep everything straight. So any writing or collaboration tool we use needs to have those features.

"You might have luck with introducing bibliographic software, but not necessarily."

I think this is another area with the potential for a huge time savings, not just for writing and editing documents but also for their own personal information management of their research sources.

Right now a couple people use Word's crappy reference management feature, and one or two use EndNote. But I'd rather get them all using something web-based and collaborative, like maybe Zotero or RefWorks? I don't know much about these yet, though, nor what would work with LaTeX (or if we would even still need a separate reference management tool if we switch to LaTeX?). I'd appreciate any additional specific advice about these programs.

"They are relying on you because they can. We'd have abused you too if we'd have had someone like you." ... "Just because it's your job to support them, doesn't mean that you shouldn't just task most of these things back to your group"

Unfortunately, I can't. A lot of this stuff is given to me with little direction beyond "make it nice" and "use your own judgment." My boss doesn't want me to bring a bunch of issues back to him or the other authors to fix, he just wants me to make a judgment call and enforce it consistently throughout the document. (I think he would be disturbed if he knew how I usually solve most consistency issues -- the material makes as much sense to me as Jabberwocky, so I just Googlefight term variations and go with the most popular -- but that's what he gets for asking someone with no science or engineering background to copyedit technical reports.)

My time costs 1/3 to 1/2 less than that of my researcher coworkers, so once a document is already as screwed up as much as the one we're working on now it really is a sensible business decision for them to just give it to me to sort out rather than have four expensive scientists and engineers spend time resolving these issues. But in the long run, it would be far more sensible for them to invest an hour or two up front agreeing on style and format issues BEFORE they all start writing, instead of wasting their time doing the equivalent of digging holes just so I can come along behind them and fill them back in. So I'm trying to come up with a system that will get them to do that.

"Just because they see that the work flow is poor, does not mean in any way that they will accept process or technological changes you suggest."

I'm optimistic because I've successfully gotten them to change their work processes in other areas -- for example, giving me their grant proposal documents at least 24 hours in advance instead of only 30 minutes before the deadline. They respond well to bribes of home-baked cookies. Plus, right now everyone is so traumatized by the soul-crushing horror of this messy 150+ page (and growing) project report that I think they're motivated to prevent such an abomination from being created again, enough that they'd be willing to move outside their comfort zones and learn new tools and processes.

"Things like the glossary were divided up and compiled to the 'person in charge' who flagged back consistency issues and retasked everybody to comply to the standard (which was decided by the group at our regular meeting)."

Thanks... anyone have any more ideas/advice on creating a house style manual for the things that LaTeX doesn't cover? I can't *make* them change their writing habits, but I think I could persuade them that it would be in their interest to do so. (After a year of evangelizing the serial comma, I've gained 2 partial converts.)

Thank you to everyone who has given advice so far. Even if I didn't directly respond to your comments here, I'm definitely going to check out all the recommended tools (Word Annoyances, etc.) and consider all the various process improvement suggestions. And I would really appreciate any more advice that people have to share. I knew this had to be a solved problem, I just didn't know where to start looking for the solutions.
posted by Jacqueline at 2:49 PM on December 29, 2009

Any real collaborative editing requires either latex plus version control, or else mediawiki.. or maybe google docs or google wave if users need wysiwyg editors. Word cannot be used for collaborative editing, period. Luckily, your academics are asking merely for basic copyediting, not collaborative editing.

I think the terminology adjustments are best handled using simple search & replace, but you'll need to learn some tricks to avoid unintentionally creating other problems. I think you need to first learn about Word's styles, which allow you to abstract document layout and formatting from content.
posted by jeffburdges at 2:59 PM on December 29, 2009

Oh, and one more thing:

"And about the 'ohgodpleasemakethepainstop': you can not 'fix them'."

But I'm an ENTJ -- I'm *compelled* to fix them. :)

"You can change how you respond to their requests. And it is your choice about whether or not you choose to 'lose your mind' because they refuse to see the value in educational modules on picky esoteric word functions like paragraph formatting."

I like to exaggerate for humorous effect and thus the tone of my post probably came across as more upset and desperate than I actually am. Also, I wrote it while in the middle of a three-hour process of converting a mish-mash of 100+ mostly plain text captions and cross-references (with different numbering schemes!) to one consistent, dynamic system. It's good that I worked on that part in the evening after the responsible parties had left or I might have stabbed someone in the neck with a pen. :)

Overall I love my job and adore my coworkers, but such horrible inefficiencies do drive me a bit "crazy." Our process is so very, very retarded that I know we'd all be so much happier and more productive if we just invested a little time and thought into fixing it.
posted by Jacqueline at 3:01 PM on December 29, 2009

Monkey tex attempts to provide online collaborative editing without the fancy confusing "daggy" features of full version control systems, although I seriously doubt it's suitable for 100 page projects. I've seen various wysiwyg latex editors too, but I've always written latex code directly myself, and never needed one.

Zotero has the ability to export bibtex bibliography databases for latex, but I've never used it since mathsci.net exports bibtex directly. I'm sure there are endnote-bibtex converters too, but again I've never needed one.

If they're serious about giving latex a spin, I'd recommend letting the younger more mathematically and technologically inclined people try it first, well latex has numerous subtleties.
posted by jeffburdges at 7:24 PM on December 29, 2009

Congrats! Yes of course using cheap labour is behind all of that, and yes it is your job, and ENTJ or not, I only wanted to highlight that Saving Them may not be technological in nature. And maybe your melodramatic humour and my ironic tone were ships passing in the night.
posted by kch at 8:09 PM on December 29, 2009

OH MY GOD MY LUDDITE BOSS HAS ACTUALLY AGREED TO TRY LATEX.

Post needs the suddenoutbreakofcommonsense tag.

Here's the good news: You will now have less work to do.

Here's the bad news: You will now have more work to do!

Migrating everybody to LaTeX, while an excellent idea that is bound to pay off in the long term, will need quite a bit of handholding. The thing you need to drill into your coworkers' heads is that, until they get the hang of things, they have to give up any attempt at formatting the visual aspects of their document, and avoid deviating from good LaTeX practice without good reason.

This is not because LaTeX is inflexible (although it is to some extent), it is because people with a WYSIWYG background expect to be able to do all sorts of dumb things, and will spend inordinate amounts of time trying to coerce the system into behaving in ways it wasn't designed to --- sort of like trying to use Word for collaborative editing of technical documents, but in reverse. Floats in particular come to mind, people get horribly upset at the fact they cannot micromanage figure placement without understanding how the float system works and what it's supposed to do.

Fortunately for you, there's tons of documentation out there, this is a definite must-read for newcomers. A big priority for moving to a LaTeX system is deciding what functionality you need ( long tables? rotated pages? chapter bibliographies?) and selecting a good set of compatible packages for providing it.
posted by Dr Dracator at 1:12 AM on December 30, 2009