Best way to manage huge amounts of data I get via email?
October 27, 2012 8:45 AM   Subscribe

Please help me find an effective workflow to manage the metric ton of digital work-related information I get via email every day. I wish the solution was as simple as Evernote but, of course, it's not.

I'm a freelancer and work from home. One of my biggest client assignments involves working with a distributed team of about 10 other contractors on ongoing projects. Communication occurs almost exclusively over email (don't get me started on how inefficient this is) and thrice-weekly conference calls. I have repeatedly suggested that we use Campfire or some other sort of online option to track discussions, task lists, etc but the group is completely resistant to change, so at this point I have to figure out a way to work within this highly inefficient system.

I receive important information from 2-6 people every time I take on a new task. Sometimes it's the same feedback from multiple people who have no idea others are saying the same thing because, email! Pre-sorting email according to task is virtually impossible because subject lines generally don't reflect which task or sub-task is being referred to. To make matters worse, some emails refer to two or three tasks and subtasks on different projects.

Add to this, the feedback and new tasks that come up during conference calls or the occasional Skype sidebar, and I'm swimming in bits and pieces of data with no effective way to track it all. Add to that, sometimes I can't actually act on a task until someone else gets back to me with their work -- i.e., I can't simply open an email, take action, send a response, and move on.

I've tried opening a threaded conversation on my screen and referring to it as I create my new widget but the signal-to-noise ratio is ridiculous. Besides sifting through the content of each email to find what's relevant to the specific task I'm working on, there's also the hassle of working my way down a thread only to find that an action I took ten minutes earlier was decided against in subsequent emails between team members.

I'm on a Mac. I currently use Gmail and its labeling features, and own what seems like 400 OS X-based task managers but to-do lists aren't what I need. I know what I need to do, I just need access to all this random info so I can do it.

I have Evernote Premium but the biggest problems with stuffing everything into it are:

1. I can't forward emails to a single notebook because they often have details I need to remember for many different tasks.

2. I have to eventually respond to each of these action items via email (which generates more email!). Leaving Evernote and heading to Gmail to track down relevant emails and provide a response is maddening.

Gmail Tasks has proven to be completely ineffective because, although it easy to jump to the email I need to answer when I've completed a task (or need more info to complete a task), there's really no good way to sort and organize the data contained within the email to which I must constantly refer.

tl;dr: I receive hundreds of tidbits of information each week via email and phone that I must track and act upon. How can I sort and organize it all, when almost everything comes to me via random and sometimes redundant email which often covers multiple subjects and therefore can't really be stuck into one label for reference?
posted by _Mona_ to Work & Money (8 answers total) 14 users marked this as a favorite
Have you asked other members of the group how they handled this?
posted by Brandon Blatcher at 8:54 AM on October 27, 2012

@Brandon Blatcher Unfortunately, I coordinate the things all these folks do and am responsible for the final "product," which I why I get way more email than any of them. Further, they don't have to worry about a bunch of details, their tasks are pretty linear, instantly actionable, and don't require anything from anyone else to complete.

Some of the team members do acknowledge we have an email problem but none are technically savvy so they just cope. Put another way, I once shared a Google Doc with the team so they could view a list I made. No sign in or editing of the doc was required, all they had to do was look at it. You'd have thought the world was ending because we "weren't using email." Sigh.
posted by _Mona_ at 9:15 AM on October 27, 2012

I suggest you take another look at Evernote using the "The Secret Weapon" methodology for managing the emails once they get into Evernote. One caveat: if you aren't already familiar with the David Allen Getting Things Done system, it won't initially make sense. If you are familiar with GTD, take a look at the way the Secret Weapon guys suggest you manage your Evernote content using tags (corresponding to the GTD system). I thought I was an Evernote power user until I read through their twenty four page manifesto and set everything up.
posted by kovacs at 9:26 AM on October 27, 2012 [2 favorites]

One feature that might be useful: If you copy and paste data from Gmail into Evernote (rather than forwarding the email to the evernote email address), Evernote sets the Source URL tag to that email in Gmail. So when you need to send a response, you can click on that Source URL, and Gmail will open to that email again.

My advice would be to copy and paste just the relevant bits of the emails into Evernote, then tag them or sort them into different notebooks based on which project they're related to.
posted by JDHarper at 10:04 AM on October 27, 2012 [3 favorites]

I'd make a different notebook for every project z, and different notes for every category. Notes could be things like Things for me to do, Knowledge, Meeting notes related to z, Things x is working on, etc. Try setting it up on something small and see if it works. Also, tags help.

Calendar categories also help with organization in Google calendar.
posted by oceanjesse at 10:29 AM on October 27, 2012

I use Google Docs for this, but I think Evernote will do exactly the same thing. The core of the issue is: email is good for communication, but terrible for note-taking and organizing. So your organizational structure has to exist outside of email and just help you find the emails when you need them. Basically, you need a big document (Google Doc or Evernote) for each project where you record the critical details so you can always find them without resorting to digging through your Inbox.

Here's my "not in email but works well with email" structure. Whenever I get a project, I create a folder in Google Docs for it (sometimes it's a sub-folder of a larger category, but that's up to you) and a "All notes and details" document. I imagine you can do something similar in Evernote. I add a Table of Contents to the document so I'll see any Headers I create. The folder in Google Docs allows me to save other project documents (e.g., spreadsheets) nearby, but if I'm looking for my notes I always know to go to the "All notes and details" document.

When I get an email or have any communication (e.g., phone call) regarding that task, I review it for critical information and react accordingly:

1) if it contains a tiny bit of critical information, I add that tidbit to the Google Doc via copy/paste or just typing it in, and note that it came from an email from X person with Y title on Z date. This takes about 60 seconds. Usually, I have a "assorted notes" or "to do / to look into" header so I can quickly find this section in my aforementioned Table of Contents and find all the little bitty details at once.

2) If the email contains a huge amount of critical information, such that it would be cumbersome to re-type all of it, I copy and paste the entire email into the Google Doc, including the email header (who sent it, when, and subject). I type a short description of the email ("12/10/11 email from Terry with names of new clients") and make that the header. Then I can forget this email exists yet always find this in the aforementioned Table of Contents.

3) If I send an email to someone with questions or followup, I record that in the "Notes / To Do" section so I can follow up later. When they reply or call me with a response, I either copy that email into the document, or just make a note that they replied on X date and what their answer is so I don't have to go digging for it.

4) On phone calls, I open one or more documents ahead of time and take notes, and create a header that says "Phone meeting about Project X 10/27/12." Again, now my notes are all in one place.

That's about it. Really, less than 10% of my emails contain information important enough to go in one of these documents. Aggressively labeling and sorting every email then searching through email for all of that detail would be drinking from the firehose: I'd be overloaded with not-critical email chains. Instead I grab the truly important details and put them in a document. Everything else remains stored in email so I can get to it, but if I need to know details of a project I can go to my document and 95% of the time the details are in there, or I'll know what day I received the email with those details so I can look it up.

This does require some manual work. It's not automatically forwarding emails for each project, it's manually copying and organizing them. But the time and frustration it will save you might be worth it, if you find you're currently drowning. 5 minutes spend typing a note or copying an email will save you 15 minutes of looking. And just imaginge if you needed details from 5 different emails - they would all be in the same Google Doc or Evernote note so you can see them all at once and even copy/paste the key points into a single list.

Moreover, how much manual work it requires depends on you: if you aggressively say "this isn't critical," you'll end up doing less manual work b/c you don't have to document it. But if you are extremely detail-oriented and have to save lots of stuff, then you'll end up doing more manual work - but that might be okay if it means you know where all of your critical information is documented and you don't have to go digging through email to find it.
posted by Tehhund at 1:45 PM on October 27, 2012 [1 favorite]

Would it be possible to set up something like RT plus the command processing tool? Then the team would have a purely SMTP based interface with your project management system.
posted by pwnguin at 2:21 PM on October 27, 2012

I am an information junkie who deal with quite a bit of information load (but not necessarily by email; in fact I treat my inbox as a sacred place and tightly control what goes there). Find as many "buckets" as you need to store and retrieve all this information but don't let it get out of control.

Since you already use Evernote, I hope you are using their Chrome extension. You can select relevant bits of your email (or any part of any webpage) and hit the button (or a keyboard shortcut) and it'll pop up a window where you can tag the information. It's smart about suggesting the right notebook to put stuff into based on your previous behavior. If I try to evernote a NYT article, it goes into my notebook called longform. If I try to grab a code snippet from StackOverflow, it goes into my code bits notebook. This also allows evernote to preserve the correct context (instead of cutting and pasting).

Other useful Evernote tools:
Popclip - The beauty of this tool is that you can enable the Evernote extension and then send any selection from anywhere on your mac to your Evernote.

If you use Hazel, you can also automate a bunch of Evernote related tasks. I have a smart folder that any document I drag in there will get imported into Evernote and promptly deleted. In your case, save gmail attachments to this folder and bam, it's now in your Evernote.

I don't also rely exclusively on Evernote for note taking. I find it a bit slow for text-only stuff and I prefer to use it as a bucket to capture odds and ends that I don't want cluttering my file system. For plain text notes, I use Notational Velocity as the client. It opens in 1 second, you can type up notes (during a call) and just close it. Done. No saving nothing. The same app opens on your phone too so you can retrieve notes on the go (and sync with simple note, dropbox whatever).


Pinboard is simply brilliant. For $25/yr, you get archivial bookmarking. This means that any page you bookmark there will also get crawled and cached for you. Think of it as your own private Google cache. I have the Pinboard extension for chrome, and also a keyboard shortcut for it. I send 40-80 links there daily. Takes 2 seconds to tag and I can find all the content later even if the page is deleted within a few hours.
Pinboard also works brilliantly with ios apps like Tweetbot.

For Skype, get Call Recorder so you have every single call saved. You can split the tracks with the tool too, so when someone emails you two months later about the last conversation, you can quickly pull it up for a reference.

For Gmail, have you tried Boomerang? Seems ideal for you since you seem to work in email exclusively, boomerang can work alongside so you can task things without generating more email.
posted by special-k at 3:13 PM on October 27, 2012 [3 favorites]

« Older I want to love sex again!   |   Functioning with bad jet-lag? Newer »
This thread is closed to new comments.