So many complex projects. So much information. How do I process it all?
September 26, 2014 1:04 PM   Subscribe

I'm being crushed under the weight of all the information I need to process. Most of it revolves around my job. I've thought about it ad nauseum, and I sincerely don't think there's the option to just back off from certain responsibilities to lighten my load. Delegation to others is also not an option, at least at the moment. I just have to be able to absorb all the information coming into me, sort it, process it in my mind, generate new thoughts based on it, and then ACT on it. I need some way of keeping this all organized, and it should be malleable and adaptable since things change all the time around here.

Me? Engineer. Executive. Small engineering company with huge ambitions. Current staff not up to necessary level where they can be trusted with delegated tasks, so everything comes back to me. Wishy-washy, impulsive boss/dad. I've gone into more detail about this in my other posts.

I just have so many pieces of info to keep track of. A single email might tell me a few different things in the message and come with several attachments. From the message and the attachments, I'll generate even more info myself: notes, ideas, game plans and tasks, questions, and maybe instructions to to others (usually, "STOP! Wait on doing that.") One of the tasks I'm often asked to do or that I generate for myself is to check the design of a piece of equipment or a system. That involves gathering even more information about what others have already done, doing my own research about how it should be done to see if they were on track, and checking to see what dates have been thrown around by others for when this is supposed to be done.

So I need to keep track of all of it to have the full picture, so I can give good advice, stop people from going too far down the wrong paths, manage reasonable and unreasonable customer expectations, and then do a whole lot of complicated engineering myself on top of that since others aren't going to be able to produce what's needed in many cases. We are actively searching for and interviewing to try to upgrade the age, experience, and capability of our engineering staff to try to solve some of these issues, but, for now, you might as well fold all that interviewing into the picture as yet another thing I need to keep track of.

Because of the sheer scope and complexity of everything, I think this situation falls beyond what a lot of the common time-, project-, and information-management systems out there can handle based on what I've read about them. I could be wrong, but I'm currently stumped, and I figured you guys would be my best bet at getting some fresh thoughts on the matter. I need a system where I can dump all the information from a small, complex company, sort it and draw the connections between it, and make changes easily when, say, the customer decides to suddenly come by in a week and expects to see something finished that was further down on our priority list. I want to be able to break everything down into granular tasks small enough to actually accomplish in a couple of hours, and it would be good to be able to mark these things as done when they are indeed done. At the same time, I'd like the research notes and ideas that I've come up with to aid me to be close at hand as well.

I'm not ruling any ideas out, but I will tell you what I've dabbled with before. I've used Gantt-type systems like MS Project and Smartsheet. I haven't gotten super proficient at them, but the best aspect is being able to easily show dependencies between tasks, and the worst aspect is being forced to assign times for everything since I often don't have a good idea how long something will take, or I just want to think about the sequence and breakdown of tasks without getting hung up on the time allotments.

I've done mindmapping, which is great for breaking down big tasks into smaller ones, but is kind of cumbersome when it comes to drawing connections between tasks on different branches. Also, the mind map gets crowded easily. I sort of like the idea of a big chart with lots of central ideas in circles (bigger ones could be more pressing or bigger in scope than smaller ones), with branches off of them and also connections to other circles. Like a bunch of interconnected mindmaps. This is a people map I found and too complicated to read, but it kind of illustrates what I vaguely have in mind. I have no idea whether this is a good idea or not; I just find mindmaps to have potential but significant limitations when it comes to illustrating the big picture. I've also never seen any software or website that could make diagrams like that.

Finally, as a simple, just-get-moving approach, I've often just made Word documents where I'll just write down everything relevant to the thing I'm trying to work on, usually using a basic outline structure. It's a simple, raw way to aggregate all the information I've learned, all the ideas I've had, and all the attempts (failed or succeeded) that I've made to do something. However, these documents get looooong and since they're linear and are designed to just get everything down on paper quickly, they tend to do poorly at showing priorities or interconnections between tasks and information. I literally have to write something like "Look at I.C.6.g" next to items to connect them. Making changes is alright, because I can draw a strike through old information, but it still might hard to find in the dozens-to-hundreds of pages that get generated.

So far, I just haven't hit on something that continues to work for me, and that leaves me getting behind on everything (not that I have a reasonable workload, but I could make better progress if I wasn't trying to grab the necessary information I need out of the cloud of my memory or my overflowing inbox.) I'm up for any suggestions, and I'm not ruling out Gantt charts, mindmaps, or simple info logs; I may just be using those techniques in the wrong way. Thank you for your thoughts.
posted by KinoAndHermes to Work & Money (12 answers total) 25 users marked this as a favorite
Do you need it to be completely out-of-the-box? If you badly need this degree of leverage, you may not be able to find something out-of-the-box that works well enough.

Are the connections and requirement paths to remind you to follow them or should they perform some function as well?

I understand the reactive items must be tied together, but are the proactive goals/plans/ideas necessary to the project or can they be put into a separate system to be re-integrated once they are fleshed out and you've committed to accomplishing them?
posted by michaelh at 1:29 PM on September 26, 2014

Perhaps you've seen these already, but I'm in a similar position in my work and find David Seah's task and time management tools very helpful. What I like about them is that they're adaptable. I'm particularly fond of the task order up, which I've adapted for use for tasks that appear for each project, which I then break down when I have a moment.

I've created my own system using several of these tools, and what I find works for me is a) spend 5-15 minutes at the beginning, before lunch, and the end of the day simply looking over everything and "tidying" up tasks and processes; and b) block out blocks of time wherein no calls, no e-mail, and no meetings are allowed, and allow me to simply focus on 1 task at hand. That task may be organizing a project or actually working on it. YMMV, but interruptions are the biggest obstacle to understanding the complex flow of information I have to process.

It takes some time to get colleagues in the "habit" but I find that, especially with subordinates, knowing that I have "office hours" much like a professor gives them the gift of focus for their time, too. I maintain this with clients as well, although as diplomatically as possible (very few blocks of time are more than an hour long).

I also keep a journal, and spend a few moments here and there looking over my journal and moving pieces of it to various projects. The journal is more for keeping track of ideas and connections than for actual tasks or planning.
posted by barchan at 1:59 PM on September 26, 2014 [3 favorites]

I'm in an organization where some of us use Trello, which is a collaborative "personal kanban" system. The general idea is that you have a board (or boards), which has lists; these lists often represent phases of completion but can be whatever you want; each list has a card that represents a specific task, at whatever level of granularity you want. Cards can have a title, description, checklist (for sub-tasks), people assigned, due date, etc—they're free-form. They don't have a tagging system per se, but you can put whatever text you want in the description and filter based on that.

Because it's collaborative, I can create a card and tag other people with it if it is their action item; I can still look at where they are and bug them if they're not done. We typical leave comments as we work on things so that if one of us gets hit by a bus, our progress has been documented. We use "parking lot/to-do/doing/done" as our lists, but I'm thinking of creating a different board using months as lists, because we have a lot of recurring stuff that always happens around the same time. Once we got the board populated, we would just make a copy of the whole board for each year and check stuff off as we go.

Like any system, it only works to the extent you stick with it, and like any system, it's easy to put off using it when you're busy.
posted by adamrice at 2:05 PM on September 26, 2014 [1 favorite]

You need to try some dedicated project management groupware and see what you like. Try Wrike, Basecamp, or AtTask for instance.
posted by shivohum at 2:12 PM on September 26, 2014

Best answer: I use a personal wiki, specifically VoodooPad, rather than mind mapping, to keep track of complex, changing, ill-structured information. Deprecated stuff gets shoved into a link at the bottom of a page for informal versioning. Top level constantly evolves. Fast capture and effortless structure, after you get used to it. I cross-reference like mad and I use the search features to review and aggregate and not repeat myself.

I use google spreadsheets, one tab per complex project as "project logs". After I do something, or if I'm going to do something, l log it on that tab. Each complex project becomes a semi-linear story, stretching verrrry slightly into the future.

I use index cards for the insane bleeding edge, sort of like GTD but more a blend of stuff/actions/projects.
posted by zeek321 at 4:44 PM on September 26, 2014 [2 favorites]

Best answer: Me: engineer, executive managing product development team of > 100 people, globally distributed, been doing this kind of management for > 20 years.

A few thoughts here:
1. You might try changing your delegation approach. One of the things I find is the busier the person, the more they need to process information in a consistent format. So, if you need to delegate something to an employee, you might structure the request in a way that makes sense for your process. E.g. a standard set of urgency flags (routine, urgent, DOFIRST) with a standard format, time, and method for reporting progress. You might need to experiment a bit to find the best approach, but you're the boss :).
2. The other thing is to look at what kind of tool works best for you. I'm a very visual person. I have tried keeping task lists and the like on the computer, but after several experiments over the years with different tools, I find a notebook (paper, pen) works best for me. I keep all my email in one big unsorted pile on my computer and rely on memory and searching to find what I need. Important action items get written down in my notebook with visual cues (-> for action items, *-> for important action items, and *->DO THIS NOW for stuff that really has to get done right away and that I've forgotten). I also have found over the years that when I take on a new job, role, project, I have to spend more time getting or staying organized, including, at times, setting aside time weekly to scan my inbox and my notebooks. I moved to my current company 3.5 years ago, and for about the first 6 months, most weekends I spent 4 hours in the office doing this before I built the context in my brain.
3. The other thing I do when I step into a role where I am completely overwhelmed is to try to make tiny improvements to start carving out time so that I can get/stay more organized. Tiny because initially that's all I have time for. These improvements might be for me individually or for my team. I don't spend a whole lot of time figuring out what is the best improvement to make (because who has time for that!) -- I just let intuition guide me here. My notebooking method is one of those improvements.

Anyway, good luck to you -- sounds like a tough situation.
posted by elmay at 4:52 PM on September 26, 2014 [3 favorites]

I am a programmer. In an environment where I was serving in multiple roles (team lead, database designer, product liaison, direct client communication, and code monkey), I used a combination of low-tech (a notebook from which nothing was ever erased, that I always took with me to every meeting) and high-tech (a personal wiki and org-mode in emacs). The key really is sorting of data... there has to be a "top level" with a ton of churn in order to capture the horrible hour-to-hour requirements, but at some point you have to filter it into "knowledge" or "worthless." I find it helpful to have this in a text editor because it's so easy to rearrange everything.
posted by sonic meat machine at 4:59 PM on September 26, 2014 [2 favorites]

Read Getting Things Done by David Allen, which is about task management for executives. Also look into Evernote or OneNote for keeping all the info and linking between it easily (you can create hyperlinks within the notebook to a page or even a specific paragraph somewhere else).
posted by the agents of KAOS at 5:01 PM on September 26, 2014

Response by poster: Thanks for all the great suggestions, guys. Seriously, it's all been helpful, and I appreciate the empathy as well.

I doesn't have to be one monolithic system, because I highly doubt that exists. It could be a combination of low-tech and high-tech options like other people have talked about. If we're talking about computer programs or websites, it would nice if the connections between tasks actually took me to those parts of the projects when I clicked on them. Even I had to have a whole folder of charts to describe a job, as long as I could easily follow one to the other, that'd be nice.

Thanks for the website. I can already see a lot of useful stuff in a cursory look over it. I also really like the idea of "office hours." A gigantic problem is that I'm basically "on call" for anything at any time, whether it be by knocks on my door to talk, instant messages, emails with the immediate expectation of response, or many meetings. We've implemented a company-wide calendar system recently (TeamUp), and I think I'm going to schedule "office hours" on it, while continuing to explain to others to not schedule me for other commitments. I definitely need to demand that my time be more respected, although, the fun part will be demanding that from my dad, who is the worst offender. Excellent advice.

I was recommended Trello back when I asked my first question on here, and I definitely like the card system. It doesn't cover everything, but I think it could be a component of the overall system I'm looking for. My secretary also uses it, and I'll look at it again with her using some of your collaboration ideas. We've already agreed we should use it more to keep on the same page with tasks she's working on, and since she's more important to me than the other engineers, I should definitely try to enhance our collaborative efficiency.

I've had both Wrike and Basecamp recommended before, especially Wrike. I've looked into both, but haven't dug deep enough to really evaluate either properly. I hadn't heard of AtTask before, so I'll take a look at that one as well.

The personal Wiki idea is the kind of thing I was hoping I'd find in here: something I hadn't in any way thought of before. Honestly, I'm a little excited about it now, because it fits the type of all-encompassing, interlinked information model I'm looking for. It's a much more robust system than my Word documents, which are the best system I've devised so far...when I use them. I'll be looking at VoodooPad tomorrow for sure. I'm also glad to hear that others use stuff like index cards to keep track of immediate tasks. I use sticky notes on my desk, when I can keep it clear. Seems stone aged, but it has been effective for what it does when I use it.

Your post is just full of good information, and I really appreciate it. I'd just recently started sorting my emails, but it quickly became too complex, so I'm glad to see that someone who knows how to manage just keeps everything in one inbox. I was already thinking about going back to that, so I could search for and filter messages more easily. I also definitely feel more at home writing things down with pen and paper, which I think has led to some of my failures at the computerized systems. I think there's a place for both, and I think when I really need to get ideas down in a way I can understand them best, that's the way to go for me. It's also nice to hear someone with experience actually validate the kaizen, small-improvements approach. I'm partially through the book, but I always like to hear if different approaches I read about actually have worked out for people in the real world. I'll be coming back to your post frequently, I think.

sonic meat machine:
Definitely, I'm excited to see the personal wiki idea seconded in your post. I agree that the sorting is the important part. It's particularly hard in my company to just get ALL the raw data I need, so I'm often just in that mode, but the sorting is usually the bottleneck once I am ready to proceed. I think you're right about being able to filter stuff into "knowledge" and "worthless." I can probably reduce the amount of info I need to sort by a huge percentage by taking that approach, although I'll definitely have to figure out a polite way of explaining to a lot of people why their work got sorted into "worthless." My goal always has been to get as much right the first time, but this is definitely the first bloody step. I think I'll stay away from emacs, though. I switched to Chem E from EE after a year in school, but I still clearly remember the late nights programming Java in emacs my freshman year. :)

the agents of KAOS:
I own GTD, and, as with so many useful books on time management, I ironically get stuck on finishing them because of time constraints. I know it's the bible of this sort of thing, though, so I'll definitely get back to it. And Evernote seems to be integrated into everything nowadays, so I should probably fire up my account for that as well and see how it's improved since I last dabbled in it.

Thank you all very much. I get a little nervous posting these questions, because almost my whole history on here has been me searching for answers to parts of this same puzzle. At the same time, it has helped me to crystallize what I truly need, and I think this thread is the culmination of that. I shall definitely be coming back to it often to read over the suggestions. I appreciate all of your help.
posted by KinoAndHermes at 7:36 PM on September 26, 2014

As a variation of the Wiki idea I suggest giving OneNote a try.

I'm a software developer and great problem solver, but by nature I've never been very organized. Over the last year and a half I've been thrust into a situation similar to what you describe and OneNote has become my pillar of organization.

I use two 'notebooks': one is personal, and the other is shared with the entire team. It's much more free-form than any Wiki system I've used and very easy to do formatting (including tables, to-do lists, etc.) as well as cross-referencing between notebooks, sections, pages, and specific paragraphs and words. The shared notebook is great in that it can be edited by multiple people simultaneously (even the same page) in close to real-time, with changes color coded in the margins with the initials of the person who last made an edit.

You can also paste emails and attachments, draw, record audio and video, etc.
posted by GoldenShackles at 8:33 AM on September 27, 2014

Response by poster: GoldenShackles:
I've seen OneNote mentioned a lot over the years and several times in this thread. Since you've suggested it as an alternative to the wiki idea I like, I really should try it out, as well. As with most Office programs, I often feel like I'm not exploiting their huge functionality to the fullest, though, especially since it's already on all my computers. If anyone has any suggestions of a good book or website or whatever for learning OneNote efficiently and comprehensively, I'd love to hear them. Thanks.

Also, I see Evernote mentioned a lot, so if anyone has any good suggestions on where to learn to use Evernote to its fullest as well, I'd be grateful.
posted by KinoAndHermes at 9:49 AM on September 27, 2014

Evernote and one note pretty much do the same thing. The advantage of Evernote is that it is not a MS product and therefore works on Macs and the like.

Although I don't like the names :) the "For Dummies" or (even worse) "For Idiots" books are usually decent intros. A quick search on Amazon turns up both "Evernote for Dummies" and "One Note For Dummies".
posted by elmay at 2:44 PM on September 27, 2014

« Older Roommate and bed issues   |   How do you wear a winter coat with a blazer? Newer »
This thread is closed to new comments.