Workflow using SharePoint 2010 or Python 3.x or ?

January 22, 2018 12:34 PM

I have a big, dumb, needlessly complicated, manual process I would like to automate (insofar as possible) in SharePoint/InfoPath 2010 or possibly Python 3.x. Can you help me think through how to do it?

I work in Scientific Affairs at a big company. Somebody in Marketing heard that I built R&D's ticketing system for work requests (in InfoPath and SharePoint 2010), and they asked for help with this abomination:

1. Ideas are coming to us as a manually-populated PowerPoint file (essentially a stock form)
2. Then we currently have a crude survey-monkey form where a committee scores the ideas based on about 20 questions.
3. We are then manually transferring to survey monkey data into excel to make calculations
4. Then, we are transferring that to another file to display the results in a clean, concise shareable manner.
5. Next, a Scientific Affairs person would review the ideas and then align ingredients in your database to ideas with a corresponding ingredient need.

Here's what I proposed to them as a first pass.

1. Person-with-Idea fills out fields on an InfoPath form (a ticket), person-with-idea clicks Submit, form routes to committee members.
2. Each committee member individually scores the idea according to your 20 questions on the same ticket
3. All ticket data can be automatically exported to Excel by clicking one button OR the calculations could be done right in SharePoint and sent to all committee members
4. I could write you an Excel macro to do that, OR maybe the SharePoint output would meet this need. How pretty does it need to be?
5. The ticket could then route to Scientific Affairs automatically OR back to Marketing for more decision-making, then Marketing clicks a button to send the ticket to Scientific Affairs.

I think the InfoPath/SharePoint solution would work, EXCEPT I'm not sure how to automate a pretty deliverable -- could InfoPath/SharePoint fields be used to populate a series of Word forms?

Alternatively, I know a tiny bit of Python, enough to know that I could do the whole thing in Python! Which would be really fun! But like I said -- it's only the tiny bit of Python I know, though I do have books to help. The advantage I see in using Python is that I could make beautiful reports out of it, formatted however they want, with pictures they supplied, etc. Maybe the Python app would use Box (which we have), rather than SharePoint?

What do you think? The italicized part of my question is the ask -- how would you do it? Thanks for any ideas!

posted by pH Indicating Socks to Computers & Internet (11 answers total)

For the love of god, avoid sharepoint programming at any cost. Sacrifice firstborn if you have to
posted by hleehowon at 1:02 PM on January 22, 2018 [6 favorites]

Seconding hleehowon on the despising of all things SharePoint. Burn it with fire.

Consider Google Forms, which dumps the data straight into Google Sheets, an excel-like web application. And the powerpoint deck could go straight into Drive via the form upload.

The ease of use benefits on those 3 web applications in tandem will make it too easy for folks to submit ideas, then you'll have a different class of problem to deal with in a few months.

Best luck!
posted by enfa at 1:14 PM on January 22, 2018

This would take about an afternoon for an experienced python dev using Django or Flask (or Ruby dev using Rails,etc).
If you don't have experience developing web apps, there will be a mild learning curve.
The pretty part could be handled by html, css and any of many JS-based graphing packages, like plotly, graph.js, etc. You can also easily export to excel from python.
This is all pretty standard, but if you've never done it before, you'll have plenty to learn.
The language itself is the easy part, learning how to use it to do stuff is where 99% of the effort resides.
posted by signal at 1:15 PM on January 22, 2018

As much as I deeply share the same sharepoint sentiment installing Django or other CMS may be more effort in some cultures than a hybrid solution. There seem to be some python interfaces to sharepoint. Do what works easy in one and extract and reformat in python.
posted by sammyo at 1:42 PM on January 22, 2018 [2 favorites]

As an experienced Python dev, I'd like to push back on the idea that an experienced python dev could do this in 'an afternoon'. I'd want to do a lot more requirements gathering on all of the moving pieces and parts and that all by itself would take an afternoon.

I don't know anything at all about SharePoint, so I wouldn't be comfortable making a recommendation about the feasibility of working with that vs. another solution.

Sorry I'm not more helpful here; I just want you to feel like, when it winds up taking a lot longer than an afternoon, that you aren't necessarily doing something wrong.
posted by Kwine at 2:01 PM on January 22, 2018 [3 favorites]

Response by poster: Jeez, you guys don't much like SharePoint! SharePoint has been pretty good to me; it's what the dept. uses, I built pretty GUIs for it, and I know how to make it do what I want, mostly. Python -- I'm still hooked on phonics. But eager to learn! Learn what, you guys???

installing Django or other CMS may be more effort in some cultures than a hybrid solution.

QFT - I'm not IT, I'm a lowly scientist, also SharePoint site owner. It's highly unlikely the bastards in capes would let me implement something new for a group -- or do anything in, say Google docs, etc., though I do have admin rights on my own work computer and a reasonable level of access on SharePoint.

I'd want to do a lot more requirements gathering on all of the moving pieces and parts and that all by itself would take an afternoon.

Also QFT - All I know about what they want/have now is above -- which isn't nearly enough to build it in SharePoint where I am comfortable, let alone Python.

But tell me this: Can you suggest a flowchart-style skeleton process, that would at least give me an inkling of what I need to learn?
posted by pH Indicating Socks at 2:29 PM on January 22, 2018

most commercial organizations who ended up using sharepoint found it ok to mediocre

most commercial organizations i know who tried to customize sharepoint found it an existential threat to their business by it being so fucking shit. or else their programmers all started drinking problems or something like that

if you need to deal with excel files in python, use pyexcel, which conveniently has a lotta stuff for a lotta customizations with excel.

but i agree with whoever suggested goog forms. if you need programmatic adjustments on that one, export it to excel sheets and work on the excel sheets themselves with pyexcel

high-up vp's from microsoft i knew hated sharepoint. the people who worked on sharepoint hated sharepoint, including management and stuff. burn it with fire
posted by hleehowon at 2:38 PM on January 22, 2018

Can you suggest a flowchart-style skeleton process, that would at least give me an inkling of what I need to learn?

I would start by identifying the *necessary* inputs and outputs. The data appears to go through transformations through (at least) powerpoint, survey monkey, excel, "another file" for display. I'd be trying to figure out whether anyone is secretly or not-so-secretly depending on having, e.g., the survey monkey surveys sitting around, for another process that is independent of what you know about. Right now it's unclear what part of that process exists for no reason or because it's what someone knew how to do on the one hand, and what part exists because it's what someone wants/needs on the other hand.

To help accomplish this, I'd want to identify people with a stake in this process beyond "Someone in marketing" for sure. Maybe "Someone in marketing" is on their way out and the rest of that organization is very happy with the process. I think the best way to proceed depends a lot on the culture in your organization.
posted by Kwine at 4:10 PM on January 22, 2018

Mixing Microsoft Office docs and python is pretty much terrible. Every new version leads to more and more frustration. It’s something we dedicate pretty much a full time programmer to. Yuck.

Share point isn’t the worst, I’d go down that path.
posted by advicepig at 5:03 PM on January 22, 2018 [1 favorite]

Just to pile on, I knew a big consulting firm whose business model was installing Sharepoint intranets for their customers, and they didn't use their own Sharepoint intranet because it was too complicated and nobody knew how to use it.
posted by signal at 6:32 PM on January 22, 2018

I hate sharepoint because I have spent a good chunk of my career reverse-engineering and maintaining a connection to a particular terrible , customized, hacked-together, sharepoint system to get data back out of it. HOWEVER. You need to be very careful creating a bespoke system in python or rails or whatever, unless you want your job to become building and maintaining that system, becasue it doesn't sound like you have any in-house development staff. If you can do everything in the standard sharepoint toolset, without customization, I would actually stick with that.

Now, if you want to transition from scientist to web developer, this also sounds like like a great opportunity, because it sounds like your organization has plenty of problems it needs to solve! Just be sure that that's an opportunity that you want to jump on, because otherwise you might be stuck building internal business process apps when you really want to be solving problems at the bench.
posted by rockindata at 7:35 PM on January 22, 2018 [2 favorites]