Measuring influence: how do political lobbyists or similar groups measure their success?

June 23, 2008 9:37 AM

Measuring influence: how do political lobbyists or similar groups measure their success?

The remit of a group I work with in our very large corporation is to represent the needs of the sales and support teams to the engineering and product teams. They don't have any power over eng and product as such, but their voices are listened to and their requirements taken into account. Essentially, their job is influencing people to take sales and support into account when making product decisions.

This leads to the tricky problem of how to measure their success, which I'm trying to find a good answer for. I haven't been able to turn up anything relevant on the Google, possibly because I'm really not sure what I'm searching for. Also, it occurred to me that political lobbyists face similar issues (or at least they do in my rather basic understanding of political lobbying), and there must be other professions I'm not aware of that have the same issue.

Do any of you charming MeFites have any insight into this kind of issue, experience with it, or resources you could point me to? Many thanks as ever.

posted by StephenF to grab bag (5 comments total)

I can't speak directly to the metrics used by political lobbyists, but at its core, it's probably just a simple matter of tabulation -- that is, calculating the percentage of issues on which the elected official votes in accordance with the request of the lobbyist. Directly applying this to your example, just tabulate the issues that you bring up to the product teams, and then calculate the percentage that are addressed. You might need to tweak it bit to account for issues being partially addressed, but that should be pretty easy.

I think you're overthinking this, though -- a simple survey (using for example, a five-point Likert scale from 1-strongly disagree to 5-strongly agree) gauging the extent to which sales/support staff feel that their interests are represented should suffice, then look at the changes in their attitudes over time.
posted by Doofus Magoo at 10:00 AM on June 23, 2008

Not really precisely in your topic range, but policy researchers trying to measure their success in the development sector often use the International Development Research Centre's "outcome mapping" tool. Check out the website too. These guys look at measuring the impact of agricultural research and have done work to investigate how to quantify success. Academics use bibliometric measurements (citations and publications), but it doesn't sound like that's your ticket. Policy research, rather than lobbying, but the desired outcomes are often similar, and you might get something you can use.

It's a pretty difficult thing to measure success. If you have access to online journals this article goes into some of the difficulties: There's a bunch of problems doing any sort of complete analysis: the impact of lobbying on policy sometimes takes a long time, it can never be attributable to one actor, and there are outside forces which can make it happen or not. Check out this and this for different takes on the godawful complexity of it all. It's perfectly legitimate to run away screaming when you realise you can't do a good impact assessment - some organisations focus on simple output measurements, and don't stress too much about impact.
posted by YouRebelScum at 10:06 AM on June 23, 2008 [2 favorites]

Doofus Magoo is at least basically right with regard to how political lobbyists measure success. I've worked on both sides of the lobby - for lobbyists and as a legislative staffer constantly lobbied - so I feel like I have a pretty good idea of how they measure.

Essentially, you measure success based on how realistic your goals are. It seems simple, but you have to first establish what your goals are. If you are lobbying on behalf of Planned Parenthood in the conservative south, your goal will likely be simply to defeat bills restricting abortion. If you are lobbying for the same organization in Mass., you will likely be more proactive, perhaps trying to get more money for reproductive services. If you're a big business, your goals are based on your business needs - you probably will care about all tax issues, and if, say, you're trying to build a new plant, you will be working to get the necessary permission. If you're a coal power company or some other polluter, you'll also be watching to kill any environmental legislation.

Once you know your goals, which can exist anywhere, and even off, a continuum that runs from killing bills that hurt you to passing bills that help you, you can fairly easily measure your success on those specific goals. But, of course, there is much more to lobbying than just the passage or non-passage of bills. For many, many organizations that don't have the political capital that a major corporation or trade association has, the most important part of lobbying is often to just get people aware of your issue. I think people outside of the political process forget this, because it's not how you think of lobbying or politics usually. The fact is, though, that 99% of people and 99% of legislators don't really care either way about the goals of most lobby groups.

A few years ago in TX, for example, there was a big fight over how to tax chewing tobacco - do you tax it by weight or do you tax it unit? For the vast majority of people, it's a pointless debate - I think, actually, that the change would have even been revenue neutral. But it matters greatly to the players involved - the big companies - Skoal, etc. - I believe wanted it taxed by unit, whereas the smaller producers wanted it taxed by weight. The change essentially would shift the tax burden from one group to the other in a way that affected no one else.

In cases like these - which, I feel, are the majority of cases - the whole point of lobbying is to convince legislators and their staffers to care. So, a huge expenditure of lobby resources is not spent simply on asking for a yes on this bill or a no on this other bill, but rather it is spent on just making people understand your cause - convincing them of its essential logic or fairness. This is a hard road to tread, and in measuring your success there is no possibility of developing any accurate or quantitative metric. It boils down to how many people did you meet? Did they like you? More importantly, did they like you more than they liked the other guy?

I feel like I've rambled off a bit, but the point is that in political lobbying there often isn't any useful metric of success - at least not for most lobbyists I know. They all keep track of bills and outcomes and all that, some with more vigor than others, but ultimately, for any lobbyist, the true measure of success is whether they are keeping their clients - that is, their customers - happy.
posted by ecab at 12:13 PM on June 23, 2008

For individual staff, it might be harder. But political advocacy groups seem to set measurable real-world outcomes, establish baselines, and then measure the improvement. What wouldn't work as well would be measuring actions, since there are many possible strategies for achieving the goal, and you want the staff to be creative there.

For example, let's say that I'm an affordable housing advocate. My group wants people who work in a certain city to be able to afford the median-priced home there. We establish a baseline. Right now, let's say only 9% can afford a median-priced home. (Hello San Francisco!)

Next comes the creative part. How do we boost that percentage?
- Does my team get the city to pass a law that says 30% of homes built must be affordable? Oh, the council would not vote to approve that? Well then, does electing a pro-housing councilmember become one of the campaigns?
- Or does the group get the city to subsidize nonprofits who would directly build cheap housing? Oh, the city doesn't have enough money to do that? Do you then work to pass a statewide bond to supply that funding?
- Or does the group work to raise salaries around the city, so more of the population could afford those expensive homes?
(You could use intermediate campaign goals as another possible measure of success.)

Here's a real world example: the California League of Conservation Voters. They have an annual scorecard measuring how often someone voted pro-environment. Then, they try to elect pro-environment candidates. Over time, they hope that the average score of all legislators goes up. Obviously, there's some art in establishing those indicators, and some luck and circumstance in meeting them.

Can you create measurable indicators with the help of the sales and support teams? Then, once the team is very clear on the goal, your lobbyists can work creatively to build whatever support it takes to get there. (Get Susan promoted instead of Joe since she listens to Sales. Create a Sales and Support Scorecard and get the departments to feel competitive. Who knows what they'll come up with. :) )
posted by salvia at 9:33 PM on June 23, 2008

Many thanks all for your answers and time :-)
posted by StephenF at 6:35 AM on June 24, 2008