Combating Perception vs. Reality Issues
January 14, 2011 8:57 AM   Subscribe

I need some resources for creating more convincing analysis of real data in order to persuade a group of people who tend to prefer anecdotal evidence in decision-making.

I spend a lot of time compiling and reporting on details of project timing, profitability, and performance at work. I compile detailed reports for other members of my group. Recently, I have been running into issues when people review my analysis, but don't believe it because they are convinced by anecdotal experience ("my way is faster because it seems faster to me, even though your analysis says it isn't").

Are there any books or websites that you can recommend that will help me create clearer, more persuasive and to the point presentations of my data? Also, do you have experience with success in convincing people to use facts vs. anecdotes in making decisions?
posted by elvissa to Work & Money (14 answers total) 5 users marked this as a favorite
 
I don't know of any books or websites that may help you, but one way to address this would be to add comparisons between your proposal and "their way". It would require you to do the analysis for "their way" as well, but it would strengthen your claim if you can say: "90% of the time your way takes X amount of time and Y amount of resources, while what I propose would cut this down to W amount of time, and Z amount of resources."

Since you seem to have gathered the data already, such an explicit comparison may be relatively easy.
posted by TheyCallItPeace at 9:03 AM on January 14, 2011


I don't know of any research, but have you tried running pilot programs with your ideas? That would generate anecdotal evidence you could include with your actual data.
posted by dadici at 9:06 AM on January 14, 2011


I think it just takes time and practice to develop a skill / intuition for presenting data clearly and convincingly. If you do this often at work, that's a great chance to experiment with different methods of presenting and communicating your data.

The books of Edward Tufte are good. There are also quite a few good blogs on the topic of visualizing and communicating data. I like: Well-formed data, Flowing Data, and Information is Beautiful.
posted by JumpW at 9:09 AM on January 14, 2011


Dadici has it. Generate your own anecdotes using the superior methods you've figured out.

People have varying degrees of cognitive bias in favor of stories, and it sounds like you've got a group particularly susceptible to that - good luck.
posted by downing street memo at 9:10 AM on January 14, 2011


Sometimes you need to explicitly state the entrenched belief before you present the data which contradicts it. If you can present the information as a sort of "popular misconceptions" thing, implying that you too were of the belief that X worked best, but were won over by the obvious superiority of Y (and here are the figures!), it may help to get others engaged.
posted by le morte de bea arthur at 9:15 AM on January 14, 2011 [1 favorite]


Are there any books or websites that you can recommend that will help me create clearer, more persuasive and to the point presentations of my data?

I feel like you're trying to use science to convince fundamentalists that god doesn't exist. What I mean is that you and the people you're presenting to have different standards of truth.

I think you may need to somehow transform your data into anecdotes and stories. They'll be grounded in your standard of truth, so you're not lying, but you'll be speaking their language.

I don't think it will work for you to present your "science" ever more clearly and carefully. What if they just don't care?

Also, there is truth in their anecdotal experience, for them. Listen to what they're saying. Are you taking their concerns into account or are you brushing them aside? What you want is buy-in. Therefore you need to legitimately address their concerns, in their own words, on their own terms.
posted by zeek321 at 9:20 AM on January 14, 2011 [1 favorite]


Show them how their anecdotes/counterfactuals are already in the data. So for example if you are comparing two methods of doing something, don't just say the average time with method A was 23 days and with method b was 17 days. That opens it up for them to say "But I used method A and it took 10 days!" as though that somehow contradicts your finding.

Instead say: "Method A and B were each used 50 times. This table shows how often each method took over the 50 trials.
               A        B	
10 or less    11       16
11 to 20       9       21
21 or 30      11        6
31 to 40       4        2
41 to 50      15        6
    Mean      26     18.6
Now as you can see, both methods sometimes worked very quickly (under 10 days) and both sometimes took a long time (more than 41 days). However, Method B was for more likely to result in fast completion of the project and less likely to result in delays. In fact, the average time to completion for method B was almost 8 days faster!"

Then if someone says "But I used method A and it took 4 days" you say "That's great, you must have been one of these 11 people, here. It definitely happens, but of course in general you're better off going with method B."
posted by If only I had a penguin... at 9:44 AM on January 14, 2011 [2 favorites]


Do you include savings estimates in your analysis? "This method will save the company $4k/mo" might get corporate types' attention a lot more than "This method is 23% more efficient".
posted by auto-correct at 10:08 AM on January 14, 2011


In "The Art of the Long View", Peter Schwartz discusses using scenarios to present his futurism predictions in order to better connect with the audience. This is basically what others in this thread are saying: Make up anecdotes that support your data, find ways to tell stories about your data rather than just presenting your data.

And the fact that Schwartz's airplane reading style business book about "the future" (ie: usually the sort of thing that's completely disposable) has stayed in print for two decades suggests that it might be useful to you in better building these narratives. It has been for me.
posted by straw at 12:07 PM on January 14, 2011


If only I had a penguin...: "Now as you can see, both methods sometimes worked very quickly (under 10 days) and both sometimes took a long time (more than 41 days). However, Method B was for more likely to result in fast completion of the project and less likely to result in delays. In fact, the average time to completion for method B was almost 8 days faster!""

I totally can't see, though. I do not speak statistics. At all. Penguin's lovely table is completely without meaning to me, as in it communicates nothing to me. Numbers I can understand are limited to: totals, percentages %, averages, mosts and majorities.

You can help me understand what you are trying to say and get me on board by telling me a story about a person in the study who happens to be in your mean or whatever. If Cathy used to use the photocopier but now gets through 18% more work using a scanner, I can understand that and will probably support the purchase of scanners.

If you are going to force me to look at numbers that are not financials, please give me a pie chart.

I am sorry if this breaks your statistics loving heart.
posted by DarlingBri at 2:23 PM on January 14, 2011


Just watched this video I think you'd like: Creating a Data Driven Culture, and while his experience is mainly about Google Analytics, the basics are similar:

1. Focus on outcomes. Especially early on.
2. Divorce your opinion from the data. It's not your data, or your analysis.
3. Empower analysts. Generating and reviewing regular, structured reports should only be a small portion of time spent.
4. Solve for people.
5. The business owns the data, not IT or one or two analysts.

Sounds like #2 might be part of the problem. It might also be worth pulling anecdotal evidence out and examining it.
posted by pwnguin at 4:37 PM on January 14, 2011


DarlingBri: Numbers I can understand are limited to: totals, percentages %, averages, mosts and majorities.

Those are totals. Method A took less than 10 days 11 times, Method B took less than 10 days 16 times. Nobody with basic numeracy should have trouble understanding this (open a grade 5 math textbook and you'll find a table like this), and no one without basic numeracy should have a decision-making job.

I think the problem with using hypothetical or real anecdotes is that it suggests that anecdotes are a valid form of evidence and encourages people to believe that they are. Using them to help people understand is one thing, using anecdotes to persuade is dishonest since anecdotes are not evidence and that which we use to persuade should be evidence. So first persuade with evidence and if that leaves people with "but why would B be faster...?" then use a hypothetical or real anecdote to demonstrate/explain why.

The reason anecdotes are not evidence is that there's no way to adjudicate between multiple anecdotes. So I tell you a story about Cathy and the scanner and you tell me a story about Joe and the photocopier and we are at an impasse. You need numbers (like totals) to tell you whether Cathy or Joe are more common. The problem is that when you tell people "Cathy is more common" some people seem to hear "There's no such thing as Joe" and want to tell you their Joe story as if that somehow constitutes evidence that Cathy isn't more common. So acknowledge that both Cathys and Joes exist, but provide evidence that Cathys are more common.

Though if you insist those could easily be two pie charts and make the same point.

Also, if people can only interpret numbers that are money (really? because numbers are numbers and are interpreted exactly the same way, so this would be some soft of emotional block and not a skill issue.) then fine, turn the numbers to money. Assume each day costs $1000 and you can do the exact same table with money. I'll add labels this time:
                             Number of Projects
Total Cost of Project
                           Method A    Method B	
$10,000 or less                 11       16
$11,000- $20,000                9        21
$21,000 -$30,000                11        6
$31,000 -$40,000                4         2
$41,000 -$50,000                15        6
               Average     $26,000     $18,6000
There. Now if you have someone with some emotional block preventing them from reading tables that don't have dollar signs in them, you'll have them covered.
posted by If only I had a penguin... at 7:49 PM on January 14, 2011 [2 favorites]


If only I had a penguin...: "numbers are numbers and are interpreted exactly the same way, so this would be some soft of emotional block and not a skill issue."

To restate DarlingBri's comment a bit more generously, executives are often focused on their personal goals (set by their boss), frequently measured in dollars. If you find out their goals you can tweak the presentation to the things you can safely assume they're interested in, which can be vital. If their bonus is based on growing revenue, and you're presenting on how to cut costs, that alone is a great incentive to discredit the data.
posted by pwnguin at 8:28 PM on January 14, 2011


The problem with the people that are saying "numbers are numbers, data is superior and everyone SHOULD understand raw data, and if they don't they're incompetent then they shouldn't have jobs" is that "should" has very little value in the real world.

These are the people who are in charge of making decisions, if you don't like how they're forming their decisions then you have 3 choices. 1) quit, and go find a job with a culture that more closely aligns with your way of thinking 2) give up and accept the way things are and the fact that you're not going to change the way things are done at this company 3) figure out how the people you're dealing with process information, and change the way you're presenting your data to more closely align with their needs.

Refusing to adjust the way you see/do/process/express things doesn't make you right, it makes you rigid, inflexible, and ultimately obsolete.

O.P. - clearly not aimed at you as you've identified the problem at hand an sought advice on how to adjust your approach.
posted by dadici at 10:47 AM on January 18, 2011


« Older Colors/Syncing/Calendars   |   Looking for a recipe from "Recipes For a Small... Newer »
This thread is closed to new comments.