Is there a minimum sample size required to use the bell curve for performance management?
July 21, 2012 3:20 PM Subscribe
Is there a minimum sample size required to use the bell curve for performance management? I was wondering if small teams (3-5) can use the normal curve / bell curve for categorizing employees by performance? I have a feeling that the sample size needs to be much larger than that (3-5) for the bell curve to apply. Anybody know if there is a minimum? If possible please share link to supporting evidence.
Most things in the world are normally distributed when you look at a large enough sample, and 3-5 is no where near adequate to infer normality or non-normality, unless you have a reference group at a very similar large corporation with say, 100 or 500 employees, and even then there would be non-trivial issues with making sure your employees are really like theirs (matched samples). Depending on what you need to know and how much data you have from each employee (ie sales data over time, etc) bootstrap techniques may get you somewhere with your small sample, but why aren't descriptive statistics on each employee adequate? What are you hoping to find out that you can't by simply comparing Employee As average sales for November with Employee Bs average sales for November? Or comparing Employee As average sales with the whole group average?
I ask because the wording of your question and the fact that you're asking in the first place makes me think you do not have enough statistical expertise to apply the kinds of techniques or analysis you're thinking of, especially if it has real consequences for performance evaluation, compensation, etc. Don't try to use math that's over your head in the name of "scientific management."
posted by slow graffiti at 3:52 PM on July 21, 2012
I ask because the wording of your question and the fact that you're asking in the first place makes me think you do not have enough statistical expertise to apply the kinds of techniques or analysis you're thinking of, especially if it has real consequences for performance evaluation, compensation, etc. Don't try to use math that's over your head in the name of "scientific management."
posted by slow graffiti at 3:52 PM on July 21, 2012
The goals of statistics are to 1) infer about a large population from a small sample, and 2) quantify the uncertainty of a measurement. The math accounts for sample size, so it's unlikely that a sample of 3-5 could give you meaningful inferences about a large population.
As an employee, I'd seriously question any manager who used statistics to categorize a team of 3-5. I'd think you are hiding behind the statistics when you deliver bad news. "It's not my fault your salary got cut, the numbers told me to do it."
posted by scose at 4:13 PM on July 21, 2012 [1 favorite]
As an employee, I'd seriously question any manager who used statistics to categorize a team of 3-5. I'd think you are hiding behind the statistics when you deliver bad news. "It's not my fault your salary got cut, the numbers told me to do it."
posted by scose at 4:13 PM on July 21, 2012 [1 favorite]
Response by poster: Thanks supercres and slow graffiti.
I am just curious since a lot of companies use bell curves for performance management.
I didn't think it made sense unless the teams were large (at least 30). Managers are required to force rank their employees and align them to a normal curve! Also, I thought the sample had to be randomly picked. Companies don't pick employees randomly. They don't go grab every 17 the guy etc. So that could be another reason why the bell curve would not be an appropriate tool to use for performance management. Thoughts?
posted by r2d2 at 4:16 PM on July 21, 2012
I am just curious since a lot of companies use bell curves for performance management.
I didn't think it made sense unless the teams were large (at least 30). Managers are required to force rank their employees and align them to a normal curve! Also, I thought the sample had to be randomly picked. Companies don't pick employees randomly. They don't go grab every 17 the guy etc. So that could be another reason why the bell curve would not be an appropriate tool to use for performance management. Thoughts?
posted by r2d2 at 4:16 PM on July 21, 2012
If the company is using a reference curve made up of data from all of its employees in Position X (say 500 Sales Associates) and then asking you to find where your three to five direct reports fall on that curve, that could make sense -- assuming that the original curve was constructed sensibly and your reports are basically representative of the population used to construct it (assuming they're not Programmers instead, or they all work the night shift or something).
It could be used to help assess their performance relative to the company average, but not relative to each other. For comparison within your small group, you'd want to use a simple ranking structure.
Are you aligning their performance statistics against a pre-constructed reference curve, or are you being asked to build a curve using only the data from your direct reports? The former is mor sensible than the latter by far.
posted by Scientist at 4:33 PM on July 21, 2012
It could be used to help assess their performance relative to the company average, but not relative to each other. For comparison within your small group, you'd want to use a simple ranking structure.
Are you aligning their performance statistics against a pre-constructed reference curve, or are you being asked to build a curve using only the data from your direct reports? The former is mor sensible than the latter by far.
posted by Scientist at 4:33 PM on July 21, 2012
Companies in this day and age don't really have to sample. The curve can be made of the actual population: the actual sales data, say, for every single sales rep in the company, and you can very easily from that point tell who is where in that population. It doesn't really extrapolate to the population of all sales representatives selling that product in every company everywhere, because it's not random, but it doesn't have to; once you're looking at every sales rep you have, then if you need better performance out of the group, taking the bottom 10% and either giving them more training or paring them to replace them with hopefully-better staff... that makes perfect sense.
Statistics are a tool. In the case of a team of 3-5 people, they are not the right tool for the job. Joe might have the worst sales numbers on his 5-person sales team, but if he does a great job at helping others, he might also be the most valuable person on that team. Joe might have the best numbers on a poorly-performing team and be such an awful person that he's causing productivity to suffer. As a team leader, you can address these things, and should. The people who are dealing with corporate strategy and a thousand employees in that role, for example, don't have that access, and need different tools. Or in other words, curving a small group is like trying to turn a screw with a hammer. But hammers are still perfectly useful in other respects.
posted by gracedissolved at 5:42 PM on July 21, 2012
Statistics are a tool. In the case of a team of 3-5 people, they are not the right tool for the job. Joe might have the worst sales numbers on his 5-person sales team, but if he does a great job at helping others, he might also be the most valuable person on that team. Joe might have the best numbers on a poorly-performing team and be such an awful person that he's causing productivity to suffer. As a team leader, you can address these things, and should. The people who are dealing with corporate strategy and a thousand employees in that role, for example, don't have that access, and need different tools. Or in other words, curving a small group is like trying to turn a screw with a hammer. But hammers are still perfectly useful in other respects.
posted by gracedissolved at 5:42 PM on July 21, 2012
Have you looked at the control chart that W. Edwards Deming used? He hated ranking of employees, but pushed for proper use of stats and measurements.
posted by jade east at 6:23 PM on July 21, 2012
posted by jade east at 6:23 PM on July 21, 2012
If I worked on a team of 3-5 people and my boss said, "We are going to assume everyone's performance as measured by metric a, b, and c is normally distributed…" and then tried to make some management decisions based on that, I would quit that job immediately. It's idiotic.
I guess what annoys me so much about it is the compounded situation of assuming that you need some sort of statistical framework like this to measure how well 3-5 people are doing their jobs, *and then on top of that choosing a completely moronic way to do it*. Imagine if baseball GMs did this. I'm not really any kind of a baseball fan but I would assume player performance on baseball stats is not normally distributed. Why would it be? You didn't sample the whole population's baseball skills, you lopped off one of the distribution and made them professional ball players.
Who's to say your team isn't like that? Never mind the blunt instruments that would have to feed into this system, for a team of 3-5.
Basically, this sounds like having a really stupid idea, and then finding an even stupider way to implement it. Like "Hey, I'm going to cut off my feet so I don't have to waste time putting my shoes on every morning. Furthermore, I'm going to do it with this pair of antique silver spoons."
Most things in the world are normally distributed when you look at a large enough sample
Is this actually true? I think people are familiar with normal distributions, and they follow intuition, and so people see them everywhere. Although this is related to a general pet peeve I have with anyone seeing a roughly symmetric-about-the-arithmentic-mean distribution and saying "ITS NORMAL LET US MAKE A BUNCH OF CALCULATIONS THAT ARE ONLY VALID IF THIS DISTRIBUTION IS NORMAL WHICH CLEARLY IT IS BECAUSE LOOK IT IS SHAPED LIKE A CLOCHE." Aka "one of the key ingredients in every single global financial crisis of the last 100 years"
posted by jeb at 6:27 PM on July 21, 2012 [2 favorites]
I guess what annoys me so much about it is the compounded situation of assuming that you need some sort of statistical framework like this to measure how well 3-5 people are doing their jobs, *and then on top of that choosing a completely moronic way to do it*. Imagine if baseball GMs did this. I'm not really any kind of a baseball fan but I would assume player performance on baseball stats is not normally distributed. Why would it be? You didn't sample the whole population's baseball skills, you lopped off one of the distribution and made them professional ball players.
Who's to say your team isn't like that? Never mind the blunt instruments that would have to feed into this system, for a team of 3-5.
Basically, this sounds like having a really stupid idea, and then finding an even stupider way to implement it. Like "Hey, I'm going to cut off my feet so I don't have to waste time putting my shoes on every morning. Furthermore, I'm going to do it with this pair of antique silver spoons."
Most things in the world are normally distributed when you look at a large enough sample
Is this actually true? I think people are familiar with normal distributions, and they follow intuition, and so people see them everywhere. Although this is related to a general pet peeve I have with anyone seeing a roughly symmetric-about-the-arithmentic-mean distribution and saying "ITS NORMAL LET US MAKE A BUNCH OF CALCULATIONS THAT ARE ONLY VALID IF THIS DISTRIBUTION IS NORMAL WHICH CLEARLY IT IS BECAUSE LOOK IT IS SHAPED LIKE A CLOCHE." Aka "one of the key ingredients in every single global financial crisis of the last 100 years"
posted by jeb at 6:27 PM on July 21, 2012 [2 favorites]
If you're talking about stack-ranking, that's for bigger groups of people.
posted by rhizome at 7:49 PM on July 21, 2012
posted by rhizome at 7:49 PM on July 21, 2012
I think the OP was referring to the fact that organizations have decided that the bell curve applies to any team, regardless of size, so that your rating of your team is 10%-30%-40%-20% or whatever the ratio is.
Which essentially means that regardless of any number achievement, you would have to rate a percentage of your team in the bottom 10-20% (the percentile approach).
In other words, use the hammer, but a smaller one for small teams.
OP: I don't think statistics is applied during performance appraisals, since no one looks at statistical significance or variation or confidence levels. "Just bucket your resources" - I heard one senior, well-respected VP say.
posted by theobserver at 8:54 PM on July 21, 2012
Which essentially means that regardless of any number achievement, you would have to rate a percentage of your team in the bottom 10-20% (the percentile approach).
In other words, use the hammer, but a smaller one for small teams.
OP: I don't think statistics is applied during performance appraisals, since no one looks at statistical significance or variation or confidence levels. "Just bucket your resources" - I heard one senior, well-respected VP say.
posted by theobserver at 8:54 PM on July 21, 2012
Stacking bonuses are an insane, pseudo-scientific idea. Please don't do this, the world is bad enough as it is.
posted by Yowser at 11:19 PM on July 21, 2012
posted by Yowser at 11:19 PM on July 21, 2012
r2d2: "a lot of companies use bell curves for performance management.
I didn't think it made sense unless the teams were large (at least 30). Managers are required to force rank their employees and align them to a normal curve!"
Ah, stack ranking, aka the technique that destroyed Microsoft.
posted by Lexica at 10:35 AM on July 22, 2012
I didn't think it made sense unless the teams were large (at least 30). Managers are required to force rank their employees and align them to a normal curve!"
Ah, stack ranking, aka the technique that destroyed Microsoft.
posted by Lexica at 10:35 AM on July 22, 2012
I just finished my degree in organizational behavior management. In my opinion, the stats are only useful for prediction and quantification of measurements. To that end, you need a large number of observations to find any useful data.
If you wanted to rank employees best to worst, applying a normal curve to 4 people would be worthless. You would have one bad, two middle, and one good. But that doesn't tell you anything you don't already know. So, if management wants to make you do this, you can do it, but it'd be worthless.
In my opinion, a normal curve is only useful for baseline measurements. It doesn't take into account the potential for improvement. Performance Management should be all about improvement (after all, that's where the money is). If baseline performance scores at p1 = 13, p2 = 13, p3 = 14, p4 = 17, then that will get "curved" and let you fire someone or whatever. But with true performance management, that performance could be upped to p1 = 25, p2 = 29, p3 = 23, p4 = 20. Which is clearly not reflective of the original measurement.
Either way it doesn't matter, because "best" and "worse" is completely unrelated to standards of performance. People think that good workers = hard workers, and bad workers = lazy, sinful, unworthy workers. But that's about as far from scientific performance management as you could get. Investigate what the standard of performance is, and then work with your employees to achieve that. But if you are going to make them work harder, you better expect to pay them more.
posted by rebent at 12:20 PM on July 23, 2012
If you wanted to rank employees best to worst, applying a normal curve to 4 people would be worthless. You would have one bad, two middle, and one good. But that doesn't tell you anything you don't already know. So, if management wants to make you do this, you can do it, but it'd be worthless.
In my opinion, a normal curve is only useful for baseline measurements. It doesn't take into account the potential for improvement. Performance Management should be all about improvement (after all, that's where the money is). If baseline performance scores at p1 = 13, p2 = 13, p3 = 14, p4 = 17, then that will get "curved" and let you fire someone or whatever. But with true performance management, that performance could be upped to p1 = 25, p2 = 29, p3 = 23, p4 = 20. Which is clearly not reflective of the original measurement.
Either way it doesn't matter, because "best" and "worse" is completely unrelated to standards of performance. People think that good workers = hard workers, and bad workers = lazy, sinful, unworthy workers. But that's about as far from scientific performance management as you could get. Investigate what the standard of performance is, and then work with your employees to achieve that. But if you are going to make them work harder, you better expect to pay them more.
posted by rebent at 12:20 PM on July 23, 2012
« Older What sort of server & network setup would work... | Instad of right-click, "key-click"? Newer »
This thread is closed to new comments.
posted by supercres at 3:45 PM on July 21, 2012 [1 favorite]