Hiring great workers for a not-so-great task?

February 9, 2005 10:49 AM

How do I find and hire good people to do a data collection task that requires some intelligent decision-making but is also tedious? [+]

I am a doctoral student at a large public university working on gathering data for my dissertation. As part of my data collection, I will be gathering some biographical information on more than 5700 corporate directors. The information I need to gather isn’t available online. The task involves flipping pages through multiple books to find the information, and then reading short biographical sketches of the individuals and making coding decisions.

The work is tedious, but it does involve making some intelligent decisions and it is very important that it be done correctly. I will be doing some of it myself, but there is just too much for me to do it all (without postponing graduation indefinitely).

I have some funding that I can use to hire some people to help me and I want to get the most out of it. However, the problem is that I have not had good luck in the past hiring people to do this kind of work. First, I don’t have any good way to initially select them. I don’t know what questions to ask that would really show me that this person is going to work hard and be conscientious. Second, the people I did choose either ended up being very slow or very inaccurate. So, I am looking for help or suggestions on how to go about selecting good people and also about how to get them to do the task both quickly and accurately?

Thanks in advance.

posted by bove to work & money (32 comments total)

bove: like any job involving a particular skill set, there should be some sort of test. Create a test with 10 corporate director subjects and provide the relevant information to the candidates. Explain that the test is timed and that they will be scored based on a combination of speed and accuracy. Determine which is more important to you. Decide accordingly.
posted by FlamingBore at 10:57 AM on February 9, 2005

ha. i was going to say this is what grad students are for, but then i read the more inside. maybe you need to postpone the project until you have grad students of your own ;o) or, perhaps more constructively, is there some way that you can get undergrads to help you in exchange for them becoming part of the project in some way (so they have some intellectual investment in the success, as well as the monetary award)?
posted by andrew cooke at 11:01 AM on February 9, 2005

I think FlamingBore's test is a good idea. Have you thought about recruiting undergrads from your department or a similar department? Try approaching professors to ask them to recommend an advisee or student (especially in a big lecture class). Sometimes departments maintain email lists for this purpose.

This might help ensure that you have a pool of candidates for the job that at least have some understanding of your field/why this type of work is important, and also some interest in it. Plus, research experience like this would be an excellent feather in one's cap when applying to grad school, as would the recommendation you would be able to provide them with.

Beyond that, ask for references and follow up with them. Prepare a list of questions in advance--and try to make them specific. Instead of just "Was this person a good worker," try "Was he/she ontime? How did he/she handle a situation if he/she wasn't sure how to deal with it? Would you hire him/her again?" Although be forewarned that some employers will decline to answer these questions in fear of a lawsuit.

On preview: what andrew cooke said
posted by handful of rain at 11:06 AM on February 9, 2005

FlamingBore: I have thought about that idea. My only worry is that I will have individuals work hard during the test, but that after they are hired (since the task is hard to supervise) they will not work as quickly, etc.

andrew cooke: On your first suggestion, that is why I am trying to graduate quickly. I can't wait to have my future doctoral students doing this for me. Your second suggestion is great, but I am not teaching right now and therefore I am not sure how I could get them intellectually invovled, but I will think more about it.
posted by bove at 11:07 AM on February 9, 2005

involved (I thought I ran spell check), sorry
posted by bove at 11:09 AM on February 9, 2005

One idea I had was a variation on FlamingBore's. I thought that I would do a preview list of 20 names. Then, instead of interviewing, I would just hire the first 10-15 people that apply and tell them that I want them to start off just doing 20 names. Then, I would fire all of the ones who were slow, or who were inaccurate. This way I would be getting results that are likely to correspond with how they would actually work for me.
posted by bove at 11:14 AM on February 9, 2005

yeah, i couldn't think of much either. how many? if it's only a few they could be co-authors. maybe there's an undergrad conference (? do such things exist?) they could present a poster paper at? keen undergrads would like something like that for their cvs...
posted by andrew cooke at 11:15 AM on February 9, 2005

(so i guess i'm assuming that they could somehow be involved in basic analysis as well as data collection)
posted by andrew cooke at 11:16 AM on February 9, 2005

>my only worry is that I will have individuals work hard during >the test, but that after they are hired (since the task is hard >to supervise) they will not work as quickly, etc.

It will take up more of your time, but what about holding a weekly research group meeting (with free donuts). How much will you offer per hour? Assuming that you can't offer a very high hourly rate, keep in mind that sometimes you get what you pay for...might you have better luck paying one trustworthy person a higher rate than to pay several people a lower rate?

Also, try asking your workers to submit itemized time sheets if you're worried about them padding the hours. Instead of just saying 9:00-11:00. 8:30-10:00, etc., ask them to specifiy what they did during that time (i.e. I researched Mr. X, Mr. Y, and Ms. Z). If people do this sort of accounting right from the start it shouldn't be very much of a hassle (though it's terribly hard to recreate these records after the fact).
posted by handful of rain at 11:17 AM on February 9, 2005

I'd look for college juniors or seniors with at least one previous job where they did some administrative/office work and had to make independent decisions. Check their references. If you can, pay them as much per hour as a secretarial job would pay in your area.
posted by sophie at 11:17 AM on February 9, 2005

andrew: I am in the business school, and business school undergrads are usually not the sort that care about their cvs.

handful of rain: I actually have some flexibility on the rate of pay etc. Also, I like your second idea.

Thanks for the suggestions so far. As anyone who has been in graduate school can tell you, this is an important issue for me, so I appreciate your ideas.
posted by bove at 11:21 AM on February 9, 2005

Find a library school, such as this 'un, and hire everyone who wants to go into cataloging or reference. Talk to the department and discuss some way to frame the job that will make it look good on the employees' CV when they're done. If you can get them credit, so much the better. The best part is that, unlike business school types, librarians-in-training have pretty much given up any notion of making good money and will be much cheaper to hire.

Failing that, call the reference desk at the public library 5,700 times.
posted by stet at 11:31 AM on February 9, 2005

I'd also try to make this as efficient and painless as possible for the student workers. I like the idea of having each student log which corporate director he coded, so I'd include this in the training for the job.

My idea: set them up with the tools they need to do this coding work quickly. I'm thinking of PDAs.

Coding 6000 complex entries out of books is going to be a bear, especially if you have to go, look up a data element, write it down, and then transcribe it into some excel sheet or database. Could there be an easy way (i.e., no complex coding) to either give students a PDA to work on? Either they could take their notes on the local "notepad" application. Or, if there's wireless in the B-school library (or whever the books are), could they use a web front end to your data entry to enter data as they find it?

You could hire 2 or 3 students, and, along with their pay, they'd get the use the PDAs for the duration of their employmenet. Or, if you've got more money than that, you could even give them the PDA (or PDA and wireless card).

I don't mean to throw technology at a problem that might not need it, but this might be a great application for in-library data entry. This might vastly speed things up. I don't know what the effect would be on accuracy.

Each PDA costs somewhere between 10 (for local only Palm Zire) and 50 hours of a student's wage. So it's not *that* big of an expense.
posted by zpousman at 11:32 AM on February 9, 2005

As both a research-involved undergrad and a former grad student, I think one of the best things you can do overall is to create a positive atmosphere. I gladly worked for my advisor for peanuts as an undergrad because I enjoyed the time I spent in meetings with him (and also because I was interested in going on to grad school and saw that the research assistant job was worth far more than the few bucks an hour I was being paid).

So, have a weekly meeting and bring in free donuts, thank your workers (undergrads?), tell them they do a good job, acknowledge their help in your research paper or thesis, offer to be a reference in the future.

Even if business students aren't terribly interested in grad school or their CVs, try canvassing professors you know for recommendations of one or two good undergrads. You're sure to find at least a few who are thinking about long-term goals.

And, yes, look at related departments or schools--i.e. the library school suggestion above. What about political science or journalism departments?

And, if you do the itemized time sheet thing, try not to make it seem like you're cracking down. If you set that system up right from the start it shouldn't be a very big problem, but it may cause hard feelings if you try to put it in place midway through the project.
posted by handful of rain at 11:33 AM on February 9, 2005

Seconding (thirding?) stet's advice re: library school. We are all greedy for projects or skills or programs to put our CVs, or people to use as references, or things other than coursework that we can get credit for.
posted by librarina at 11:40 AM on February 9, 2005

You should also check to see if your university has any sort of research center that processes large amount similar kinds of data, such as a survey center that does a lot of mail surveys, and ask them about how they hire, train, and data check.
posted by claxton6 at 11:52 AM on February 9, 2005

stet: that is a great idea. (You even got the right university, although not too hard if you look at my profile). If the students are invested, I would worry much less about shirking, quality, etc. and this might be a perfect way to do that.

handful: I have also done lots of this kind of work. Of course, when I have done it I tried to do it well, but as you said it was because it was for projects I was either involved in, or it was for faculty that I had a good relationship with. However, your other suggestions about creating a good atmosphere are useful.

zpousman: The library has a computer lab in it, so my data collection people should be able to bring the book to a computer, and enter data directly into an Excel spreadsheet.

Thanks again for all the ideas so far.
posted by bove at 12:11 PM on February 9, 2005

I'd run a help wanted ad that says it's tedious work with some decision-making involved, then sort through the applicants in the hiring process using the techniques listed above. Some people like tedious work- being left alone to sort information would be nerdvana for me, and I can't imagine I'm the only one.
posted by headspace at 12:22 PM on February 9, 2005

I know decisions are involved on each individual piece of work, but it sounds like workload is pretty much the same for each entry. Have you considered paying it as piecework? If you offer to pay $X per coded entry, you only need to worry about accuracy, not speed, because it won't matter all that much how quickly each person works as long as you're getting enough of them done per time segment to meet your eventual deadline.
posted by jacquilynne at 12:27 PM on February 9, 2005

"though it's terribly hard to recreate these records after the fact"

Recreating may be difficult, but padding detailed records is almost easier than padding simple timesheets.
posted by mischief at 12:42 PM on February 9, 2005

Recreating may be difficult, but padding detailed records is almost easier than padding simple timesheets.

Perhaps...but in this case you'd have to submit the coding work you did to the database. It would be pretty easy to tell if it had not been done when you claimed to do it.
posted by handful of rain at 1:02 PM on February 9, 2005

As mischief notes, the problem here is that I am not supervising the work. For instance, to find the information they need about a director they may only have to consult one source, however, it may also take five sources. So it is difficult for me to know how much time each record took to collect. This is why getting good people is important.
posted by bove at 1:13 PM on February 9, 2005

jacquilynne hit it square on the nose - as long as they're getting the entries done in a reasonable amount of time, it's the best way to go.

Plus, it's an excellent motivator; you'll find that people will work much more quickly & efficiently when there is immediate incentive to do so.
posted by icey at 1:31 PM on February 9, 2005

icey and jacquilynne: I have thought about a piece rate system. The biggest obstacle to implementing it is the administrative bureaucracy that I am a part of makes it virtually impossible.

However, the other problem is that I am then only rewarding speed. This worries me. I am also very concerned about accuracy, and I don't know how to reward that in a piece rate system (especially because it is impractical for me to check their results).
posted by bove at 2:01 PM on February 9, 2005

"submit the coding work"

Sure, you submit the data, and then pad the time it took to complete. Amazing how 3.25 hours can become 4.

mischief, ex-Consultant
;-P
posted by mischief at 2:02 PM on February 9, 2005

Are you obligated to hire students? If not:

I spent a summer working for a real estate title insurance company doing title searches from deed and lien books in the county courthouse. From your description, this is virtually the same job. It demands accuracy, good judgment, and efficiency without much supervision. So another place to look for people who have the skills and the temperament might be the deed room of your county courthouse. (Or maybe it's all computers and there aren't any books or any title searchers anymore. Now I feel old.) There are (or used to be) plenty of self-selected, self-motivated researchers who do this kind of work for a living.
posted by Alylex at 2:04 PM on February 9, 2005

Here's another vote for getting undergrads who want to go to grad school. (they don't have to be in the bus-school). Ask colleagues (profs and grad students) to reccommend undergrads who are hard-working and trustworthy and interested in grad school.

Don't just send them out to get the info...Explain your research, give them a few papers to read in the area (yes, pay them for the time it takes to read them). Tell them about the data you're using (not just the data they'll be gathering, but all of it) and how you'll be using it. Ask for their ideas. This isn't just to get them engaged -- they'll need the context to make judgement calls. Expand their tasks if you can -- ask them to read the papers that come out of the project and comment and things like that.

Remember that while you may view this as drudge work, a student who is learning about how research is done will be happy to get an insider view and to develop a relationship that can lead to a great grad school reference letter.

I did things that were far more tedious as an undergrad and I was happy to do them because I was learning how research gets done, getting an inside veiw of academia and building contacts (and more generally, I was treated very well and worked in a good atmosphere).
posted by duck at 2:26 PM on February 9, 2005

Oh...and I wouldn't have padded my time. First of all, my boss was treating me well and being kind to me and giving me great opportunities. Secondly, I'm a big chicken afraid of getting caught at stuff like that and if I would have been worried that being caught would hurt my grad school references.
posted by duck at 2:32 PM on February 9, 2005

An additional thought, depending on what your available funds look like and how many students you can find to hire would be to assign the same set of records to more than one person. Assuming that this is a task that two conscientious people will come up with the same answer on most of the time, you'd be left only needing to verify the entries that don't match amongst your various reviewers.
posted by jacquilynne at 2:46 PM on February 9, 2005

Just hire other grad students, especially if they can do the work over a summer. Hiring other students in your program over a summer would be ideal, but I don't know what B-school people do in summer. ABD people, especially those who've run out of regular funding. They need the money, they're used to working alone for long periods, and it's probably no worse drudgery than their own work is.

I myself coded approximately 200 squintillion pieces of mass-mail from US Representatives to their constituents because a fellow grad student had a small grant for his dissertation; it later turned into this .
posted by ROU_Xenophobe at 3:24 PM on February 9, 2005

Alylex: I do need to hire students, because getting access to certain things in the library that I need requires you to be a student at the university.

ROU_Xenophobe: MBA students during the summer are off at very high paid internships. Business doctoral students have support throughout the year and are not looking for other work.

All of that being said, there are a bunch of great ideas here that I will use.

Thanks again for all of the thoughtful responses. You have made my first AskMe question quite a good experience.
posted by bove at 7:35 PM on February 9, 2005

I'd still bet on ABDs from most any other field, but especially a social science other than econ, as good coders.
posted by ROU_Xenophobe at 8:33 PM on February 9, 2005