How to do 12 half-hour programming interviews in a row?
January 18, 2014 7:04 AM   Subscribe

I am programmer who has done dozens of phone screens and about a dozen on-site interviews. I will soon be sent back to my alma mater to do my first day of "on-campus interviews," which means they will put me in a room for seven hours and parade a dozen candidates through at half-hour intervals. Could anyone with experienced in this sort of thing look over my plan and give me any advice they have? I'm especially concerned about evaluating people fairly under these conditions.

My schedule is three candidates at half-hour intervals, a fifteen minute break, three candidates, an hour for lunch, three candidates, a fifteen minute break, and three candidates.

The bulk of my interviewing experience has been with phone screens, which are usually thirty minutes just for programming, not including pleasantries to open, or time for the candidate's questions at the end, or time afterward for me to write up my notes. I've also done some on-site interviews, which are at least an hour of programming, with another interviewer, and some time to discuss afterward. For both of these, it's rare to do two a week, never mind a dozen a day.

So my concerns are
- evaluating people in much less time
- cutting people politely if they go over time
- evaluating way more candidates in a single day

The plan so far:

- To help myself stick to the thirty-minute limit, I went around and got my coworkers to suggest a bunch of shorter questions. But of course if people get lost in the weeds they can take as long as I'll allow on any question.

- Based on what people have told me about setting expectations, I'm going to start with this rehearsed introduction: "Hi, I'm Anonymous, from The Firm. I'm going to ask you a quick question about computer science. I would like you to discuss the algorithm in English and then write the solution in any programming language you choose. Feel free to ask questions and think out loud. In fact, I strongly encourage you to talk as you go, because I will almost certainly have to kick you out before you can finish."

- I'm going to kick them out at twenty-five minutes, mid-sentence if I have to, so I have a few minutes to take notes before the next guy walks in. I'm usually told not to think about race, gender, etc., but for this I've been advised to write a few words about physical characteristics and clothes just so I have a chance of remembering who was who at the end of the day.

- I've been told my job is not to produce the usual multi-facted evaluation but just to make a quick judgment as to whether this person is obviously terrible. Anyone who isn't obviously terrible will get another interview.

- I'm going to keep a water bottle and some snacks in the room, and take a nap during my lunch break if I can, just so I don't start failing people just being I'm tired and cranky. But I'm pretty sure I'm going to be tired and cranky by the end of the day anyway.
posted by anonymous to Work & Money (13 answers total) 1 user marked this as a favorite
 
Bring air freshener.
posted by srboisvert at 7:45 AM on January 18, 2014 [2 favorites]


So how complex of a problem do you plan to present? Are we talking FizzBuzz or something like "write me a spell checker"? My gut says you're going to find 30 minutes is way too tight a timeframe to do what you plan unless the question is a very simple and well-designed weed-out question.

I think you're going to find the process is a lot slower doing F2F interviews with freshouts. They will be a lot more nervous than your typical experienced candidate doing a phone screen.

Thinking back to when I was a freshout, if I met with a person from AnonCorp and they told me this:

"...because I will almost certainly have to kick you out before you can finish."

...I'd comply because I'd be too nervous to speak up, but inside my head I'd be thinking "this guy obviously expects me to fail. AnonCorp are a bunch of assholes".

Remember that on-campus interviews are also P/R for your firm. Think hard about the face you present.
posted by JoeZydeco at 7:52 AM on January 18, 2014 [1 favorite]


You employer gave no advice here? Are you sent by yourself? I've done a bunch of these in the past and we always set some ground rules for folks to follow. But here's my advice:

Don't give a problem that you think most people will take longer than 15 minutes to solve. Most people will in fact be unable to solve it in the time period anyway, in my experience. This type of interviewing is usually supposed to replace a phone screen, but standards for straight out of college hires are almost always different than those for experienced hires. I'm not kidding when I say that problems that boil down to nested for loops are often the right type of thing to ask people to solve (or FizzBuzz, as mentioned above). Yes, people CAN get lost in the weeds, and that is usually a sign that they won't work out. But you should not let anyone flounder for 25 minutes only to then kick them out. Their time is up in 15 or maybe 20 minutes, so that you can answer their questions EVEN IF YOU DON'T WANT TO HIRE THEM. I think it is also important to give them a few minutes at the beginning to tell you a little about themselves and warm up.

You're not being asked to find people to definitely hire, just to go on to the next level, so don't overthink it. You are there to evaluate how articulate they are, how enthusiastic they are for your employer, how well you think they would fit in to the company, and whether they are even worth screening further.

If you message me I'm happy to give you my go-to question for college hires that should take about 15 minutes.

Here's how you politely cut them off. They are rarely making progress at this point, so you want to do something along the lines of "This is great progress, but I'm afraid we have to stop so that I have time to answer any questions you have for me." And it's perfectly ok to say "we only have a short period of time so I may have to interrupt you in order to stay on schedule" at the beginning of the interview, but your script right now is way too aggressive.
posted by ch1x0r at 8:06 AM on January 18, 2014 [2 favorites]


Anyone who isn't obviously terrible will get another interview.

This being the case, I'd say you're probably over-thinking it.

Maybe you work in a more formal setting, but in my experience, this sort of thing is usually handled in a 15-20 minute phone screen.

The key is to decide what your company values most. I've been in companies where the initial screen was 100% problem-solving, 0% programming. Some the opposite. Are you hiring for an architect role where deep knowledge of algorithms and data structures are vital or is this a junior dev where not being able to implement a quicksort algorithm is no big deal? Do you care how well they know a particular language?

In practice, if you're asking anything that requires discussion you've got time for a max of two questions. As a result, I'd encourage you to prepare three or four questions that are situated at a level where marginally terrible folks will struggle and eventually fail while marginally acceptable people will struggle and eventually succeed. A lot of this will depend on what they're studying and how much experience you expect they'll have by this point (in other words: screening freshman for summer internships is a completely different ballgame from screening CS masters degree candidates for full-time employment after graduation)

I'm usually told not to think about race, gender, etc., but for this I've been advised to write a few words about physical characteristics and clothes

This is probably a hair paranoid, but I would advise you to not write a single word about their gender or race/ethnicity. Just don't do it. Find some other way to remember who they are. It might be 100% innocent and have no bearing on your hiring decision, but if one person sees you wrote "[race], [gender]" on your interview notes, you could be in for a bad time (even if that person is a co-worker).

Instead, if you have access to their resume, I'd look at something before they come in that makes them fairly unique. Where was the last place they worked? Where did they go to school before the current place? Did they have some kind of weird tech on their resume? Some org or hobby you find distinctive? Decide this before they walk through the door. Then, in your head, from then on you can think of them as the Guy Who Worked at Random, Inc. Associate that with whatever physical cues you need to put a face with a name if you absolutely have to remember what they look like (although you really shouldn't need to remember what they look like to make a hiring decision).

In fact, I strongly encourage you to talk as you go, because I will almost certainly have to kick you out before you can finish.

Find a better way to say this. "These are really short interviews and I don't want to keep the folks after you waiting, so please feel free to discuss at length, but please don't feel bad if we don't get all the way through an answer, [your school] is really particular that I get you all in and out on time. " Something.

Also, don't freak out. It's only 12 people and, frankly, you might as well run it like you do a phone screen.

As an aside, because I don't think this is the style you're really going for: I vastly prefer to ask people to talk about something THEY are good at instead of something I wish they were good at. I'll find some project, previous employer, etc. that they mention and ask them questions about it, diving as deep as they're willing to go. People who are terrible and/or faking it out themselves pretty quickly.
posted by toomuchpete at 8:09 AM on January 18, 2014


I would be upfront that there are major time constraints and if you are interested a longer interview will follow. Tell them you will end the interview regardless of the conversation at 25 minutes and you are not trying to be rude. In addition they shouldn't worry that they don't get to say everything thet want. That way people are not confused when you say ' okay that is it goodbye.'
posted by AlexiaSky at 8:10 AM on January 18, 2014


I've done these. I felt the same way the first time. Your plan sounds good and you'll be fine.

Some thoughts in no particular order:

– Pick a really simple programming question. You really are looking to set a baseline that filters the completely non-technical from those with actual talent or experience. 99% of them have done zero real programming to this point, so you need problems that can be solved with only the slightest insight and the most basic knowledge. FizzBuzz is good (but change it up a little, they read blogs too); the one I used was given two RECT structs, write a function that tells me if the two rectangles intersect.

– Don't worry about completely correct answers or syntax. You want to find people who seems smart. Smart people will learn. Just ask yourself, "did this person seem smart?" Other questions to help you might be, "did they seem passionate about technology / my company?" and "would I want this person in my office everyday asking me questions?" and "would I want this person on my team, reporting to me?"

– If you pick the right question, you don't have to cut them off. You just give them the hints they need to get it done, or you just say, "okay, we're running out of time but I see where you're going so it's fine."

- Short answer trivia is fine too. "What's a struct? How is it different than a class?" "What's a pixel?" "What is a thread?" "What is a pointer?" "What is a buffer overflow?" "What is an advantage of CPU cache over RAM over disk storage?" "What is one difference between C and C++ and / or Java?" etc.
posted by jeffamaphone at 10:02 AM on January 18, 2014 [2 favorites]


Plan to wrap the interviews up in 20-25 minutes so you have time to make/edit your notes as you go, because by the time you get to the afternoon, things are going to run together.
posted by Good Brain at 11:03 AM on January 18, 2014


Mod note: From the OP:
I should probably clarify that of course I was not instructed to take notes on protected classes. This company is actually goes well beyond the letter of the law here; I remember my manager got a resume with a headshot, and muttered, "Great, just bias me already."

I won't take those notes if it could be a problem, but I am very concerned about mis-attributing things and evaluating a candidate based on what some other guy did. I'd appreciate any suggestions for reducing that risk.

ch1x0r, I'd love to read your interview questions. I should stick to the questions my company uses, but maybe by reading your questions I can calibrate myself to what you're looking for. Could you email them to askmefi255524@gmail.com ? It'd be great if you could also include a first-round question for experienced candidates and a final-round question for summer interns, for comparison.
posted by restless_nomad (staff) at 1:45 PM on January 18, 2014


I would recommend varying your puzzle/design question. If not hourly, then at least switch to a different one in the afternoon. Unlike typical phone screens, all of these candidates are in one geographical location, and they talk.
posted by ceribus peribus at 1:57 PM on January 18, 2014 [2 favorites]


You're overthinking this. You'll be able to identify the truly below par people in the time you have and from what you said that's the only thing you need to remember about any of them. So stop worrying about confusing them in your mind. Just determine what you are asked to determine.

You can make that decision after each candidate because the bad ones will not become any better by comparison and if there are a lot of bad ones so be it.
posted by koahiatamadl at 2:04 PM on January 18, 2014


If you want to make notes on appearance etc, put them on post-it notes that you can pull off the file later.
posted by AnnaRat at 2:17 PM on January 18, 2014


I do this - same number of interviews, same schedule - every 4 months, and have for the past 3 years. My team regularly includes 3-7 students.

You have a very solid plan. Your initial script is good, but be sure to include a small amount about The Firm, and the role within The Firm. Remember that despite the time constraints you are selling this position too and students at good schools (I usually recruit from University of Waterloo, I imagine any decent engineering/math school the students will have similar opportunities).

My schedule is usually:

Minutes 0-4: Quick run down of my The Firm, my The Program and my The Role I'm hiring for, and any questions they may have. Warning that an audible alarm goes off and that a relatively straightforward programming question which they can solve in any language is coming.

Minutes 4-9: Background questions from their The Resume, usually trying to drill down into the actual work they did - any projects, what their role in the project was, what technical challenges they face and how they beat them.

I ask every student "Tell me about a project you worked on that you loved - either for school, work or something done in your spare time." If they can't, that is a strong negative indicator to me. Most can, and this is often the most telling question I ask. It is also almost always a sure-fire way to pull a nervous nerd out and get them a bit fired up about something.

Minutes 10-19: FizzBuzz. It is super cliche and anyone past their second year of school has probably (though only 60%ish) seen it, but if they have that's actually a good sign - it means they research the typical stuff facing them which is a positive indicator. Out of the 12, at least 2 will astound you with how much they struggle, and 1 or 2 will do something interesting. Be sure to let them use any language they like - the guy who did it in Haskell then explained what everything was to me still sticks out to me 18 months later (he did not accept my offer).

Minutes 20-24: Their turn for questions. These are typically "describe a regular day at The Firm", "What are the salary expectations", "Can you describe a project I'll be working on" and one or two other provided by their professional development courses.

Minute 24: Audible alarm goes off.

Minute 25-29: Padding. Frequently this is unused and gives me time to make notes.

I make very light notes, basically "Yes", "No" or "Maybe", and in the Yeses I usually think about ranking. "C is better than A but not as good as B" and give them a 0.0 - 1.0 score.

The time during the programming question is also usually a great opportunity for taking notes based on how things have gone.

It is very rare for students to take the full 10 minutes for the programming question, if they take more than 5 something is either wrong or they are focused on making things exactly right.

All this goes out the window if someone pulls out a demo. We then usually go a couple of minutes long on the demo because it's amazing to talk to someone about something they've built.

Finally, take some time on your breaks to get out of the interview room. Walk around the building, go to the bathroom and splash some water on your face, grab a coffee (many schools I've been to give it to interviewers free) if that is your thing. And bring cash, if the school you're going to is like the one I do, they don't take credit or debit cards, preferring cash or the campus payment system cards.
posted by cCranium at 3:34 PM on January 18, 2014 [2 favorites]


My system for note taking was to just put my initials on the top of their resume. All the way to the left was a strong no-hire. All the way to the right was a strong hire. With the obvious spectrum in the middle. That way I could remember later what I thought of them.
posted by jeffamaphone at 3:49 PM on January 18, 2014 [5 favorites]


« Older If I Eat Any More Leaves I Am Going To Turn Into A...   |   Black Hats May Have Had Remote Access to My Home... Newer »
This thread is closed to new comments.