help me describe the IT worker that I need, then evaluate the candidates
March 7, 2015 11:12 AM   Subscribe

At my Pretty Big company, I run a large Unix-based system that does Something Important, and we need to find someone to help me, and ultimately to take over running of the system. My two challenges right now are A) how to describe what I need to the HR folks, and B) how to evaluate the candidates that they come up with. Can you help me describe what I need, and come up with a way to test the people that show up?

While it runs on Unix, there is little actual "coding", rather a lot of minor system admin. Moving files around, restarting processes, grepping through log files to find out what happened, occasionally applying software updates. There are a LOT of other skills (centered around the actual business) that they will need to learn. I don't want to have to teach them the admin basics.

So here are the knowledge items that I thought would be good to list in the position requirements:
- Unix file system (/, /var, /home, partitioning, permissions)
- log file handling tools (piping, cat, tail, grep, more/less)
- package management (RPM, APT)
- networking from command line: traceroute, ssh, ftp (command line ftp, not graphical)

Initially, we are stating "Unix/Linux" and "Perl scripting / shell scripting" in the position description, but honestly that's really a litmus test. There isn't much script writing. The basic skills above are what I really will be checking for. I don't want to list "Linux administration" because I'm afraid that someone who's done point-and-click admin on some graphical OS will think they are qualified.

I also don't want to just list the skills above, because I'm afraid someone could just google those terms, cram/practice before the interview, and then fake their way through a whiteboarding exercise. I really want someone who is already familiar with these tools, and their subtleties, because I need to teach him/her some other things ...

This is frankly not very challenging stuff -- I just need someone who's comfortable at a Linux command line, someone that I don't have to explaining grep and piping and the filesystem to. There are other skills (more intangible) that I'll be looking for, both in reaction to what the Last Guy couldn't do. The hire will need the ability to communicate, so that (for example) he/she can understand the problem that users are complaining about, and can explain complex systems to lower level people. They'll also need to be "self motivated", which is a buzzword that probably everyone says they have, but in my workplace I see sadly little of (yes, that could be a workplace problem, I can't change it).

Then I'll need to evaluate the candidates. Do I just give them a test, make them whiteboard their answers? That seems cruel, but hey I guess that's what I have to do, huh? What are the various techniques for evaluating this? Again, this is not a pure coding position, not even close, it's more of a "motivated technician" position.

The candidates are likely to have come from a different technical field (with skills we need) and who have some Unix/Linux skills on the side.

There is a vendor that we can fall back on for support, so this person won't ever by flying alone. But we do need to have the skills internally because the system is so huge that the vendor's support group could never be expected to helicopter into every given problem.

Well, that's what I've worked up so far. I'll check here over the next couple days. Thanks in advance!
posted by intermod to Computers & Internet (14 answers total) 1 user marked this as a favorite
linux sysadmins don't point and click. you are safe to use this description, imho.

i would ask some questions related to:

how environment variables are set
what /proc is all about
how to secure mysql/postgres
monitoring of system limits on space/files/memory
posted by lescour at 11:46 AM on March 7, 2015 [1 favorite]

Best answer: Lots of really good stuff on serverfault such as this one
posted by z11s at 12:03 PM on March 7, 2015

Best answer: It reads like a junior sysadmin position to me, and you could emphasis in the job description the communication requirements. Off the top of my head - "In addition to the sysadmin requirements, this job also requires collaborative problem solving with both technical and non-technical people alike, and communicating proposed solutions in a manner understandable by everyone."

For the interview, I'd warm up with the basics, basic sys-admin type questions. Thais is where you test their knowledge of all the tailing, grepping, rpmming, sshing, etc that you care about. If they fail here, you're done. If they do ok, you can move onto the next phase of the interview. But keep in mind the specific details of the usage of rpm (or whatever) are easily looked up and learned, and are not the definition of pass/fail. This is the crappy part of interviews -- if i was at my desk, I can use man pages to my little hearts content, as long as I know what i'm looking for. Not so much in an interview or on a whiteboard.

For this second part, I'd propose to them some of these real life problems (sanitized as necessary) and ask how they'd solve them. The actual solution they come up with is at best only equally as important as the questions they ask to clarify the problem, so you can be a little deliberately vague or unclear, as non-techs (and techs as well) sometimes are. I'd say here you care more about how they think and how they communicate the solution to you than the actual solution itself (unless it's completely bonkers, of course).
posted by cgg at 12:30 PM on March 7, 2015 [2 favorites]

Yeah, this is a junior with communications skills (bedside manner) and command line proficiency.
posted by rhizome at 12:46 PM on March 7, 2015 [3 favorites]

In addition to the idea of whiteboarding an answer to a programming question, maybe also have them respond to a typical support question so you can get an idea of how they explain things to a less technical user.
posted by aimedwander at 1:28 PM on March 7, 2015

I would just use the position description to generally describe that you're looking for a junior Linux sysadmin (along with the support skills, soft skills, etc...) rather than specifically listing things like familiarity with the file system, permissions, file handling, etc... Similarly, if I were to hire a Rails developer to work on a web application, I'd just say that in the description, but I wouldn't include details like "read parameters from http requests" or "output potentially relevant information to application logfile."

Once you get applicants, you can phone screen them for basic technical knowledge. If they don't know what /home is, you can be rid of them very quickly.
posted by zachlipton at 1:37 PM on March 7, 2015

Best answer: I don't want to list "Linux administration" because I'm afraid that someone who's done point-and-click admin on some graphical OS will think they are qualified.

This is what we have phone screens for. But if you really don't want to use the term Linux administration, what about Site Reliability Engineer? Cuz really, "site" doesn't have to be a website, if that's not what you're supervising.

As far as evaluating candidates, we have some questions of the following nature:

1. Here's the output of a ps command; explain what each process is doing.
2. Here's some sample output from netstat, and a pipeline using sed, awk, grep and other coreutils. Walk us through it, and explain what the output means in English.
3. Explain what happens when a user types in into a browser and presses enter.

Finally, as far as motivation goes, a lot of that is up to management (which sounds like you?). Set some quarterly goals, and discuss how they contribute to the organization's mission, and give positive feedback for jobs done well. Maybe even rewards.
posted by pwnguin at 1:40 PM on March 7, 2015

Best answer: I agree with everyone that junior sysadmin with people skills is a good description of the position.

Have you personally done much hiring before? Something that I got wrong, then got better at, was listening to my instincts about the person; tbh, the technical capabilities for a job position are the easiest part to check for - do a few whiteboard tests, ask them how they got the skills and what they've used them for in the past, and you can be pretty sure whether they've got experience or are just bluffing/remembering last night's cramming.

You get a much better sense of whether someone's right for the job by talking about almost anything else. Depending on the candidate's age, I ask about how much they enjoyed their degree course, whether they do anything recreational with their tech skills, what they learned at their last job that wasn't related to the mechanics of doing their job, that sort of thing.

You'll warm to some candidates, and be cool on others. Trust that instinct, at least to the extent of deciding between two people who could both do the job and the one you like more looks worse on paper.
posted by Devonian at 3:35 PM on March 7, 2015

At least at the places I've encountered it, Site Reliability Engineer is a senior sysadmin++. It's someone who has serious sysadmin chops as well as software development skills (well beyond just scripting). Maybe it varies, but the SREs I know would be extremely overqualified for this. Junior sysadmin sounds about right to me.
posted by primethyme at 4:34 PM on March 7, 2015

I suggest you check out this USENIX systems administrator job description book. The core job descriptions chapter is available free, on-line, and might serve as a useful jumping off point.
posted by grudgebgon at 9:28 PM on March 7, 2015 [1 favorite]

I'd start the selection process with a hands-on technical screen. I'd spin up an AWS instance and have the applicants perform the types of tasks you're concerned about: (e.g. ssh to the machine, navigate the filesystem, parse a log file (e.g, given a fake httpd log, tell you the top 10 IPs by # of requests), upgrade/install some packages, etc.

My agenda for the call would be to learn what they're looking for in their next role, describe the company and the current role, let them ask questions, then hop into a screen-share, watch them perform each of the representative tasks, then open it back up to them for additional questions.

I tend to save softer behavioral questions for the on-site interviews because the pass rates on them are much higher than they are for technical questions, and they're easier to evaluate in person.
posted by grudgebgon at 9:48 PM on March 7, 2015

Response by poster: Thank you all for the answers so far! I just got back here after a full day offline and have skimmed the results. I will digest more fully in the next couple days. More feedback is appreciated but what's here already is great!
posted by intermod at 2:52 PM on March 8, 2015

For positions on my team we ask the candidate what he/she would do if were asked to do something and did not know how.

The answer we want to hear is "I would look it up on the internet".

We have not always had that response unfortunately.
posted by andreap at 12:40 PM on March 14, 2015

Response by poster: Just following up to thank the folks here again. I'm still planning on hiring someone, but the opening got pushed out to later in the year. I was expecting to get my hire earlier, and then I didn't :) I'll be referring to the advice here as I do that. Thanks!
posted by intermod at 11:54 AM on August 27, 2015

« Older Let's save that money! (Spy-free edition)   |   Helping an anxious kid Newer »
This thread is closed to new comments.