Democracy is the worst form of government, except for all those other forms that have been tried from time to time.
– Winston Churchill

I get asked a lot by non-technical people why it’s so hard to find good technical talent. How it is that we can do so many interviews, yet make so few offers. Are we too picky? Is our technical interview process too hard? How can someone have graduated from an elite school, have great internships, and have offers at companies X, Y, and Z (all great companies with tough interview processes), yet not be able to make it through our interview process? We spend a fair amount of time agonizing over this.

We desperately want candidates to succeed

First off, let’s get one thing straight. The goal of the interview process is to expose awesomeness, not to make a candidate feel bad and waste everyone’s time. We’re investing many hours of engineering time for each candidate, from team leads, managers, and directors, up to and including several VPs. We’re genuinely hopeful when someone comes in, and when it doesn’t work out, we’re bummed.

Sitting through an interview with a candidate who can’t work through basic logic, doesn’t have a firm grasp on the syntax of her or his “favorite” language, and slowly, laboriously scratches out code that makes no sense whatsoever, is a kind of existential hell. The urge to help while knowing you can’t, the waste of time, the knowledge that candidates frequently have no idea they’re doing poorly, even while writing code that’s five kinds of crazy, are all tremendously frustrating.

On the other hand, a great interview with a candidate who knows exactly what she or he’s doing is a joy. Getting through the fizzbuzz equivalents quickly and having time to focus on the fun, difficult problems – talking about different approaches, seeing the code flow – this is a great time! I’m always a little embarrassed with these candidates, since I worry that my problems are too easy, and at the end of the interview I always joke that they should ask me to write some code (no one has, yet, but I think it would be fun :).

False negatives

A lot has been written about the preferability of false negatives over false positives, and I’m not going to rehash that here. But let’s go back to that hypothetical candidate from above. They’re graduating from MIT, with internships at Facebook, Google, and Amazon, and offers from a half dozen top companies. Then they bomb the technical interview. How can this be? Some possibilities:

• Fail

Your interview process just doesn’t work.

• Social proof

They got lucky with the interview on the first internship, or the first company had low expectations for a freshman. The second company saw the first internship and decided to give them a chance, regardless of their interview performance. By the time they got to the third internship, no one was willing to buck the trend.

• Cheating

Interview questions sometimes show up on Glassdoor and elsewhere. It isn’t wrong to research common interview questions (though it’s more than a little questionable to target the company you’re going to be interviewing at), but you should never pretend not to know a question if you’ve seen it before. Always let the interviewer know you’ve seen the question, and let them move on to something else. However, every year we get some people whom we suspect knew the questions ahead of time.

• Bad day

Sometimes, a candidate just has a bad day. Maybe their plane got in late, they couldn’t sleep, they ate something bad, they’re especially nervous for some reason… No way to know, but also not much you can do to counteract this.

Sometimes, one or more of the interviewers will have the sense that the process has failed the interviewee. These are always idiosyncratic cases, and we do our best to figure out a different way to get the information we need. Sometimes this means offering a contract-to-perm position, or an internship-to-full time position. These have typically worked out well, and I’d like to find ways to do more of them.

Coding as Jazz Piano

Q: What do you call the person who graduates at the bottom of their class in med school?
A: “Doctor”

When people ask me why it’s so hard to find great candidates, I like to compare it to finding great jazz pianists. Jazz piano is hard – everyone gets that at a gut level. There are a lot of bad, mediocre, and good jazz pianists out there, but if you’re trying to hire someone great, you’re going to have to look for a long time. You don’t want to go out to a jazz club and listen to someone who’s good – you want great. And at the end of the day, it doesn’t matter if someone went to a great school, or practiced every day, or played in the school band. All that matters is whether or not they’re great.

People seem to believe that graduating from a good school with a CS degree will qualify someone for an engineering job. Which makes logical sense, but isn’t true. Learning to code is hard, and takes a lot of work, most of it outside the classroom. Some people will never learn to code. Education and years of industry experience are no guarantee of ability. So sure, a lot of people audition, but few make the cut.

I like the jazz pianist metaphor because people get it. There’s a certain requisite obsession required to become a great coder, a lot of late nights and personal projects, a lot of playing around, and a love of the code. You can’t get there solely through good grades and assignments handed in on time, any more than you can become a great pianist by going to classes and preparing for recitals. You have to do something wrong a dozen times to know the different flavors of wrongness, and know what it looks like in each uniquely messed up approach. You have to start your own band a couple of times, gig a lot, try stupid stuff until something works. Jazz piano and coding are hard.

Until they aren’t.

The metaphor also gets at the other side of the problem, the curse of knowledge. For people who are completely comfortable coding, who know their theory inside and out, and are native speakers of code, it’s very hard to understand how people can’t understand basic, critical concepts (like pointers, flow of control, etc.). Not that these people are always right, or can answer any coding question perfectly, but at some point the code becomes an extension of one’s thoughts, and all that remains is figuring out the actual problem.

Sometimes we get it wrong

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.
– John Wanamaker

My peers and I have spent a lot of time interviewing candidates at all different levels. We know each others’ questions, approaches, and difficulty levels. We’ve talked through many different candidates, and have a language for describing their performance and characteristics. We know who focuses on what, who looks to identify bad candidates quickly, who looks to uncover greatness. We have well-defined axes with which to evaluate candidates.

And, of course, sometimes we get it wrong. But while I’m sure that we sometimes miss out on great hires, I’m equally certain that we’ve dodged some pretty big bullets. Unfortunately, there’s no way to know which is which, and at the end of the day, we can only do as best we can.

Follow me on Twitter