Table of Contents
Interviewing is far from perfect. Any employers’ goal is to tighten the correlation between interview success and job performance. I call this discrepancy the ‘interview gap’: the set of all work functions where an interview is a poor predictor.
Gaining alignment in any kind of interview is difficult. As of now, most of my experience comes from recruiting for undergraduate clubs for student designers, developers, and ‘product managers’ 1. I doubt these roles are as competitive as they are at Stripe, but in smaller pools, you get to assess candidates in ways that are not possible at scale.
This post is a collection of thoughts on problems with the technical interviewing and application processes I ran at Hack4Impact and Illinois Labs.
Problems With Hiring
Long term vs Short term Problems
Interviews can only test how individuals solve problems in the short term, but are accepted as a predictor of long term problem solving ability in lieu of experience. The closest approximation possible is taking a problem directly the job and facilitating a candidate’s approach. Here, the gap still persists as a time limit alongside unfamiliarity which can hurt candidates, yet has little affect on job performance. Inversely, some candidates may be great at debugging code academically, but fail to design a new system.
Experience Bias
Generally, a poorer interview is passable alongside demonstrated experience. An issue we face by having a candidate pool of undergraduates is a lack of experience. Candidate years typically skew right, with a larger amount of underclassmen applying. A lack of experience in our applicants can make some excelling individuals shine, but will generally shift more weight to an interview. Experience bias puts leverage on the interview gap.
Project Assessment
Projects are an incredibly difficult way to assess an applicant’s technical skills. There is no value in rewarding applicants for having cool project ideas. Ideas are cheap, implementation is not. Following this line of thinking, we should probably read their code. Here we run into our next problem: there is no timely, or meaningful way to assess implementation without understanding the product decisions made. This rabbit hole is not worth exploring. I recommend using projects as a surface level heuristic for capacity to learn. Instead of trying to figure out how “good” a project is, see what types of technologies this candidate is picking up, how fast they are doing it, and if they can briefly explain their choices.
What to do
Given these limitations, what actually works?
Some signals I’ve found are: great personal websites, having strong opinions, and just being enthusiastic about the work in conversation 2. What you are really trying to figure out is, respectively: attention to detail, deep engagement with work, and ability to contribute to a team.
Interviews should feel more like technical conversations than interrogations. The best signal comes from learning how someone thinks through a problem they have already solved.
I’ve been interviewed this way at startups: a conversation about something I’ve built and work through the decisions made. Even if the interviewee hasn’t built a software product (or any product!), they should show they have gone deep. Having tried to fake it, I’ve found this interview is pretty hard to brute force as it lets one probe decision-making, not just outcomes.
I think it’s hard to recognize great talent (besides referrals) without a good amount of experience with the role or profile. Having taste essential. The interview gap will always exist, but supplementing with other signals is a good way to fill it.
Footnotes
-
We call them ‘product managers’, but they are really just project and people managers. I think conflating the two roles is probably a mistake, but it works when you want a no latency between scoping, delegation, and delivery. ↩
-
It’s hard to think about interviewing without considering the meta. There are certainly people who are just good at “interviewing”, but not fun to work with. After a while of either interviewing or talking to the candidate it’s pretty obvious who is who. ↩