My interview coding test
6 December, 2017
When I became a manager a few years ago, I started interviewing candidates. At first, I used to run interviews the way my team lead would run them - with a small coding test that applicants were asked to code in C/C++ with a few gotchas. After a while, I started noticing a problem. This exercise relied a lot on their ability and knowledge of C or C++, which was not primarily what we wanted to test.
I decided to talk with my team members about this and I particularly remember one of them telling me about their perception of the issue - somebody's desire to do math is strongly correlated to being a good developer. It's not about being good at math but rather liking and being interested in math as a proxy for liking logic. From there, and after multiple updates, I built up a new math-based coding test. I start by asking the question: "Do you know the quadratic equation?". Their answer to that says a lot. If people start saying "Argh, I hate math", it's a sign. If they don't know it, I tell them about it, as it's not about them knowing the equation or not. Then, I ask them to code a simple algorithm to solve the function I just gave them, preferably on a whiteboard (for freedom). The language doesn't matter, even pseudo-code is fine. What's particularly interesting about this equation is that it's easy enough to code a simple algorithm yet hard enough so that they will make mistakes (I can only think of 3-4 people who got it right the first time among 200+ interviews). The common mistakes are dividing by zero, not realizing the square root of a negative number is imaginary, and returning multiple numbers. I tell them how many mistakes they have made. "I can see xx mistakes," with no further details about what they are. I then ask "Knowing that, what would you do?". That's when it starts to become interesting. How do they react? Do they ask questions? Give up? Silently stare at their code? Test their own code? Try another method? Any method towards resolution is viewed positively but people giving up obviously fail in my mind.
When hiring, I'm looking a logical frame of mind and math provides a common background to test this out. In my experience, I've never seen anybody that dislikes math AND was a great algorithm programmer. There is no perfect way to evaluate if someone will be a good developer so you must rely on indicators. This test is quite a good one (among others). Moreover, engineers make mistakes all the time but what's critical is how they react to them. That's exactly why I ask "You have xx mistakes, what would you do about it?" rather than "What are those mistakes?" or "How would you fix them?". Here is what I am trying to detect in applicants through this test:
- That they don't give up
- Flexibility on how to solve a problem
- Appreciation for logic
Kevin Perko, Head of Applied Research and Data Science at Scribd, explains how standardizing the interview process -- and particularly the interview questions -- resulted in better hires and skill set fits.
Head of Applied Research / Data Science at Scribd
Matt Pillar, VP of Engineering at OneSignal, shares how he improved the reliability of high scale systems by securing investment in infrastructure and on-call services.
VP Engineering at OneSignal
Matt Pillar, VP of Engineering at OneSignal, shares how he had to abandon a technology investment his team was pursuing that neglected the real customer problems and instead focused on the brilliance of the solution alone.
VP Engineering at OneSignal
Jackson Dowell, Engineering Manager at Asana, explains how he moved his project forward by coming up with a clear model of the system and problem that provided guidance to the team and helped communicate their efforts outward.
Engineering Manager at Asana
Nimrod Perez, CTO and VP of Engineering at Wobi LTD., shares how he successfully implemented a number of changes, including re-writing the entire system, to help the organization accomplish its goals.
CTO and VP of Engineering at Wobi LTD.
You're a great engineer.
Become a great engineering leader.
Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.