How I Improved My Company’s Hiring Process

Problem

Around three years ago, I was working as an Engineering Director at Betterment. At the time, I was in tasked with leading our consumer-facing engineering organization. My team was around 20 engineers, and while we wanted to continue growing the team, we weren't concerned with massive growth. However, I started to notice a problem with our interview process - our acceptance rate was quite low. We would find people and would want to hire, but they would turn us down. Another issue was the recurring difficulty for our hiring managers to make decisions. We weren't confident in our assessment of people, so it was difficult to determine who we should and shouldn't give offers to.

Actions taken

At the time we had fairly standard interview process, the candidate would go through a phone screening call that would involve writing some code, and we would then bring them on-site over the course of two separate half days. Our technical vetting focused on writing some code to solve algorithmic and data structures problems on a whiteboard. It was clear to me that we had an opportunity to improve our hiring process to increase our confidence in hiring decisions and drive up offer acceptance rates. I also knew that in the future I would be delegating the responsibility of making these hiring decisions, and I wanted the process to be tighter before I proceeded with that delegation. I started shadowing the process to more fully understand what was happening and went out of my way to talk to candidates who were turning us down to gather as much feedback from them as I could. I met internally with people on the hiring team and with recent hires who who had recently been through the process. It was a little tricky to ask newly hired people about the downsides of our interview process, so I was careful to build a little bit of a rapport to provide them with the psychological safety to provide candid feedback. We also focussed on diversity and inclusion. There was a group of engineers who got together to talk regularly about diversity, and I sat down with them to talk about what we could do better. One of the more revealing questions I asked them was, "what could we have better communicated during the interview process that would've made your decision to join Betterment a no brainer?" Next, I started working with our hiring team to iterate and make changes. One of the things we changed was how we technically vetted candidates. We moved from the whiteboard to peer-programming on a laptop. We looked to put people in the most natural setting possible so they could really show us what they could do, and developed a problem that started out simple but became increasingly complex. Rather than telling us whether someone had passed or failed, this method allowed us to see exactly how much someone knew. In addition, we split up the problem into separate parts with separate interviewers, so we could see how the candidate transitioned between working with different folks on a continuous problem. We started using an Ask Me Anything session. Candidates get to meet some engineers who aren't involved in grading them, and get to ask questions about the company and its culture. This allows them to ask questions they might be otherwise hesitant to raise. We also changed the structure of the interview process so it took place over just one day, as people found it much easier to schedule. However, we iterated on this a few times and now we give candidates a choice of whether they prefer the interview to take place over one or two days. To decide on which candidates to offer jobs to, we ask everyone involved in the interviews to fill out scorecards in Greenhouse. We try to balance the subjective and objective nature of evaluating candidates by asking interviewers to write free-form responses to targeted questions, and we put the final decision making power into the hands of the hiring manager. If we decide to make an offer we often try to make it within 24 hours of a candidate being onsite.

Lessons learned

Our interviewing process used to use one of the most common interview practices - whiteboarding computer science problems. However, I came to realize that just because everyone does something doesn't mean it's the right fit for your company, and it may be pretty mediocre. It's okay to try different approaches. In addition, the interview process isn't solely about interviewing candidates. It can also give your company a competitive advantage. We survey people to find out why people join, and our hiring process has become a leading reason for why people join Betterment. Over time, our acceptance rate more than doubled. We have since replicated this process for other engineering functions and have found similar success in these functions. In terms of certainty, I definitely feel a lot more confident in making hiring decisions because of the information I am now getting out of the process. This is also true for other hiring managers. - we surveyed the hiring team, including the hiring managers, and 100 percent of people reported that they were happy with the improvements to our process after the changes, and the number of people who felt confident making decisions also improved.