A Thorough Methodology for Administering Consistent Interviews

Problem

"There are multiple parts to the hiring pipeline. I find two areas to be the most challenging parts. The first is how to have consistent interviews and the second is setting in place a solid interview process. Yet, I have implemented methods that cater to both of these issues as well as making it essential that the candidate has a positive experience through the process."

Actions taken

"I have a philosophy about hiring that for some may sound counter-intuitive. My particular take is that the interview process is not actually about hiring but about building relationships and ensuring that candidates have a positive experience with your brand. One outcome of the interviewing process, though, is that you get to hire people who are exceptional and do well during the interviews."

"In order for people to have this positive experience that I speak about, you need to make sure that your interviewers are trained in two areas: One, how you want your company to be represented to the candidate; and two legalities - making sure that anything that is discussed is acceptable during an interview process."

"I have had success in ensuring that interviews are consistent by placing both a junior and a senior interviewer in each interview. One of the advantages of partnering interviewers is that regardless of a candidate's level I can get varying perspectives on potential compatibility in the workplace. For example, when the candidate is a junior engineer I can ask the junior interviewer if this is somebody that he/she would like to work with. For a manager interviewer I would ask if this person would be valuable to the team, and that he/she would be okay if that person had to report to them. In a different circumstance, if the candidate is a senior level engineer, I would ask the junior interviewer if the candidate taught them anything as they went along, if they felt mentored, and then use the senior interviewer for vetting that experience."

"During each phase of the interview process I coach and train my interviewers to maintain a tone of complete positivity. Thus, if somebody is struggling through a question, I advise my interviewers to help them walk through the problem and answer. This is so that the candidate doesn't get frustrated and can maintain a level head. It's also a great way for them to feel the collaborative environment of our company. Even if we are walking the candidate through things, giving them plenty of hints, and pulling them too much through the interview, the candidate, him or herself, is still going to have a positive experience - we probably just won't go for the hire in that case. It's really about building out the culture through what happens within the microcosm of the interview."

"The interview process itself is structured in a way to keep it consistent across candidates. There are multiple steps that each candidate has to pass through.

• Code Pairing: This is a collaborative session of light algorithm solving. It's three people in the room - the candidate and the two interviewers - working together to solve a problem. It's not just about the interviewers watching the candidate work but making sure that they know how to solve problems when others are involved.

• Whiteboarding: This varies depending on the role. For front-end engineers we are drawing rectangles and talking about a website and CSS. If a back-end engineer is being interviewed then we might actually do some code on the board, or talk about general database structure and setup. If the candidate is a more senior engineer we might discuss distributed systems, architectures, and have them brainstorm ideas as we go along.

• Managerial Interview: Again, this is dependent on what level the candidate is. The point being that at this final stage a higher level member of our current organization goes in and speaks with the person in order to vet for him/her. If it is a junior-senior candidate I usually have the hiring manager go in. If the candidate has applied to be a manager or for a director-level position than I will usually ask the VP for fifteen minutes of his time."

"In every round it is incredibly important to keep things consistent, warm, and friendly. That way even though you may not hire the candidate, he/she will go out and talk to their friends and share with them the great experience that they just had. The interview process is a way to build up the tech culture of your company and to exhibit a positive perspective from the outside."

"After the interview process you want to have a system in place to gather information about how well the interview went from the interviewers point of view. This is very important in the long run. I have implemented a standardized form that I require every interviewer to fill out. It contains a series of 11 criteria that is ranked from 1-5, from not sufficient to highly sufficient. There's a space to leave notes, though it is rare that I actually read through them unless something is borderline. The final question on the sheet, though, is 'Do you want to work with this person? Yes or No?' The answer to this question trumps the total of the score."

"With every round of candidates that goes through the interview process I will get aggregates for those scores and observe where people are trending based on that round. I also look at the outliers on either end, especially those who do poorly because then I want to know how someone was brought in for an interview if they weren't actually ready for that position. Was it my fault, a fault with my process, or maybe my interviewers were being too aggressive? This survey can answer that."

"After you build out a library of these forms you might see that the ratings scale fairly high or low overall. You can also compare an interviewers' scores over time. This will give you solid quantifiable data on where the scores stand and the proper ways in which for you to evaluate them. For example, if the mean score of the library is 20 out of 50 and a candidate scores a 19, then you probably want to bring this person in for round two, hire them, and get an offer ready. While they weren't even close to a high score, relatively speaking the score ranked high compared to the average of your interviewers. Thus, you can fairly judge the score against the average rather than on the highest achievable value."

"One of the final steps that we do in the interview process is a retrospective. It's the same idea as Scrum, since we are dealing with hiring Sprints. We always get together and review what went well in the interviews, what didn't go well, starts, stops, and continues. Having this as part of the process allows us to continually evolve and change in order to reflect the organization."

Lessons learned

• "Be sure that you always have individuals in the interview room who are trained and represent your brand well. If you have aggressive interviewers than your brand is going to come off as aggressive and unwelcoming. This will surely have a negative impact on your company in the short term and in the long run."

• "Make sure that the questions reflect the type of work being asked for and that the questions are consistent. It is fine to change questions from season to season but you want to ensure that you are vetting for the same things and that they make sense. You can revise items if they are not working, but if they are then make sure there is consistency there."

• "Be sure to get some type of standardized intake that you can refer back to later. You want to be able to see historic trends of how interviews have gone, as well as how interviewers have ranked candidates. This is so you are able to make judgements and assessments for the future regarding the type of people that you want to bring in, along with the skills you care (or don't care) about."