How to Create an Unbiased Hiring Rubric for your Product Team

Problem

Recruiting product managers is no easy task. Contrary to popular belief, a process becomes harder to define as the organization grows and the product team increases its coverage area. Your hiring strategy should evolve with your company. This challenge arose for me when I joined Box and realized the hiring process hadn't been revisited in two years. With my impending need to hire product managers, I was compelled to address this problem.

Actions taken

When attacking challenges in existing systems, I like to take a four-step approach:

• Familiarize myself with the historical context of the situation
• Understand the current state
• Define what an ideal end-state or outcome
• Start working incrementally toward that ideal outcome As I began the discovery process, I found that the company's hiring process was reliant on a leveling rubric and institutional knowledge as opposed to an objective hiring rubric. I interviewed engineering managers, product managers, and product leaders in an effort to understand where the existing approach might be falling short of expectations. What I found was that Ïwe had a loosely agreed upon, but undocumented set of traits we believed were associated with successful product managers at Box. Therefore, our ideal end-state appeared to be a consolidated list of traits along with questions and evaluation criteria. Documenting everything along the way, I mutated the responses I received in those internal interviews into a six-dimensional rubric of (mostly) mutually exclusive traits. By identifying those traits, we'd be able to map them directly to interview questions. These are the required traits of successful PMs at Box (in no particular order):
• Strategic thinkers and structured communicators
• Customer focused
• Technically adept
• Culturally additive with the right attitude
• Made of leadership material
• Possess the PM skill set (execution, time and priority management) We then started to build a list of interview questions and evaluation criteria. You can imagine each of these traits as one axis in a two-axis rubric. These traits can be plotted vertically to define the dimensions of successful product owners. Horizontally, you can plot questions along with an answer rubric to delineate seniority and answer strength. With the help of our other product managers, we assessed whether our current questions fit into these areas, and we developed new ones to fill in gaps in our evaluation criteria. Interviewing has since become more structured, more objective, easier and more successful.

Lessons learned

• Define specific traits you're looking for in a product manager. This will allow for more objective evaluation during an interview.
• A strong rubric enables you to delegate the responsibility of evaluating specific traits to interviewers on your the panel. Those interviewers can then have very targeted conversations to evaluate their assigned traits.
• With the debatable exception of technical aptitude for some PM roles, we found these six traits to be cross-cutting and generally universally applicable to all PM roles currently at Box.
• The dimensions of candidate evaluation should be as mutually exclusive as possible. You do not want ambiguity in what you are evaluating as an interviewer or else you risk introducing unnecessary subjectivity.
• The rubric applies structure to our hiring process at Box in two ways.
• Tactically, we now have a customized and objective evaluation rubric. Following an interview, the interviewers are given the option to fill in scores on all six of these traits and explain how well each candidate exemplified them.
• We can come together as a hiring panel and paint a full picture of the candidate across all the dimensions that we know to make successful product managers at Box. This enables us to make more objective decisions about whether or not this person would be successful.
• I were to go back, I would click a step deeper into the structural process of the interview. i.e. length and layout of our interview panels. We will do this soon and it will be an important step in maximizing our ability to evaluate PM candidates. -