How to Create an Unbiased Hiring Rubric for your Product Team
28 November, 2018
Recruiting product managers is no easy task. Contrary to popular belief, a process becomes harder to define as the organization grows and the product team increases its coverage area. Your hiring strategy should evolve with your company. This challenge arose for me when I joined Box and realized the hiring process hadn't been revisited in two years. With my impending need to hire product managers, I was compelled to address this problem.
When attacking challenges in existing systems, I like to take a four-step approach:
- Familiarize myself with the historical context of the situation
- Understand the current state
- Define what an ideal end-state or outcome
- Start working incrementally toward that ideal outcome As I began the discovery process, I found that the company's hiring process was reliant on a leveling rubric and institutional knowledge as opposed to an objective hiring rubric. I interviewed engineering managers, product managers, and product leaders in an effort to understand where the existing approach might be falling short of expectations. What I found was that Ïwe had a loosely agreed upon, but undocumented set of traits we believed were associated with successful product managers at Box. Therefore, our ideal end-state appeared to be a consolidated list of traits along with questions and evaluation criteria. Documenting everything along the way, I mutated the responses I received in those internal interviews into a six-dimensional rubric of (mostly) mutually exclusive traits. By identifying those traits, we'd be able to map them directly to interview questions. These are the required traits of successful PMs at Box (in no particular order):
- Strategic thinkers and structured communicators
- Customer focused
- Technically adept
- Culturally additive with the right attitude
- Made of leadership material
- Possess the PM skill set (execution, time and priority management) We then started to build a list of interview questions and evaluation criteria. You can imagine each of these traits as one axis in a two-axis rubric. These traits can be plotted vertically to define the dimensions of successful product owners. Horizontally, you can plot questions along with an answer rubric to delineate seniority and answer strength. With the help of our other product managers, we assessed whether our current questions fit into these areas, and we developed new ones to fill in gaps in our evaluation criteria. Interviewing has since become more structured, more objective, easier and more successful.
- Define specific traits you're looking for in a product manager. This will allow for more objective evaluation during an interview.
- A strong rubric enables you to delegate the responsibility of evaluating specific traits to interviewers on your the panel. Those interviewers can then have very targeted conversations to evaluate their assigned traits.
- With the debatable exception of technical aptitude for some PM roles, we found these six traits to be cross-cutting and generally universally applicable to all PM roles currently at Box.
- The dimensions of candidate evaluation should be as mutually exclusive as possible. You do not want ambiguity in what you are evaluating as an interviewer or else you risk introducing unnecessary subjectivity.
- The rubric applies structure to our hiring process at Box in two ways.
- Tactically, we now have a customized and objective evaluation rubric. Following an interview, the interviewers are given the option to fill in scores on all six of these traits and explain how well each candidate exemplified them.
- We can come together as a hiring panel and paint a full picture of the candidate across all the dimensions that we know to make successful product managers at Box. This enables us to make more objective decisions about whether or not this person would be successful.
- I were to go back, I would click a step deeper into the structural process of the interview. i.e. length and layout of our interview panels. We will do this soon and it will be an important step in maximizing our ability to evaluate PM candidates. -
Marc LeBrun, VP of Engineering at Flow Kana and a co-creator of the Apple Mac, delves into the importance of understanding different personality types in the workplace and explains why the traditional Golden Rule -- treat others as you want to be treated -- doesn’t always apply.
VP Engineering at Flow Kana
Alessandro Pintaudi, Product Management Director at Payfit, comes up with an exciting proposal of transforming software developers into product engineers by establishing cross-functional context analysis and shared objectives.
Product Management Director at PayFit
Marc LeBrun, VP Engineering at Flow Kana, shows the value in establishing a collaborative relationship with a withdrawn but highly-capable employee. We can then use that bridge to draw the person back into the team and elevate everyone’s performance.
VP Engineering at Flow Kana
Kowsheek Mahmood, Principal and CTO at ArchetypeTech, explains how he adapted an ineffective team by determining and implementing team-evaluation processes to better align the team on product delivery.
Principal & CTO at ArchetypeTech
Viacheslav Bessonov, Chief Technology Officer at Algalon Capital, outlines how he improved communication internally - with his fully distributed team, while also improving communication with the customer - located across various time zones.
Chief Technology Officer at Algalon Capital
You're a great engineer.
Become a great engineering leader.
Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.