How to Create an Unbiased Hiring Rubric for your Product Team

Hiring
Internal Communication
Product Team

28 November, 2018

Jack Hirsch, Director of Product Management at Box, offers insight on the success of his six-dimensional hiring rubric against the ever-evolving growth patterns of the company.

Problem

Recruiting product managers is no easy task. Contrary to popular belief, a process becomes harder to define as the organization grows and the product team increases its coverage area. Your hiring strategy should evolve with your company. This challenge arose for me when I joined Box and realized the hiring process hadn't been revisited in two years. With my impending need to hire product managers, I was compelled to address this problem.

Actions taken

When attacking challenges in existing systems, I like to take a four-step approach:

  • Familiarize myself with the historical context of the situation
  • Understand the current state
  • Define what an ideal end-state or outcome
  • Start working incrementally toward that ideal outcome As I began the discovery process, I found that the company's hiring process was reliant on a leveling rubric and institutional knowledge as opposed to an objective hiring rubric. I interviewed engineering managers, product managers, and product leaders in an effort to understand where the existing approach might be falling short of expectations. What I found was that Ïwe had a loosely agreed upon, but undocumented set of traits we believed were associated with successful product managers at Box. Therefore, our ideal end-state appeared to be a consolidated list of traits along with questions and evaluation criteria. Documenting everything along the way, I mutated the responses I received in those internal interviews into a six-dimensional rubric of (mostly) mutually exclusive traits. By identifying those traits, we'd be able to map them directly to interview questions. These are the required traits of successful PMs at Box (in no particular order):
  • Strategic thinkers and structured communicators
  • Customer focused
  • Technically adept
  • Culturally additive with the right attitude
  • Made of leadership material
  • Possess the PM skill set (execution, time and priority management) We then started to build a list of interview questions and evaluation criteria. You can imagine each of these traits as one axis in a two-axis rubric. These traits can be plotted vertically to define the dimensions of successful product owners. Horizontally, you can plot questions along with an answer rubric to delineate seniority and answer strength. With the help of our other product managers, we assessed whether our current questions fit into these areas, and we developed new ones to fill in gaps in our evaluation criteria. Interviewing has since become more structured, more objective, easier and more successful.

Lessons learned

  • Define specific traits you're looking for in a product manager. This will allow for more objective evaluation during an interview.
  • A strong rubric enables you to delegate the responsibility of evaluating specific traits to interviewers on your the panel. Those interviewers can then have very targeted conversations to evaluate their assigned traits.
  • With the debatable exception of technical aptitude for some PM roles, we found these six traits to be cross-cutting and generally universally applicable to all PM roles currently at Box.
  • The dimensions of candidate evaluation should be as mutually exclusive as possible. You do not want ambiguity in what you are evaluating as an interviewer or else you risk introducing unnecessary subjectivity.
  • The rubric applies structure to our hiring process at Box in two ways.
  • Tactically, we now have a customized and objective evaluation rubric. Following an interview, the interviewers are given the option to fill in scores on all six of these traits and explain how well each candidate exemplified them.
  • We can come together as a hiring panel and paint a full picture of the candidate across all the dimensions that we know to make successful product managers at Box. This enables us to make more objective decisions about whether or not this person would be successful.
  • I were to go back, I would click a step deeper into the structural process of the interview. i.e. length and layout of our interview panels. We will do this soon and it will be an important step in maximizing our ability to evaluate PM candidates. -

Related stories

Managing Different Personality Types: No “Golden Rule”!
30 May

Marc LeBrun, VP of Engineering at Flow Kana and a co-creator of the Apple Mac, delves into the importance of understanding different personality types in the workplace and explains why the traditional Golden Rule -- treat others as you want to be treated -- doesn’t always apply.

Leadership
Internal Communication
Coaching / Training / Mentorship
Team reaction
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Developing High Performing Teams: How To Transform a Software Developer Into a Product Engineer
30 May

Alessandro Pintaudi, Product Management Director at Payfit, comes up with an exciting proposal of transforming software developers into product engineers by establishing cross-functional context analysis and shared objectives.

Cross-functional collaboration
Product Team
Collaboration
Alessandro Pintaudi

Alessandro Pintaudi

Product Management Director at PayFit

Leveraging the Team to Salvage a Frustrated High-Performer
24 May

Marc LeBrun, VP Engineering at Flow Kana, shows the value in establishing a collaborative relationship with a withdrawn but highly-capable employee. We can then use that bridge to draw the person back into the team and elevate everyone’s performance.

High Performers
Internal Communication
One-on-one
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Aligning an Ineffective Team to Realize Product Potential
21 May

Kowsheek Mahmood, Principal and CTO at ArchetypeTech, explains how he adapted an ineffective team by determining and implementing team-evaluation processes to better align the team on product delivery.

Team reaction
Managing Expectations
Underperformance
Firing
Internal Communication
Productivity
Reorganization
Kowsheek Mahmood

Kowsheek Mahmood

Principal & CTO at ArchetypeTech

Resolving Remote Communication Issues
19 May

Viacheslav Bessonov, Chief Technology Officer at Algalon Capital, outlines how he improved communication internally - with his fully distributed team, while also improving communication with the customer - located across various time zones.

Remote
Internal Communication
Users Feedback
Viacheslav Bessonov

Viacheslav Bessonov

Chief Technology Officer at Algalon Capital

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.