A Guide to Hiring Excellent Product Managers

Problem

Because the Product Manager role can be so amorphous many companies don't know how to evaluate what it takes to be a great product manager and instead look for people with domain expertise. Even for me, when I joined Coursera, I thought I needed a PM that really understood education, learning science and how universities work. I was wrong and I learned it the hard way (e.g. by hiring to that profile to no avail). Instead I learned that one needs to come back to the basics and look for PMs that possess the core skills needed to succeed in the highly cross-functional and interdisciplinary role. As a PM you have to understand the business (not just financially, but also strategically), the technology, the market, the internal constituents, the customers - it's almost laughable the many distinct areas a PM needs to understand. And that's what makes hiring for them so hard. How does a hiring manager assess for all of this?

Actions taken

For my PM interviews I give each interviewer an assignment or job. This helps us cover more ground and avoid asking the candidate the same questions in each session. The jobs are to assess: Communication Structured thinking Design thinking Vision setting / strategic thinking [And cultural fit, which I won't address in this framework] Let me unpack each of these. Communication: can the candidate distill complex ideas into simple to understand frameworks? Can they sufficiently set context such that the team is motivated. You are looking for people that can work through context, not control. What I mean is that it's much better for a PM to be able to inspire a team or help them understand the direction and thus allow the team to innovate rather than for a PM to dictate every feature to a team to build.
Things to watch out for - do they: Clearly state their hypotheses and assumptions? Seek to understand by asking clarifying questions? Sample Questions: How would you describe Coursera to a kindergartner or your grandparents? Suppose you were interviewed by a reporter and you needed to tell the story of why Coursera credentials matter. What would you say? Imagine you needed to motivate a university instructor to make their classes available anytime, how would you go about doing so? Structured thinking: Can the candidate evaluate logical reasoning and do they have an excellent command of metrics? Can the candidate logically break down a problem and measure complicated social behaviors? Good answers to all questions show: Passion to solve the problem A sound logical framework for breaking down the problem. A creative use of proxy metrics to estimate the hard-to-measure behavior. Insights into the core problem we're trying to solve Sample Questions: How do you measure 'learning'? Suppose you notice . How do you get to root cause. (e.g. drop in MAU in Summer.) Set a one-year goal for the [insert team]. What three metrics would you create to drive the team in the right direction? Design Thinking: Evaluate raw design sense, ability to iterate and consumer empathy. Does this person understand basic concepts of good design (simplicity, well-reasoned interactions and flows, applies design patterns from other products)? Are they open to feedback and able to learn quickly from feedback? It's important to include 3 stages in whichever question you choose so you have a natural structure to give feedback: Idea generation (does this person have good instincts of what problems to solve?) Wireframing on the board (can they formulate good flows?) Justifying design decisions (can they critique their own designs and express why?) Sample Questions: Design a brand new mobile experience for [insert example]. What features do you think we should build? Draw up wireframes (limit to two screens for time's sake). ; draw another round of wireframes. (note: does not have to be wireframes, can be conceptual) Think of a time that during your project, design and engineering had completely different viewpoints on the right user experience. What did you do and how did it resolve? How do you approach getting user input to your designs, cite examples and how it changed (or you decided not to change it) based on the feedback. What to look for Great answers show good product sense and justification for their ideas and decisions Great wireframes have good justifications for why the flows exist ("This screen needs to have one action because we want the user to focus on completing this task in order to make the rest of the app useful") Great candidates take your feedback to heart and tweak their designs / concepts Vision setting / strategic thinking: Evaluate if the candidate is capable of distilling a problem to its essence and proposing big, bold ideas. You want to weed out those who play it safe and those who are less likely of finding the next big thing. Sample Questions: What are the trends impacting [insert industry]? (list out as many as possible). Pick one or two and be creative about what that means for the industry in 5-10 years time. If time, roll back to what that means for [insert company] in the next 3-6 months. Good answers show: Humility ("I know I don't have all the context; here are questions I would ask...") Logical deduction ("Here are the trends I'm seeing...") General common sense ("People will still wear clothes...") Strong, controversial point of view ("Manufacturing is so cheap that fashion can be tailored to individual tastes...") Understanding of psychological/sociological concepts ("Fashion is a reflection of and a function of social circles") Bad answers: Not understanding the big-picture spirit of the question ("Stores like Nordstroms will be around because they have lots of cash and can stay afloat") Immediately jumping into small features Generally speaking, the more senior a PM candidate the more they need to nail the vision questions; whereas, the more junior candidates need to be proficient in at least one of the areas, but not necessarily in vision setting.

Lessons learned

A good Product Manager is a good Product Manager, regardless of domain. It is more important to hire for the skills that make a good Product Manager rather than hire mediocre PMs with excellent domain knowledge.