Back to resources

Avoid Failure: Know Your Hypotheses

Managing Expectations

27 January, 2021

Michael Smith
Michael Smith

VP of Innovation at Omnipresent

Michael Smith, VP of Innovation at Hadean Supercomputing, explains how to avoid failure by writing your hypotheses down and testing them.

Problem

For Project 1, we were on our 4th UI refresh of the social media app, without rigorously testing it with the market. We felt sure that this new design was better than before. The CEO, who was an ex-designer, kept churning through new designs and we were coding but not shipping.

In Project 2, working this time as an engineer, we felt sure that redesigning this component would greatly speed up the system and get us more users. After a huge redesign and cost, we got exactly zero more users.

For a digital cinema compression and projection system - Project 3 - the technology was superior but the system did not fit into the users' workflow. A technologically inferior product was adopted by the market, and our products were mothballed.

I lived through each of the instances above as an engineer, all of which had product or project management. Each project ended in failure. The hypotheses that "we felt sure" about were not data-driven.

 

These failures led me to one of my major learnings that I took to heart as a product manager: understand the hypotheses you make about your product and test them.

Actions taken

As a product manager, I attempt to understand the assumptions on all projects, so we don't trip over them. Do we truly understand the users' desires for the app without talking to them? Is responsiveness useful to users, and have we measured the performance and its impact on usability? Are fantastic quality and compression ratios the most important, when the users' workflow affect their adoption even more?

As a PM, ask those questions that might be blind spots that could trip up the product. Always understand what's driving what you're building and the data that underpins your plan. What assumptions are you making that impact the user and business? It's very common for group-think and biases to action to create unspoken assumptions - basically unproven hypotheses - and these can sink your projects after you've already sunk time and money into them.

Lessons learned

As a PM, you should be able to say "we're implementing this because we got this data from talking to users and it looks like our hypothesis is true". Do you have enough data to believe that underlying hypothesis? Do you need to put more discovery or testing in your plan before implementation (a little goes a long way)? An early conversation in the project is low risk and low controversy yet is highly effective and definitely less costly than missing an incorrect assumption. Asking for complete certainty for a project is prohibitive, but double-checking your fundamental assumptions before major development will give the feature much higher odds of achieving the expected outcomes.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

The Optimization and Organization of Large Scale Demand

4 May

Kamal Qadri, Senior Manager at FICO, drives the importance of setting expectations when optimizing large-scale requirements.

Managing Expectations
Delegate
Team Processes
Prioritization
Kamal Qadri

Kamal Qadri

Head of Software Quality Assurance at FICO

The Necessary Structures of Time Management

14 April

Suryakant Mutnal, Engineering Manager at PayPal, discusses the importance of time management and the necessary structures in order to create internal consistency.

Goal Setting
Managing Expectations
Remote
Deadlines
Productivity
Roadmap
Prioritization
Performance
Suryakant Mutnal

Suryakant Mutnal

Engineering manager at PayPal

Navigating Your Role Change: From IC to Engineering Manager

13 April

Anuj Vatsa, Engineering Manager at Carta, describes his journey of becoming an Engineering Manager and shares some tips for easing into this new role.

Managing Expectations
Handling Promotion
Personal Growth
Leadership
Feedback
Career Path
New Manager
Anuj Vatsa

Anuj Vatsa

Engineering Manager at Carta

Bring It All Together: How to Implement Processes in a Nutshell

17 March

Dhananjay Joglekar, Head of Consumer and Channel Growth at Craft.co., shares how he involved all the stakeholders who felt the impact of changes in the organization.

Managing Up
Managing Expectations
Cross-Functional Collaboration
Dhananjay Joglekar

Dhananjay Joglekar

Head of Consumer and Channel Growth at craft.co

What Challenges Do AI Startups Face?

8 February

Adhiguna Mahendra, Chief AI and Product at nodeflux, decodes his ambitions on combating the challenges of the AI and ML system in a fast-growing startup.

Customers
Managing Expectations
Feedback
Ownership
Motivation
Strategy
Performance
Adhiguna Mahendra

Adhiguna Mahendra

Chief AI and Product at nodeflux

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.