How Metrics Can Help to “Disagree and Commit”

Company Culture
Leadership
Productivity
Reorganization
Sharing the vision

31 March, 2020

Matt Nemenman, Director of Engineering at Lyft, shares how he relied on data to introduce strategic changes that challenged the company status quo.

Problem

On the path to building the best online apartment rental marketplace, my company decided to differentiate from the competition by focusing on a five-star renter experience. We introduced a customer service call center: whenever a renter was trying to contact a property they would actually be connected to our customer service representative who would help them: answer questions, check availability, and eventually help get into the transaction and sign the lease. We hired 70+ call center reps, my team built custom software to make them efficient, and we were even searching for real estate in cheaper parts of the country to grow it further.
 

However, to someone with a technology background like myself, this strategy didn’t make sense for a few reasons. First, most modern companies tend to solve the problems in a scalable way with technology, while we were solving it with people - the most expensive asset. Second, our call center employees who were based in San Francisco couldn’t really answer real questions about the apartment community in, let’s say, Texas, and would have to forward the call to the property. Finally, cultural issues also started to emerge: a lean company was becoming bloated with a fast-growing headcount of call center employees, with all the company systems (HR, payroll, IT, security, etc) being stressed under this growth.
 

Actions taken

As a head of engineering initially, I took a “disagree and commit” stance and diligently executed on a strategy laid out by the leadership team. However, I also couldn’t shed the feeling that we are not doing the right thing. I knew that in order to bring up this existential question with the leadership team, I needed a lot more than just my opinion -- I needed to find data that corroborates it. Inspired by a book Freakonomics I started looking for a natural A/B test in our existing corpus of data. I spent a lot of time -- mostly after hours -- digging through the various metrics about our call center interaction in our data warehouse and trying to come up with a metric that would quantify the impact of our customer service center on the whole business model. My SQL skills were getting better day after day…
 

Finally, I had my “Aha!” moment. I was able to find a small subset of properties where renters were bypassing the call center. It allowed me to build a dashboard that compared transaction rates across these two classes of properties. It turned out that the transaction rate through the call center was not better, but actually slightly worse than for this set of properties. Data seemed to point out that the extra layer of interaction we forced on our renters was not helpful for the business. It was a start!
 

That was an interesting data set that I was able to bring to our leadership team and start a conversation. It was no longer just an opinion, it is a lot more difficult to argue with actual data. Collectively we designed several more A/B tests that allowed us to segment properties better, and understand how we can actually help renter where needed, and remove extra friction where not. Eventually, the call center program was scaled down a lot: we no longer needed large real estate, or custom call center software to maintain. The impact on the company path to the profitability of this change can not be understated.
 

Lessons learned

  • Data wins arguments: I was really glad that I was able to pivot company strategy with the right metrics while maintaining or improving relationships with other leaders in the company.
  • As a head of engineering, I could carry on executing the roadmap. However, I learned that leadership spans beyond your department. Engineering should play an important part in setting business and product vision. Without that, we risk working tirelessly on something that should not be done at all.
  • I learned the true meaning of “disagree and commit”. If you passionately believe that the company direction is wrong, there are many constructive ways to bring it up (using data is one of them). You do not need to “forever hold your peace”.

Related stories

Pivot to Transform Failures into Opportunities
30 May

Marc LeBrun, VP of Engineering at Flow Kana and a co-creator of the Apple Mac, recalls how one of his early prototypes failed to meet the original intent but how that failure turned into an unanticipated opportunity.

Productivity
Impact
Product
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Managing Different Personality Types: No “Golden Rule”!
30 May

Marc LeBrun, VP of Engineering at Flow Kana and a co-creator of the Apple Mac, delves into the importance of understanding different personality types in the workplace and explains why the traditional Golden Rule -- treat others as you want to be treated -- doesn’t always apply.

Leadership
Internal Communication
Coaching / Training / Mentorship
Team reaction
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Going Low-Tech to Uplevel Agile Practices
24 May

Marc LeBrun, VP Engineering at Flow Kana, describes how he empowered teams to make decisions and uplevel their process by adopting a low-tech approach to agile practices.

Team reaction
Leadership
Impact
Agile / Scrum
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Aligning an Ineffective Team to Realize Product Potential
21 May

Kowsheek Mahmood, Principal and CTO at ArchetypeTech, explains how he adapted an ineffective team by determining and implementing team-evaluation processes to better align the team on product delivery.

Team reaction
Managing Expectations
Underperformance
Firing
Internal Communication
Productivity
Reorganization
Kowsheek Mahmood

Kowsheek Mahmood

Principal & CTO at ArchetypeTech

Building Customer Driven Backlog Across Several Component Teams
16 May

Maria Petrova, Principal Product Manager at Zalando describes how she built an organizational design from product and stack in a way that tech and product effectively function in a collaborative environment.

Product
Users Feedback
Reorganization
Maria Petrova

Maria Petrova

Principal Product Manager at Zalando

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.