Elevate Spring Summit has been announced (Thu, Mar 11th)

🔥


Don't have an account? 

Reducing significant inherited technical debt

High Performers
Managing Up
Scaling Team
Dev Processes
Personal growth
Remote
Underperformance
Legitimacy
Firing
Productivity
Feedback
Coaching / Training / Mentorship

16 April, 2018

David inherited a team with significant technical debt, accrued over years fighting fires for the CEO of the company. Through significant post-morteming and process changes, the team significantly reduced this debt and changed from a reactive culture to a proactive one.

Problem

When inheriting the team and codebase of a start-up with a product that has going through many transformations to find product-market fit, I discovered quickly that there was a ton of technical debt, including silent failures, huge performance problems, outages, and regressions. The CEO was very used to getting what he needed at a moment's notice, and this merely added to the problem.

Actions taken

The most important immediate action I took was to give visibility to the CEO of the culture that had been created and the tangible, negative impact of him being used to getting what he wanted at a moment's notice. By educating him and the rest of the executive team in a very tangible way, I was able to get their commitment to change their behavior by changing the process of how things were requested and serving as the "shield" for the team whenever the CEO wanted something new. With that fire out, I focused on giving into the codebase deeply, working with the team to identify new or existing technologies that could be used to improve the current situation. From this, we created a "Technical Debt Roadmap" which identified all of the things we could do to make the situation better, prioritized based on impact and time-to-implement. We then committed about 20% of all working time to items on this roadmap and over many months focused on getting this done. Aside from this, we committed to post-mortem every issue that came up, particularly regressions and outages. We also made it a requirement that no post-mortem could be done without requiring a process change as part of the post-mortem itself -- it was never enough to say "Person X should do Y differently next time", particularly because humans are imperfect. Instead, we would require adding a process, whether it's an automated test, a new documented process to follow, or something else. While it may seem like this could lead to a lot of bureaucracy, in fact it only led to preventing the same mistakes from happening again, and it kept us honest with exactly what was required to maintain all of our existing features and functionality.

Lessons learned

When inheriting a new team or codebase, it can be easy to focus on the "symptoms" of technical debt, but if the focus is on fixing these symptoms without addressing the core problem of how culturally the technical debt is accrued, you will only ensure that this debt will come back overtime. Unfortunately, difficult conversations are often necessary, as it may require behavior change of very senior members of the organization to change some of these bad habits. In addition, it's important to find a way to still innovate quickly and accrue some technical debt along the way if it's what is in the best financial interest of the company.


Related stories

Learn or Die!
12 January

Marian Kamenistak, VP of Engineering at Mews, outlines key recommendations and resources to make learning an integral part of the engineering role and knowledge sharing across the company a steady practice.

Personal growth
Coaching / Training / Mentorship
Marian Kamenistak

Marian Kamenistak

VP of Engineering at Mews

Horizontally Scaling the Engineering Organization
11 January

Ken Pickering, VP of Engineering at Starburst Data, explains how to structure a product engineering organization to maximize results and address inefficiencies through horizontal scaling.

Scaling Team
Team processes
Ken Pickering

Ken Pickering

VP, Engineering at Starburst Data

Creating an In-House Mentorship Program
11 January

Sabeen Syed, Senior Engineering Manager at HashiCorp, shares her efforts to single-handedly establish an in-house mentorship program.

Coaching / Training / Mentorship
Sabeen Syed

Sabeen Syed

Senior Engineering Manager at HashiCorp

Even High Performers Need Direction and Support
5 January

Dmitry Nekrasovski, Senior Manager of Product Design at HashiCorp, explains how he managed one of his high-performing employees by helping them regain focus, make the right priorities, and prevent burnout.

High Performers
Health / Stress / Burn-Out
Dmitry Nekrasovski

Dmitry Nekrasovski

Senior Manager, Product Design at HashiCorp

How to Level Up Your Design Team
5 January

Dmitry Nekrasovski, Senior Manager of Product Design at HashiCorp, shares how he helped his design team level up by applying a simple, three-step approach.

Leadership
Coaching / Training / Mentorship
Dmitry Nekrasovski

Dmitry Nekrasovski

Senior Manager, Product Design at HashiCorp

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.