Reducing significant inherited technical debt

Remote
Underperformance
Legitimacy
Firing
Dev Processes
Personal growth
Feedback
Coaching / Training / Mentorship
High Performers
Scaling Team
Managing Up
Productivity

16 April, 2018

David inherited a team with significant technical debt, accrued over years fighting fires for the CEO of the company. Through significant post-morteming and process changes, the team significantly reduced this debt and changed from a reactive culture to a proactive one.

Problem

When inheriting the team and codebase of a start-up with a product that has going through many transformations to find product-market fit, I discovered quickly that there was a ton of technical debt, including silent failures, huge performance problems, outages, and regressions. The CEO was very used to getting what he needed at a moment's notice, and this merely added to the problem.

Actions taken

The most important immediate action I took was to give visibility to the CEO of the culture that had been created and the tangible, negative impact of him being used to getting what he wanted at a moment's notice. By educating him and the rest of the executive team in a very tangible way, I was able to get their commitment to change their behavior by changing the process of how things were requested and serving as the "shield" for the team whenever the CEO wanted something new. With that fire out, I focused on giving into the codebase deeply, working with the team to identify new or existing technologies that could be used to improve the current situation. From this, we created a "Technical Debt Roadmap" which identified all of the things we could do to make the situation better, prioritized based on impact and time-to-implement. We then committed about 20% of all working time to items on this roadmap and over many months focused on getting this done. Aside from this, we committed to post-mortem every issue that came up, particularly regressions and outages. We also made it a requirement that no post-mortem could be done without requiring a process change as part of the post-mortem itself -- it was never enough to say "Person X should do Y differently next time", particularly because humans are imperfect. Instead, we would require adding a process, whether it's an automated test, a new documented process to follow, or something else. While it may seem like this could lead to a lot of bureaucracy, in fact it only led to preventing the same mistakes from happening again, and it kept us honest with exactly what was required to maintain all of our existing features and functionality.

Lessons learned

When inheriting a new team or codebase, it can be easy to focus on the "symptoms" of technical debt, but if the focus is on fixing these symptoms without addressing the core problem of how culturally the technical debt is accrued, you will only ensure that this debt will come back overtime. Unfortunately, difficult conversations are often necessary, as it may require behavior change of very senior members of the organization to change some of these bad habits. In addition, it's important to find a way to still innovate quickly and accrue some technical debt along the way if it's what is in the best financial interest of the company.


Related stories

Pivot to Transform Failures into Opportunities
30 May

Marc LeBrun, VP of Engineering at Flow Kana and a co-creator of the Apple Mac, recalls how one of his early prototypes failed to meet the original intent but how that failure turned into an unanticipated opportunity.

Productivity
Impact
Product
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Managing Different Personality Types: No “Golden Rule”!
30 May

Marc LeBrun, VP of Engineering at Flow Kana and a co-creator of the Apple Mac, delves into the importance of understanding different personality types in the workplace and explains why the traditional Golden Rule -- treat others as you want to be treated -- doesn’t always apply.

Leadership
Internal Communication
Coaching / Training / Mentorship
Team reaction
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Leveraging the Team to Salvage a Frustrated High-Performer
24 May

Marc LeBrun, VP Engineering at Flow Kana, shows the value in establishing a collaborative relationship with a withdrawn but highly-capable employee. We can then use that bridge to draw the person back into the team and elevate everyone’s performance.

High Performers
Internal Communication
One-on-one
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Remote Work in Times of the Coronavirus Pandemic
21 May

Steve Hoeg, Director of Engineering at Faire, provides a detailed list of instructions that will make your remote work in times of coronavirus pandemics more efficient and productive.

Remote
Steve Hoeg

Steve Hoeg

Director of Engineering at Faire

Aligning an Ineffective Team to Realize Product Potential
21 May

Kowsheek Mahmood, Principal and CTO at ArchetypeTech, explains how he adapted an ineffective team by determining and implementing team-evaluation processes to better align the team on product delivery.

Team reaction
Managing Expectations
Underperformance
Firing
Internal Communication
Productivity
Reorganization
Kowsheek Mahmood

Kowsheek Mahmood

Principal & CTO at ArchetypeTech

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.