Balancing Tech Debt and Feature Development

Problem

We were doing a national rollout and every state required different logic of deployment and different flow of the app. The business wanted to go as fast as possible and get as much reach, as well as legal and infrastructure departments that were pushing for a fast release too. If we didn’t keep up with their pace, we would waste a ton of time and money. The pressure on engineering to hurry up was enormous.

We tried small one-offs with the first three different states and realized that it was going to incur a lot of technical debt and be really complicated unless we would stop for a moment and change how the system worked, make it adaptable for different states and then, roll it out.

Actions taken

We knew we were unable to secure buy-in from the business in the very beginning, so we did the first few states as one-offs. That helped us get a real sense of what types of changes in the system we would have to make.

Then, we started to lobby hard against the business and legal department. On the legal side, we needed them to do research and provide us with documents. On the business side, we wanted them to ensure that everything would be ready for us to put the infrastructure down and roll out in six new states instead of two that we could do with one-offs.

We bartered that deal with anyone involved so that we could take a break from rolling out in order to build infrastructure and consequently, move faster. Luckily, everyone bought into that, we made necessary changes and were able to launch in 20 states in the next couple of months.

The above situation is an example where a large tech debt issue had to be addressed in order to build something the right way. But there are situations, like a more regular cleanup, that never get prioritized and that keep building up over time. Paradoxically, as they build up, it will become harder to get them on the priority list and they get knocked out of a sprint first if something needs to get squeezed.

To make sure that the engineering team would stay happy and be able to work on cleanup, I would build a cleanup into the product. I would layer in a bit of abstraction when pitching to Product -- this thing needs to be done in order for this product to happen -- and stretch it a bit just to get it in there and then deploy it with the product. It will get momentum and emphasis by being part of the product. If you are just trying to sell tech debt cleanup by itself, it would be very difficult.

Lessons learned

• When you need to take a break and do infrastructure and refactoring work, you should focus on selling the speed that you can get on the other side of it and be able to make a tradeoff with slow now, fast later.
• When you need to address a problem of ever-emerging cleanup, you should package up a new product plus tech cleanup into one feature release, so that your cleanup gets pulled along with it.
• Some tech debt can be left to sit aside for a while, but it will nevertheless have a negative impact on your productivity. - Everything that sits somewhere for a long time would be like a bit of sand stuck on your skin -- it won’t bother you but you will constantly feel it. However, it won’t sit still forever; as it grows it can have a negative impact on the morale of the team.
• There is a people side of tech debt. As a leader of an engineering organization, your turnover rate summarizes how well you are doing. Turnover will most likely increase if tech debt remains unaddressed. If tech debt is slowing you down, good engineers leaving will slow you down further.