Delve Into Issues To Find Solutions

Problem

As a staff engineer at Betterment, a lot of the time, my role is spent looking around the organization to identify where there might be teams in need of guidance. Last year, I identified that our "DevOps team" was in over their heads. I couldn't understand why they reacted with such negativity when I asked them to help me make a change to our continuous integration process, so I set up some time to talk with them and understand their situation.

"They were having a hard time keeping folks on the team. The team was formed three years earlier as an ad-hoc team to help help us migrate from on-premise infrastructure to AWS. They achieved a tremendous amount, but when the engineering organization grew from 15 people to almost 80, the solutions they had come up with didn't scale."

Actions taken

The approach we were using for DevOps was essentially internal consulting; we had four people hastily building whatever engineers would ask for. We didn't scale the size of this DevOps team with the engineering organization, so inevitably the DevOps folks became overworked. They felt like they had to make compromised decisions on every task in order get anything done. Combine that with playing whack-a-mole fixing the issues that would crop up with previous, hacky solutions, and we had a recipe for team collapse.

"By sitting down and talking with the folks on the team, I identified that the team was completely underwater and they couldn't manage to keep their team staffed because the work required of them was pretty miserable. We figured out that the biggest contributing factor was that the team had the wrong set of responsibilities and an unsustainable, outdated role for the organization."

Rather than having a team of four people doing all of the building, testing and reliability work for 80 engineers, we needed to invert our approach. We decided to build a suite of tools to enable the rest of the engineering team to do the building, testing, and deploying of their own software. This was a huge lift and was in no way a quick turn-around, but this philosophical shift for the team was incredibly motivating. Now it is exciting to be part of this new team working toward a new mission to build tools to accelerate and standardize the way that our engineering team shipped code.

"I spent the last year reinventing that team and explaining that it wouldn't mean we had to grow the team from four people to 20 in order to maintain productivity. By building software and changing the way that the organization understood the role of the team, I was able to put the team in the incredibly high-leverage position of enabling the rest of engineering to deliver features themselves, flawlessly!"

Lessons learned

When I had a complaint about the DevOps team, I had two choices - I could just whine about it, or I could explore the issue, pitch in, and fix it. I had assumed that our deployment code and continuous integration systems were well-designed like the rest of the software that our engineers wrote every day. But I was wrong; the DevOps team was mostly just gluing things together and plugging holes.

"By having empathetic conversations with the team, I was able to see that they were misunderstood and what they wanted to be doing was different from what they were actually doing. They needed to reset."