Visualizing Your Team’s Engineering Quality
26 July, 2020
I have been managing a team that was working on maintaining and developing a large number of products, some in maintenance, others in active development mode. We had a hard time as a team -- and more specifically I, as a manager -- keeping up with the quality of the engineering work we were delivering, as well as with the quality of our products. We also lacked any framework to define and measure the engineering quality of our team, as well as any process that we could improve by iterating it.
I have decided to adopt a similar approach to Spotify’s squad health check ( https://engineering.atspotify.com/2014/09/16/squad-health-check-model/ ) with regards to engineering quality within the team - a further breakdown of one of the possible team health metrics.
The idea behind this simple framework is as follows:
Have an open discussion and collaboratively define a set of metrics the team cares about in relation to the quality of their work and the end-product. We created a heat map that had products on one side and the engineering quality matrix on the other. We singled out as particularly important things such as testing quality and coverage, monitoring, logging, alerting, runbooks, documentation, correct architectural patterns, etc.
We defined a color-coded metric value for each of our metrics (red, yellow, and green). We have agreed on a three-star (1,2,3 stars) scoring and talked about what defines each scoring.
- A baseline definition gives the metric 2 stars or yellow coding. For example, a baseline for documentation would be -- there is a Wiki page for that product and it explains the basic functionality of each component.
- A lack of the baseline gives the metric 1 star or red coding. For example, there is no monitoring in place for a specific product or is so rudimentary that is negligible.
- A set of qualitative rather than quantitative definitions would give the metric 3 stars or a green coding. That would be our ideal or North Star for our product.
- We set up a recurring bi-weekly meeting with the tech leads, according to the ownership of each tech lead to discuss the newly created heat map and extract AIs for the next sprint. We discussed which areas we wanted to focus on, starting from eliminating areas without a proper quality baseline and moving slowly toward the “green”.
Instead of aiming for perfect accuracy or entirely quantitative data, we were looking for data that would steer us in a good direction and help us track the progress. We first dealt with areas marked red identifying what should be done to make it green. In addition, it helped us to incorporate it into sprints next to the product work.
The heat map provides you as a manager a high-level overview of how your team is doing in terms of the engineering quality. Looking at the product rows will help you identify products that are neglected across many/all metrics. Looking at columns that represent metrics will allow you to identify cross-product issues such as a lack of alerting culture in the team - an insight which you will have to address horizontally as a manager.
Alex Litvak, Engineering Manager II at Uber, explains how he adjusted Spotify’s squad health check to enhance his team’s engineering quality.
Engineering Manager II at Uber
Andrew First, Co-Founder and Chief Technologist at Leanplum, shares how with a focused effort his company succeeded in reducing cloud costs by more than 60 percent in only six months.
Co-founder & Chief Technologist at Leanplum
Venkat Venkataraman, Sr. Director of Engineering @ Gracenote, shares a story about learning from what happens during a company wide initiative that has tons of potential.
Sr. Director of Engineering at Gracenote
Shyam Prabhakar, Engineering Manager at Stitch Fix, explains how design sprints helped him fix problems caused by the lack of sufficient research and overall improve his company’s products.
Engineering Manager at Stitch Fix
Shridharan Muthu, VP of Engineering at Zoosk, discusses effective communication using Slack including a recommended framework that entails three simple tips to make the most of the tool.
VP of Engineering, Backend Applications at Zoosk
You're a great engineer.
Become a great engineering leader.
Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.