Securing Buy-In for New Metrics

Problem

After considerable deliberation, we decided to start collecting the data about our key processes that managers could use to identify strengths and weaknesses across different teams. Though we were very transparent about the data we were collecting, many of our employees became anxious and assumed that we would use them to fire people who were not performing up to the standards. We had to keep re-emphasizing that these metrics were not intended to fire/demote anyone and that poor metrics wouldn’t imply that someone was a poor-performing employee. We extensively communicated what was the goal of the metrics -- to analyze what things, not people, were broken and why.

Actions taken

We decided to mainly focus on pull request flows and measure the times from pull requests being open and interacted with to being completed and merged. Each of those steps gave a lot of insight into where the team was being held up. It became apparent that teams would focus on their own work and were not helping the rest of the organization with pull requests. People were expecting someone else to take care of it, and because it was a prevalent mentality, it took a long time for pull requests to be dealt with.

As we were emphasizing specific metrics, we noticed that the team cohesion strengthened and that the team started to operate more like a team as opposed to a loose group of individuals. They could see what the things we cared about were and that we were concerned about how the team was performing rather than how particular individuals were doing. Also, the team soon realized that some of those metrics could help them grow in the direction we cared about.

However, things didn’t go as smoothly as we hoped for. Initially, the team assumed that the goal was to use metrics for individual performance assessments, and they were largely skeptical. They falsely understood metrics as something that would get them in trouble and would impact if they would get raises or not. But, as they started to improve their performance due to the metrics we introduced, they also realized that those metrics provided them with more ammunition when it came to performance assessment and salary reviews. The metrics provided them data-based arguments to discuss their performance with their managers instead of having to take their managers’ subjective assessment for granted. Moreover, the same could be applied to managers, who had some solid data to corroborate their arguments.

Even when things were not going well and the stats were poor, the sheer numbers could become a great starting point for discussing what they could improve and shed light on some aspects of their performance they were not aware of. That also meant that they could much faster fix their weaknesses because metrics would spare them thinking if their manager liked them or not. They will have an opportunity to assess their own numbers, compare them with the company’s average and track their own progress.

Lessons learned

• Communication is everything. Make sure that you communicate clearly why you want to introduce new metrics and how you intend to use them. That is essentially important for securing buy-in and enforcing more data-driven culture.
• Some people will merely assume the worst intentions. Keep re-emphasizing what your intentions are, and that will hopefully help them overcome their own negative thoughts.
• Introducing cycle time metrics around reviews, pull requests, and bug meantime to resolution proved hugely successful. Some of the metrics like bug meantime to resolution helped connect engineers with our customers and showed them that fixing the bugs is something we value greatly.
• We also found a significant value in introducing metrics around the amount of time spent on a story combined with its estimate, which should disclose how accurate people’s estimates are. A lot of people shy away from framing it as a goal, but our intent was to show the team whether they were doing it consistently or not and to establish if a one-point story is -- most of the time, at least -- half of a two-point story.