Measuring Quality and Getting the Team to Buy-In

Problem

When we first published the health framework we ran into some complications. It was created to measure our quality in three main areas - code quality, the velocity of code shipping, and impact. Those areas cover 10 metrics that we track from release to release so that we can see how we are trending; either negatively or positively depending on different areas. The complication arose when instead of developers viewing their own metrics as an opportunity to take ownership of getting to a better place as an individual, team, and organization, they formed doubts about how this data would be used. Many people believed that we would be using the metrics to single out people on their effectiveness and efficiency. What we really were trying to use it as was a conversation starter to see what areas we could tackle and improve together.

Actions taken

We were able to reassure our developers of our intended use for this framework on a really challenging project where some of the stakeholders were worried that the team wasn't producing at the level that they should have been. We did so by showing the stakeholders how the quality of the requirements were affecting our team members. Comparatively, we were also able to demonstrate to them how the team performs when a project is thorough, complete, and stable versus when it is not. They were thusly able to see how their productivity was affected by this.

Lessons learned

• The health framework is a good tool to show how a team that performs excellently can also perform very badly because of external factors. This is a powerful thing to be able to show the product management team.
• Now people see that it is being used in a positive way so they are more open to accepting that is not a watch system for pinpointing problems, but really to help out wherever we can. Some developers still hold some anxiety towards this but we are working to make all metrics public and not just those for personal use in order to ease some of this tension.
• We have made some progress in identifying the areas where we need to improve and are working with our teams to make the organization more efficient, productive, and impactful.