Measuring the Success of a Software Engineering Team

Problem

I have found that software developers don't like being asked questions around metrics and measurements. This is because they think that a number of coded lines or a number of check-ins doesn't actually tell the complete story. That being said, if you are using a standard Scrum methodology, there are useful ways to monitor your team's productivity that will also engage, motivate, and ultimately get buy-in from your software engineers.

Actions taken

First, understand what your team is capable of. A good way to comprehend this is through Scrum burndown and burnup charts. These charts give you some idea of velocity in terms of user stories and other factors. They will be different for each team and project, yet after a few a Sprints they will display a fairly accurate estimation of your team's progress and the amount of time needed to complete the remaining work.

Next, I recommend using tools that are available on the market. For example Git or another depository system. Jira is another great one as well. And while I've only looked at the demo for GitPrime, I know that it gives a lot of metrics that are automatically dug out of the system. For example, how many check-ins happen, how much of the code gets reworked, and other items that might be of interest to software engineering managers.

For me, personally, I keep track of the aspect of time. I do so through the derivation of two processes. One is the Scrum process, which is an iterative two week delivery cycle. The other is at a strategic level and includes a large task, an objective stating the need to build a feature in a particular platform. Then, I look at the time it takes my team from inception to delivery. Once it has been delivered, I focus on how easy it is to take on customer feedback and to change the feature. This is but one macro trend that I rely on.

Another trend I track is bug leakage issues in the field. How often do we have issues with releases that are happening in the software? How quickly can we make changes to a platform? This is a good measure of how well the system has been architected and how well the features have been added. While I haven't solved the problem of standardizing this across all of my engineering teams, I still use Jira reports and spreadsheets to keep track of these effects.

Lastly, I find that the Scrum team is heavily dependent on people and the team itself coming up with their own metrics. I ask my teams to decide what they would like to be measured against. If they don't like to be judged on lines of code then give me something else that would make sense. For example, the performance characteristics of our platform is very important. And this is something that is easy to measure. So I have been tracking this and have several years' worth of charts on how much our latency has improved over time.

Lessons learned

• There's an element of truth to the fact that software development can be a bit of an art rather than a science. But this is because there is a characteristic of looseness around it. It's not exactly precise and because of this it can be hard to measure. But that doesn't mean it's impossible. It's a matter of finding the right tools and metrics that both you and your team will be comfortable visiting and discussing frequently.
• There can be some danger of metrics and measurement being 'gamed'. Despite this, I still look to see how many major releases have occurred, how many patches there are, how many hot fixes, and other totaled tasks that have happened on our platform. These standard metrics are useful because they always give you some sort of information. I find that when you start getting into the more qualitative areas of measurement is where it can get a little bit trickier.
• My main advice is to go back to the basics of Scrum and Agile which is empowerment and making people feel a part of the process.