The Problem-Solving Process: A Modern, Data-Driven Approach for Engineering leaders
28 October, 2021

CEO at Insightly Analytics
Problem
In an engineering organization, when it comes to people and process efficiency, it is subjectivity-driven and not data-driven, which is a significant problem.
- We cannot tell if all our retrospectives and process changes are making our teams better or worse.
- Are we going faster or slower? Who is overloaded, and who has capacity?
- Why do we miss deadlines when enough time has been spent on sprint planning?
- Why can't we predict issues early when we check in with the team every day on the scrum calls?
- When you add a new engineer to the team, is your throughput going up or down?
All these happen due to the lack of data-driven nature. As a CTO, I had to make decisions for different parts of the engineering teams, relying on subjectivity, and as we all know, that's not always done right.
Actions taken
To tackle this situation, I reached out to other engineering organizations and leaders to understand how they were dealing with this situation and understood this had been a broader problem than I assumed. But, to become data-driven, the number one challenge was to identify the right metrics that would bring in the desired behaviors.
As part of my research, I came across the DORA metrics, which Google launched. DORA did an extensive analysis with over 33K companies globally and came up with metrics that would help bring data culture to any company. I quickly jumped in to hack some dashboards together and realized that it's a lot of work to get this data from disparate tools like Git and Jira and show them in real-time dashboards. Although I could see the value, I couldn't build it internally as it distracted our core business. That's when I went ahead and launched my startup, Insightly Analytics, to solve precisely this challenge.
Now, we use Insightly to make all team decisions using data, which is the starting point of our conversations in any scrum meeting. We were surprised with many myth busters about our efficiency and how we were investing the efforts in the wrong places due to lack of data.
Today, instead of asking round-robin about the status updates during our standup call, we look at the dashboard, understand the work completed, and finish our calls in half time. With every release, our retrospective meetings clearly show if our previous decisions are helping us get better or worse. Most importantly, we can look backward, identify patterns and celebrate successes on our improvements.
Lessons learned
- Data-driven processes will bring a universal language based on which teams can be aligned and easily communicated across.
- Having real-time dashboards with data can reduce a lot of meeting times.
- When data is available at managers' fingertips, there are fewer slack interruptions asking for status updates and reports.
Discover Plato
Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader
Related stories
5 December
Your Org Team may as well be a Sports team. Let's explore how this cohesive, multi-skilled team can be optimized for Great Group Playoff.

Jaroslav Pantsjoha
Google Cloud Practice lead at Contino
25 October
Mrunal Kapade, an Engineering leader, based in Silicon Valley, shares tips that helped reduce attrition in the remote engineering teams while leading multiple teams from startups to Fortune 500 companies.

Mrunal Kapade
Director of Engineering at Inspire Energy
26 June
Individual Contributors are familiar with a technical development framework that helps them with building products. Managers, especially new managers can leverage a parallel framework to help them build their teams while drawing analogies from an already familiar framework.

Viswa Mani Kiran Peddinti
Sr Engineering Manager at Instacart
13 June
Roland Fiala, Senior Vice President of Engineering at Productsup, raises an interesting issue about autonomy in teams: does it hinder collaboration opportunities that lead to better problem-solving? He shares his system for promoting teamwork in engineering departments.

Roland Fiala
Senior Vice President of Engineering at Usergems
13 June
Roland Fiala, Senior Vice President of Engineering at Productsup, highlights the importance of soft skills and shares how he motivates his engineers to further their careers by focusing on personal growth.

Roland Fiala
Senior Vice President of Engineering at Usergems