Plato Elevate Winter Summit has been announced (Dec 7th-8th)

🔥

Back to resources

Alleviating the Infamous QA Bottleneck

QA Team

19 September, 2021

Michael Scotto
Michael Scotto

Director of Quality Assurance at Niche.com

Michael Scotto, Director of Quality Assurance at Niche, describes how his team managed to alleviate the infamous “QA bottleneck” and drive a discussion about quality beyond the boundaries of the QA team.

Problem

Our organization has doubled in size in the last year, including the QA team I run. With a significant influx of people and changes in processes, we inevitably ran into hiccups. One of the challenges we experienced was that many tasks were piling up on the QA within our product teams, slowing down overall velocity. I had to figure out how to alleviate the fabled “QA bottleneck” by identifying the concrete challenges the QA team faced, and mapping out solutions for each.

Actions taken

To start, I gathered two QA managers who reported to me and whose teams were focused on manual testing. The three of us sat down for several hours and discussed the challenges their teams faced in-depth. “QA bottleneck” is a blanket term that can hide multiple smaller problems within it; we had to dive deeper and identify the underlying root causes.

First, we listed out all the different points in our current development life cycle that tended to cause delays or simply slow velocity, and then we ranked these periods of waiting in order of time lost. This revealed a range of opportunities for improvement. Some were matters of timing -- elements of the development life cycle that were serialized when they could be parallel, for instance, such as code review and test planning. Ultimately, though, we found that our QA team test plan review process -- where testers receive feedback from fellow testers on the completeness and clarity of their test plan -- was the area that caused the longest wait times and thus carried the highest cost.

Because most of our product teams only had one QA on them, almost every request for test plan review required resources from another team. Thus, the process had a double cost -- QA on one team had to wait to receive a review, while QA on another team had to pause their own work to perform the review. Test plan review added real value, catching gaps in coverage and presenting an organic opportunity to build context. Our challenge became clear: retain that value while reducing its cost.

Ultimately, the QA managers and I decided that we had to enable every individual product team to perform its own test plan reviews. We started by developing written guidelines, bringing more transparency to how QA conducted test plan review. I initiated the process, but since the managers below me would own the execution of any solution, I encouraged them to take more of a lead role. I wanted to step back and be more of a facilitator, asking questions and encouraging discussion that would help us define excellence.

The guidelines we came up with allowed any member of a product team -- whether they were a software developer, data analyst, or otherwise -- to give input on QA test plans. That reduced all wait times for test plan reviews significantly. We saw immediate improvements in terms of teams meeting sprint goals, estimation being more accurate in terms of both time and effort. We also saw ancillary benefits. QA’s process became more transparent, and developers were even able to identify potential bugs in their code just by reading QA’s plan, causing bug fixes to get pushed before testing even began.

We piloted this approach on one product team to start with. Once we had initial results, we didn’t have to push other teams to adopt it; they actively wanted to.

By rethinking how a test plan review could be performed, we turned a weakness in our process into a strength that allowed us to drive a discussion about quality beyond our functional team. That was much more in line with the broader mission of my department, which is to encourage all members of the organization to think more about quality. We believe that quality is not a QA team responsibility, but a company responsibility; it was great to be able to put that principle into practice while solving a real problem.

Lessons learned

  • Embrace being out of your comfort zone and keep looking for opportunities to improve. This can be challenging and stressful, but the best opportunities often dwell within uncertainty.
  • By encouraging test plan review to be thought of as a product team activity, rather than a functional team activity, we encouraged a better dialogue about quality within product teams and helped to break down a silo that was hindering velocity.
  • Being a rapidly growing company with distributed teams and relationships still forming, it can be challenging to push for large-scale changes. Starting small on one product team and achieving results that we could showcase to the rest of the organization made adoption easier. Having quick wins to share got us instant buy-in.
  • As a leader at a director level, this was also a great opportunity to practice managing through my managers beneath me. In the past, when we were smaller in size, this could have been an initiative that I would personally spearhead. I didn’t think it would be the right approach now because I was not going to be involved in the execution. By working with and trusting my managers, we were successful beyond expectations.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

Learning From Your Mistakes

13 October

Phillip Derosa, Global Director of QA at OneSpan, believes that every pitfall is a learning opportunity in disguise.

QA Team
Strategy
Phillip Derosa

Phillip Derosa

Global Director of QA at OneSpan

Telemetry in the QA Process

13 October

Phillip Derosa, Global Director of QA at OneSpan, lives to know what his players are getting into as they explore every corner of the worlds created for them.

Innovation / Experiment
QA Team
Team Reaction
Team Processes
Phillip Derosa

Phillip Derosa

Global Director of QA at OneSpan

How to Wear a Lot of Hats

13 October

Phillip Derosa, Global Director of QA at OneSpan, has never been afraid to fill in the cracks between departmental duties, often coming up with a brand new intermediary position to occupy during times of need.

QA Team
Leadership
Agile / Scrum
Phillip Derosa

Phillip Derosa

Global Director of QA at OneSpan

Alleviating the Infamous QA Bottleneck

19 September

Michael Scotto, Director of Quality Assurance at Niche, describes how his team managed to alleviate the infamous “QA bottleneck” and drive a discussion about quality beyond the boundaries of the QA team.

QA Team
Michael Scotto

Michael Scotto

Director of Quality Assurance at Niche.com

Establishing a New Function

19 September

Michael Scotto, Director of Quality Assurance at Niche, shares how he established a new functional role by formally separating automated and manual testing, and the benefit provided by giving his team the space to specialize as the company scaled.

QA Team
Team Processes
Michael Scotto

Michael Scotto

Director of Quality Assurance at Niche.com

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.