Going from no testing to full QA automation

Different Skillsets
Personal growth
Career Path
Collaboration
Legitimacy

16 April, 2018

David worked with a small engineering organization to grow and scale the Quality Assurance team from nothing to a team of 8 QA members doing significant automation.

Problem

With a small team, it's common for engineering to test their own code. Sadly, this can result in missed bugs due to the myopia that comes from testing your own work. As the work grows, this problem gets worse.

Actions taken

As we've grown the engineering organization, I've been focused on maintaining a QA-to-engineering ratio of about 1-to-4. This ensures that there's always a third party that is focused on testing. At first, these tests can be manual; this will also mean that you can hire someone affordable. A critical procedural invariant to have here, though, is that any regression, i.e. something that used to work that no longer does, needs to be taken as very high priority and resolved ASAP. Without this, your users will not have trust in your product, as they will have learned that what worked today may not work tomorrow. With this acknowledging, we've focused on fixing regressions quickly, and when we had budget for our third QA person, we hired someone who had interest in automation. From this, we started by automating the most common, least likely to change test cases (e.g. logging in, creating an account, etc.). These automated tests then get run every time new code is checked in. Through a continuous build, we're able to then catch bugs as soon as they enter the codebase. If code coverage is measured and regularly shared broadly, the team will focus on improving the coverage over time. By focusing on these tests as well as developer-initiative unit tests and other functional tests, your code is more likely to be stable in production over time.

Lessons learned

Start with no QA, but as you grow your engineering team maintain a ratio, e.g. 1-to-3, 1-to-4, or 1-to-5, whatever feels right for what your organization does. Manual testing is a great way to get started, but automating overtime is critical. Again, regressions must be treated with very high priority to maintain user trust.


Related stories

Developing High Performing Teams: How To Transform a Software Developer Into a Product Engineer
30 May

Alessandro Pintaudi, Product Management Director at Payfit, comes up with an exciting proposal of transforming software developers into product engineers by establishing cross-functional context analysis and shared objectives.

Cross-functional collaboration
Product Team
Collaboration
Alessandro Pintaudi

Alessandro Pintaudi

Product Management Director at PayFit

Leveraging Diverse Peer Groups for Tighter Feedback
24 May

With both the need for a more supportive team setting and shorter feedback cycles, Marc LeBrun, VP Engineering at Flow Kana, addresses two problems with a single solution.

Team reaction
Cross-functional collaboration
Collaboration
Marc LeBrun

Marc LeBrun

VP Engineering at Flow Kana

Leveling Up Without Direct Support From Your Manager
17 May

Summary CTO at Culture Purveyor Labs, Kisha M Richardson, who for the most part is self-taught, defines the parameters in which she was able to boost her career. Neither a lack of information nor the fact that she was pigeonholed by her first manager kept Kisha from realizing her professional reach.

Personal growth
Ownership
Kisha Richardson

Kisha Richardson

CTO at Culture Purveyor Labs

How to Bring Business Context to the Development Team
15 May

Maria Petrova, Principal Product Manager at Zalando details how she strategically mapped out features using a KPI tree to drive measurement, ultimately helping the development team understand their role.

Conflict solving
Collaboration
Maria Petrova

Maria Petrova

Principal Product Manager at Zalando

Building an Effective Partnership Between Product & Engineering
15 May

Paulo André, VP of Engineering at TourRadar, emphasizes all the benefits of enhancing partnership between Product and Engineering and explains how to achieve it.

Cross-functional collaboration
Internal Communication
Collaboration
Paulo André

Paulo André

VP Engineering at TourRadar

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.