Back to resources

Going from no testing to full QA automation

Different Skillsets
Personal Growth
Legitimacy
Collaboration
Career Path

16 April, 2018

David Murray
David Murray

Cofounder at Confirm

David worked with a small engineering organization to grow and scale the Quality Assurance team from nothing to a team of 8 QA members doing significant automation.

Problem

With a small team, it's common for engineering to test their own code. Sadly, this can result in missed bugs due to the myopia that comes from testing your own work. As the work grows, this problem gets worse.

Actions taken

As we've grown the engineering organization, I've been focused on maintaining a QA-to-engineering ratio of about 1-to-4. This ensures that there's always a third party that is focused on testing. At first, these tests can be manual; this will also mean that you can hire someone affordable. A critical procedural invariant to have here, though, is that any regression, i.e. something that used to work that no longer does, needs to be taken as very high priority and resolved ASAP. Without this, your users will not have trust in your product, as they will have learned that what worked today may not work tomorrow. With this acknowledging, we've focused on fixing regressions quickly, and when we had budget for our third QA person, we hired someone who had interest in automation. From this, we started by automating the most common, least likely to change test cases (e.g. logging in, creating an account, etc.). These automated tests then get run every time new code is checked in. Through a continuous build, we're able to then catch bugs as soon as they enter the codebase. If code coverage is measured and regularly shared broadly, the team will focus on improving the coverage over time. By focusing on these tests as well as developer-initiative unit tests and other functional tests, your code is more likely to be stable in production over time.

Lessons learned

Start with no QA, but as you grow your engineering team maintain a ratio, e.g. 1-to-3, 1-to-4, or 1-to-5, whatever feels right for what your organization does. Manual testing is a great way to get started, but automating overtime is critical. Again, regressions must be treated with very high priority to maintain user trust.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

Working Moms and Problems They Face

20 January

Yang Wang, Engineering Manager at Bond, shares how diligently she transitioned from a large company to a series of startups, all balancing between the role of being a working mom.

Goal Setting
Coaching / Training / Mentorship
Career Path
Agile / Scrum
Yang Wang

Yang Wang

Engineering Manager at Bond

Coaching Women Engineers: Professionally, and for Life

20 January

Yang Wang, Engineering Manager at Bond, shares how she coached several women engineers in her team to boost confidence and help them grow in their careers.

Coaching / Training / Mentorship
Team Processes
Career Path
Yang Wang

Yang Wang

Engineering Manager at Bond

From Big Tech to Startup: Adding Value From Day 1

19 January

Angel Jamie, Chief Product Officer at Yayzy, recalls his transition from a well-established tech company to a sustainability startup, and the major differences he experienced.

Dev Processes
Company Culture
Impact
Team Processes
Cross-Functional Collaboration
Changing Company
Career Path
Performance
Angel Jaime

Angel Jaime

CPO at yayzy

Should You Stay Up to Date with Technical Skills As a Product Manager?

19 January

Nani Nitinavakorn, the Sr Product Owner at Revolut, describes how she keeps learning hard skills to increase motivation and respect her team.

Alignment
Innovation / Experiment
Different Skillsets
Personal Growth
Ownership
Coaching / Training / Mentorship
New PM
New Manager
Nani Nitinavakorn

Nani Nitinavakorn

Sr Product Owner at Revolut

Getting Creative to Land Your First Tech Job

19 January

Nani Nitinavakorn, the Sr Product Owner at Revolut, shares how she gained her first technical position, creating a direct method to apply for jobs.

Personal Growth
Ownership
Hiring
Strategy
Juniors
Career Path
Nani Nitinavakorn

Nani Nitinavakorn

Sr Product Owner at Revolut

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.