Elevate Spring Summit has been announced (Thu, Mar 11th)

🔥


Don't have an account? 

Going from no testing to full QA automation

Different Skillsets
Personal growth
Legitimacy
Collaboration
Career Path

16 April, 2018

David worked with a small engineering organization to grow and scale the Quality Assurance team from nothing to a team of 8 QA members doing significant automation.

Problem

With a small team, it's common for engineering to test their own code. Sadly, this can result in missed bugs due to the myopia that comes from testing your own work. As the work grows, this problem gets worse.

Actions taken

As we've grown the engineering organization, I've been focused on maintaining a QA-to-engineering ratio of about 1-to-4. This ensures that there's always a third party that is focused on testing. At first, these tests can be manual; this will also mean that you can hire someone affordable. A critical procedural invariant to have here, though, is that any regression, i.e. something that used to work that no longer does, needs to be taken as very high priority and resolved ASAP. Without this, your users will not have trust in your product, as they will have learned that what worked today may not work tomorrow. With this acknowledging, we've focused on fixing regressions quickly, and when we had budget for our third QA person, we hired someone who had interest in automation. From this, we started by automating the most common, least likely to change test cases (e.g. logging in, creating an account, etc.). These automated tests then get run every time new code is checked in. Through a continuous build, we're able to then catch bugs as soon as they enter the codebase. If code coverage is measured and regularly shared broadly, the team will focus on improving the coverage over time. By focusing on these tests as well as developer-initiative unit tests and other functional tests, your code is more likely to be stable in production over time.

Lessons learned

Start with no QA, but as you grow your engineering team maintain a ratio, e.g. 1-to-3, 1-to-4, or 1-to-5, whatever feels right for what your organization does. Manual testing is a great way to get started, but automating overtime is critical. Again, regressions must be treated with very high priority to maintain user trust.


Related stories

Learn or Die!
12 January

Marian Kamenistak, VP of Engineering at Mews, outlines key recommendations and resources to make learning an integral part of the engineering role and knowledge sharing across the company a steady practice.

Personal growth
Coaching / Training / Mentorship
Marian Kamenistak

Marian Kamenistak

VP of Engineering at Mews

Building a Diverse, but Cohesive Team
30 December

Daniel Burke, Engineering Manager at Square, describes how he built a diverse team -- in all possible ways -- that championed cohesion and collaboration in their most unique way.

Different Skillsets
Building a Team
Diversity
Cultural differences
Daniel Burke

Daniel Burke

Engineering Manager at Square

Why a Risk-Driven Product Roadmap
20 January

Aram Grigoryan, Product Manager at Qualtrics, explains how a risk-driven product roadmap can help with building a product that bridges the gap between the digital and physical world.

Product
Collaboration
Roadmap
Aram Grigoryan

Aram Grigoryan

Product Manager at Facebook

Managing Cross-Team Dependencies
28 December

Frances Li, Lead Product Manager at Carta, tells of her recent experience of managing dependencies with another team and how ensuring transparency is a prerequisite for resolving them.

Cross-functional collaboration
Managing Expectations
Collaboration
Frances Li

Frances Li

Lead Product Manager at Carta

Bringing Tangible Results Within the First Quarter
28 December

Frances Li, Lead Product Manager at Carta, recalls how a sudden change of direction and decision to bring tangible results within the first quarter impacted a product she was building.

Collaboration
Roadmap
Prioritization
Frances Li

Frances Li

Lead Product Manager at Carta

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.