Shipping Products Faster with Confidence

Problem

Many early-startups or mature companies with an early-stage product are concerned about finding a product-market fit. They often fail to notice that to find the product-market fit, they need to ship faster. Getting features to a customer faster is a critical part of finding a product-market fit because the faster you put features out, the faster you can receive feedback and build something relevant.

However, as engineers, we don’t want just to ship faster; we want to feel confident about the whole process. We want to feel good about the quality and ideas that are being put out.

Actions taken

To move faster with confidence, you need to assess the full lifecycle of building features, from ideation of features to their development and through to launching. That should help you familiarize yourself with the end-to-end operating process, including potential bottlenecks. That is vitally important for your confidence. You may speed up things, but you need to know if you are working on the biggest bottleneck or something that brings the highest ROI.

For example, I found out that testing and regression verification were the largest bottlenecks in my current team. Our manual testing processes would force the release process to take a minimum of a week. After analyzing the whole process, I was certain that those were the largest bottlenecks that had to be addressed. By addressing them first, we could improve not only the speed but our confidence around quality too.

I realized that what was lacking was ownership in the quality of the code. Since someone else was running manual tests and engineers were not owning the quality, it was slowing them down from moving faster and having confidence in their own code. After identifying the problem, I decided to attempt bootstrapping the testing culture and have engineers automate more. I wanted engineers to own product quality the same way they own features. I looked at automated testing, where the gaps were, and pinpointed the fastest way to close the gaps. Unit tests are at the bottom of the pyramid, integration testing is placed in the middle of the pyramid, and end-to-end testing is positioned on the top. Our engineers were doing a bit of each, but we had to invest more into end-to-end testing to make the highest impact. That would automate away a lot of manual testing and cover for what customers were typically doing with the application.

However, that required the full bootstrapping of our testing culture and educating engineers on how customers were using the product. We executed a concerted technical investment with a list of prioritized and scoped efforts. Moving forward, engineers automate what they are doing to verify that what they were building works. Furthermore, the increase of end-to-end coverage from 10% percent to 45% percent while adopting the new tools to enhance developers’ experience significantly amplified the confidence within the team.

Lessons learned

• Measure, measure, measure! Don’t hesitate to anticipate different “what if” scenarios. We measured what was taking the most time and, based on that, were able to diagnose the problem. Ahead of doing anything, we predicted (what if) what the impact of proposed actions would be.
• An essential part of building ownership was an understanding of how customers were using the product.
• Testing code is an asset to a team that grows along with the application code. Testing infrastructure is as necessary as features are. If you are making the testing experience great for developers, it compounds on itself.