Build V.S. Buy: Empowering Teams with a Data-Driven Approach

Engineers Want to Build

In large organizations, when initiatives come up, there are build v.s. buy decisions that need to be made. Inherently all engineers and data scientists have the notion that it should be done in-house. Why create a partnership when we can do it ourselves? It’s very important for product leaders to assess the build v.s. buy, and how to make the right decision.

Around ten years ago, my company was putting a novel product into production – involving artificial intelligence and modeling. We partnered with two vendors who were leaders in these industries, specifically modeling and machine learning. As I mentioned, there were team members that were incredibly interested and thought the machine learning model could be developed in-house.

Experimentation Within a Team

I was leading the product at that time and understood that these team members wanted to take an opportunity to grow and learn. Similar to many hi-tech companies such as Google, I encouraged experimentation and implemented a schedule that allowed team members to focus on new innovation and research, 20% of the time, for professional development and had high value. In combination with this, if any individuals came up with something high impact, my team would bring it to life.

There was a very smart engineer on my team who spent a significant amount of time coming up with a machine learning model for the product we were working on. I had a few conversations with him regarding machine learning performance and set a threshold of 80% accuracy before it could be launched into production. The idea was that once it was in development, the team could continue to improve upon it and gain a higher accuracy.

He continued to work on his model, working nights and on his 20% time. Finally, he created a model, and put it in a sandbox, only to realize it was performing with only 60% accuracy. Obviously, this was not the threshold that we wanted, and we could not bring it to life with such a low accuracy. The team could have spent time enhancing the threshold but it would be at a cost.

Empowering the Team with Data

This experience taught me that instead of pushing back on team members’ ideas, it’s important to allow them to pursue their passions while being data-driven. In this example, I gave my team member the runway to experiment with his machine learning model. Even after the accuracy was only 60% effective, he continued to work on it for a few weeks to try and improve the score. Eventually, he came back to me and said that we should leverage our vendors to build the machine learning model – as they were the experts in the field.

Since I built trust with him by providing him with opportunities to experiment, his data was telling enough for him to come up with a different recommendation. While keeping the goal in mind, I empowered my team to make decisions with a data-driven approach.