Driving Freestyle Innovation Inside Large Organizations

Problem

A few years ago, I was leading a large team at a larger organization. Naturally, there were too many occurrences around with highly visible projects going on in the team. Besides, our organization had a stellar record of developing and shipping products that our customers preferred. Most of the time, our product managers were in touch with our customers to bring back necessary changes to Engineering. We then made the changes needed and shipped them back.

However, the work was too predictable, and innovation seemed to be a missing ingredient for engineers. I discovered that in my 1:1s over a sustained time. To me, it seemed like there was no room for the engineers to have some freedom and make the process work differently. That was how 90 percent of the businesses were getting by, having to fulfill requirements set by customers. In my opinion, I wanted my team to test out different features so that we had the opportunity to innovate.

Actions taken

There was no doubt on the track record of developing and shipping products of my team. I leaned on the excellent track record and proposed and made the case to my Engineering Leader to budget on our innovation plans. We wanted to promote a culture of innovation to scale our efforts. For certain products, we may have gone in the future and thought of some strategic actions to take next.

Followed by this, we got an incremental headcount for upscaling the team and finally innovating our products. We formed a three-people tiger team, one that was very vested in the idea. We also hired two more people into the regular work and diverted the two existing team members to work on innovating on this new idea. The newly formed team would sit in a corner and churn through ideas. This went on for around four months. Safe to say that it was essentially an extended hackathon.

The team wanted to develop their skill sets with machine learning technologies. They took four months to train themselves on various machine learning technologies while continuing to experiment with various parts of the product. We scrapped about 5 - 6 different ideas because we could not implement them with our product. Eventually, they came up with an amazing prototype that predicted auto-corrections and auto completions for the query language for one of our products.

Think about a time when you are searching for something on Google, and there is an automated recommendation that guesses what you are looking for. Being able to do something very similar to that inside of a data analysis product was a game-changer. We got positive feedback from various control groups and our customers. We were also able to file patents that gave the product a competitive advantage.

Lessons learned

• If you are on a tight budget and schedule, well-defined outcomes work best. On the other hand, if you have some wiggle room on the budget, you can divert some towards innovation.
• Happy engineers ship the best products. I have noticed that when there is room for innovation and engineers have some autonomy to work towards their own ideas and create something out of them, team morale is at its peak.
• Few quarters, or even a few years down the line, the skills developed in this “hackathon” can come in handy when working on completely unrelated projects.