Keeping Up With Rapid Growth for High-Performance Engineering Teams

Problem

Startups are notorious for their rapid growth, and to no surprise, engineering organizations are spearheading that growth. I joined my current company in 2016 as the first engineer, and as we speak, I am leading the engineering organization of 100+ engineers. It didn’t happen overnight and without many challenges along the way, though. Growth always brings some concerns. Are we doing the right things? Are we prioritizing correctly? Is the engineering organization as productive as it should be? Do we have enough accountability to support that growth?

Our concerns were only amplified as we came close to Dunbar’s number beyond which people cannot maintain stable social relationships. I could tell that our growth made some things somewhat more chaotic. At that time, we were structured based on functions -- data engineers, software engineers, data scientists, etc. There was a lot of shared accountability, and it often felt like we were doing waterfall within the agile framework. That strongly indicated that the time was ripe for a change.

We singled out three main principles -- accountability, transparency, and technical excellence -- that we believed should guide us through the upcoming change and help us establish a solid organizational structure that would match our aspirations.

Actions taken

Prep work

My role was evolving as rapidly as we, as an organization, grew. I was to be responsible for this ambitious project, with the CEO being an executive sponsor. For starters, we discussed if we should bring someone from the outside to help us through the process. While I didn’t doubt our capacities, I welcomed external -- an impartial -- support by someone who underwent scaling of this kind before and could tell if we were on the right track. We found a consultant whom we hired to act as our advisor. However, having that external pair of eyes made us realize that what worked in the industry may not work for us and that we would need to customize the existing solutions to our unique needs.

Accountability

Shared accountability started to take its toll, and we had to assess if some other organizational models could help us reinforce it. We did thorough research on the existing models, from the Spotify model of squads and tribes to McKinsey’s Three Horizons model. None of those matched our needs entirely, so we had to handpick different ideas from different solutions.

We restructured the team a bit. We would still call team units squads, and they would still be cross-functional, but we would group them into three large buckets based on their focus areas. The three focus areas were:

• Platform Operations -- would do most of the platform & infrastructure operation work, and their timeline shouldn’t exceed a couple of weeks period;
• Product development squads -- would implement the roadmap, and they should be three to six months forward-looking, with a small research unit doing longer-term research.
• Customer deployment squads -- would make our product useful to customers by configuring it in the context of customer data. That involved a lot of retraining of ML models, configuring data by planes, and getting the right set of data. They would be heavily dependent on customers’ cadence of operations.
  

Next, we defined the roles and responsibilities. Each of the three focus areas had multiple squads with 7 to 9 people. Squads would differ depending on their focus area. For example, product development squads would have product managers, while customer squads would have customer success managers. We tightened up our definition of roles so everyone across the organization should know what an EM or PM does. Before, though we had our job role descriptions in writing, a lot of it got outdated as the organization grew and much was left to how people would interpret certain things. So we had to define a new engineering career ladder with clear progression paths and responsibilities. We also did a lot of training for teams and managers to understand the new structure and their roles. While we were keen on keeping teams autonomous & driven by purpose, we wanted to balance that flexibility with accountability and clearly demarcated ownership.

Transparency

We were confident that our engineers were competent, but nevertheless, there were pain points with velocity & prioritization. There was a lot of confusion on who owned what and who prioritized on what. There was no doubt the confusion would only exacerbate with our continuous growth, so ensuring clarity and alignment seemed a must. We started off by transforming some of our internal processes and procedures: how the product team defined requirements, how they would get prioritized, and how they would be delivered to Engineering. We even switched to another product management tool because we felt it would increase the overall transparency.

Though we were committed to following Agile quite strictly and were using Scrum methodologies, it obviously didn’t work for all teams and all focus areas. For example, the customer team at an early stage of the configuration depended significantly on customers’ timeline and availability, while Operations were mostly working on a day-to-day basis, so Kanban seemed a more reasonable option. We also tried to do our estimations as accurately as possible and moved away from Story Points to a combination of T-Shirt sizing at the Epic level and Time Estimate at the Task breakdown level.

In the past, people would work on a ticket with no estimation, and things would be prioritized without knowing how much effort would be spent on it. We made changes to our workflows to have checks before tasks can be pulled into a Sprint. More precise estimates also helped us understand what were the things taking most effort -- ad hoc or planned tasks; developmental activities, bug fixes, or platform work.

 

Technical excellence

In the past, before we introduced squads, people with similar skill sets were grouped together and had an opportunity to learn from each other and improve on their skills. Each of those groups was a small community with its own learning practices. We took (again) some concepts from the Spotify model -- guilds or chapters, though we decided that the name didn’t matter -- adjusted them to our need of connecting people with similar skill sets. We decided to reserve 10 percent of sprint time for people to engage in activities outside their day-to-day work, whether learning or experimenting with new approaches.

We identified senior technical leaders for each of those groups -- and people based on their skillsets & interest could be part of one of these groups. That also allowed people to see what was happening across the organization and improve in their craft. For example, even within data science or engineering, we have different types of challenges that the teams solve, and this initiative created a forum to talk about such challenges people were facing while working on the different products.

We also established an architecture council that we internally refer to as the Jedi council. It consists of a number of tech leads and a few managers who would be responsible for various areas: internal processes, architecture stack, DevOps, etc. They also have the mandate to influence the product roadmap through their technical vision. We gave everyone a chance to grow in their craft by continuous learning and having a forum in which a technical vision and forward-looking innovation can be laid out and influence our products.

Lessons learned

• Even the best solutions may not match your needs. Handpick different ideas and customize them to your needs.
• If the budget allows, go for an external consultant to help you with a process you are doing for the first time. But be aware, there are internal, cultural things that are difficult to be grasped by an outsider. So, be realistic in your expectations of how the consultant can help and with what. When things moved from a more strategic to an operational level, we had to identify internal people who would be agents of change.
• Implementing process-level changes is never easy. Sometimes we would go back and forth with some processes, which may cause anxiety in the team. But if you are clear on the goals and what you want to achieve, you can quickly pivot and move forward. What initially looked like a blocker and made the implementation of new processes difficult became an opportunity to learn more about our strengths and weaknesses. No chance is easy, and you have to give it some time to take hold.
• Declaring something a rule won’t make people rush to embrace it. Though we introduced a “10 percent learning” rule, not everyone was enthusiastic about it, and we had to find other ways to encourage our engineers to participate in those activities. A similar scenario involved the architecture council, and making it functional took some time. In both cases, we streamlined our objectives, armed ourselves with patience, and had things move forward steadily.