Back to resources

Managing Expectations on a Highly Visible Project

Managing Expectations
Internal Communication
Stakeholders

9 August, 2021

Ramayan Tiwari
Ramayan Tiwari

Senior Software Engineer at Netflix

Ramayan Tiwari, Senior Software Engineer at Netflix, details how to manage expectations on a project that attracted significant interest from a number of stakeholders even before its official start.

Problem

I started a brand new project, which was much in demand even before its official start. The problem we were trying to solve was exceedingly important to the business, and many teams were interested in using our project. The craze kicked off even before we began articulating the idea and coming up with the design.

We tried to improve the Netflix recommendation system and make it more real-time and take into account the browsing habits of our users. When not playing video content, our users would navigate the home page, engage with a few rows, play trailers, look for specific titles, using thumbs up/down to rate content, etc. All those activities were important because they could tell us what a user’s intent was. If we could further ramp up our recommendation and personalization system, it would make an even more satisfying user experience.

From a data perspective, we were collecting the data that would help us understand what users were doing in real-time and capture their interactions over a longer period of time. We could go back for a month or two to learn from their activities, personalizing the system accordingly. Apparently, that was critically important to different business units, and no wonder that a number of teams, including the search, recommendation, page construction, and ranking team, were interested. The challenge I found particularly difficult was how to manage expectations given the excitement surrounding the project from the start and furthermore, deliver the features timely without compromising on quality.

Actions taken

For starters, I had to understand what were the most common use cases. To do so, I talked to a number of consumers, meeting with them one-on-one. I would also meet with other teams to get precise requirements for the projects. I would ask them very specific questions such as, How will you query the data or How will your systems integrate with the new system? I also wanted to learn about their priorities. If a particular access pattern was important to them, was it more important than the other one? I wanted to understand and establish a clear order of priorities early on. That helped me plan my milestones so that I delivered the most important features first.

Next, I had to find a balance between the most important use cases. I wanted to make sure that the use cases I was targeting in the V0 or V1 release, included use cases coming from a variety of teams and users from whom I could get feedback. V1 should be good enough to address most use cases so that people could onboard in the system and help improve it by providing feedback.

We started to build a proof of concept early on, which was somewhat unique. We would begin with a smaller project that would be easy to build but could provide users with some concrete data about what the full-scale project would look like. We built it and demoed it to our users to see what they were thinking. When one is in the early stages of a project, most of the time, one talks hypothetically: I will do this or that. Not surprisingly, users may have different interpretations of what that should be. It’s important to be on the same page from the expectations standpoint from the start.

We build a proof of concept application in a few weeks time and then host a demo for a larger audience. We explained how things would look like, how fast it would be, what data could be captured, and made available to the consumers with acceptable latencies. We asked them for their feedback explicitly: Is this the direction you had in mind? How far is this from your expectations? Based on their feedback, we formalized what V0 should mean in terms of requirements, specifications, and timeline.

We also wanted to regularly show progress to our users. Since everything was taking place after the Covid pandemic hit, demoing through a screen was the only option. I missed going to users in person and collecting their feedback chatting in the hallway. We were demoing to a larger group at least once a month, frequently doing demos to different teams weekly.

Doing demos was a great opportunity to get users interested in the project. We also built a tool that could do some visual demos. If a demo cannot be visually interpreted, it would be less appealing. When someone is producing data as a project, they should find a way to make it more attractive. Building some UI or visualization tool that could take that raw data and present it visually is critical to draw interest. We enabled users to see what was being created to be visible on their Netflix screens.

By moving from the early idea to a more mature product, we also moved from managing expectations to managing adoption. You can build something great, but your effort will be futile if no one is using it.

Lessons learned

  • We were trying out a new technology that was not mature and supported enough. Alternatives were simply not a good fit for our use case. We could have used three other available and well-supported technologies, but they were not a great fit for this particular project. We started using this new technology, and of course, as with any new technology, we expected a lot of hiccups, especially when operated on a large scale. If you are going with a less adopted solution, consider the failure domain and build tooling early on to mitigate failures. For example, when a catastrophic failure happens in your data storage, you should be able to build that state back. In this process, we helped improve the new technology around its stability at a larger scale.

  • We built the system for scale from day one. We didn’t launch this project to build it for 1 percent of users and then slowly increase to 10 and 50 percent. The reason is simple: we are building a data store that all users will use. If it gets adopted and people want to onboard, we don’t want to put them on hold for a quarter or two until we try a new system that can scale. We launched this system with 100 percent traffic from the early on. The scale is the most important requirement; if it doesn’t scale, it would be a failure no matter how good the data is.

  • Start your integrations early on. Most often, companies have multiple services and libraries that consume each other to produce a complete and cohesive system. If you are going to introduce a new system, talk to people who will be either producers or consumers of the system and make sure they can integrate. If you do this integration early on, you will create a feedback loop, which will help you understand which parts of the system you will not be able to integrate.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

How to Maintain Happiness: The Underrated Aspect of Creating Team Dynamic

2 August

Jonathan Ducharme, Engineering Manager at AlleyCorp Nord, encourages the importance of a workplace environment that cultivates mental wellness.

Personal Growth
Company Culture
Leadership
Internal Communication
Psychological Safety
Jonathan Ducharme

Jonathan Ducharme

Engineering Manager at AlleyCorp Nord

Scaling a Team in Two Parts: The Product and Manager

2 August

Viswa Mani Kiran Peddinti, Sr Engineering Manager at Instacart, walks through his experience scaling a team, product and his skills as a leader.

Managing Expectations
Product
Scaling Team
Leadership
Meetings
Viswa Mani Kiran Peddinti

Viswa Mani Kiran Peddinti

Sr Engineering Manager at Instacart

How to Organize, Manage, and Grow Your Team

12 July

Vineet Puranik, Senior Engineering Manager at DocuSign, discusses the impact of roadmaps, organization, and proper management for your teams to procure growth.

Managing Expectations
Delegate
Collaboration
Roadmap
Strategy
Vineet Puranik

Vineet Puranik

Senior Engineering Manager at DocuSign

(Re)Organizing Your Teams Using Domain-Driven Design

12 July

A proposal for how to create an org structure that will deliver software systems that you want, not ones you get stuck with.

Alignment
Architecture
Scaling Team
Building A Team
Internal Communication
Reorganization
Ram Singh

Ram Singh

CTO at REAL Engagement & Loyalty

How to Navigate Your Manager Role at a New Company

1 July

Saikrishna Desaraju, Engineering Manager at Marks & Spencer, draws from his personal experience to advise new managers on thriving in their roles.

Managing Up
Managing Expectations
Leadership
Collaboration
New Manager Of Manager
Changing Company
Saikrishna Desaraju

Saikrishna Desaraju

Engineering Manager at Marks and Spencer