Back to resources

Managing Expectations on a Highly Visible Project

Managing Expectations
Internal Communication
Stakeholders

9 August, 2021

Ramayan Tiwari
Ramayan Tiwari

Senior Software Engineer at Netflix

Ramayan Tiwari, Senior Software Engineer at Netflix, details how to manage expectations on a project that attracted significant interest from a number of stakeholders even before its official start.

Problem

I started a brand new project, which was much in demand even before its official start. The problem we were trying to solve was exceedingly important to the business, and many teams were interested in using our project. The craze kicked off even before we began articulating the idea and coming up with the design.

We tried to improve the Netflix recommendation system and make it more real-time and take into account the browsing habits of our users. When not playing video content, our users would navigate the home page, engage with a few rows, play trailers, look for specific titles, using thumbs up/down to rate content, etc. All those activities were important because they could tell us what a user’s intent was. If we could further ramp up our recommendation and personalization system, it would make an even more satisfying user experience.

From a data perspective, we were collecting the data that would help us understand what users were doing in real-time and capture their interactions over a longer period of time. We could go back for a month or two to learn from their activities, personalizing the system accordingly. Apparently, that was critically important to different business units, and no wonder that a number of teams, including the search, recommendation, page construction, and ranking team, were interested. The challenge I found particularly difficult was how to manage expectations given the excitement surrounding the project from the start and furthermore, deliver the features timely without compromising on quality.

Actions taken

For starters, I had to understand what were the most common use cases. To do so, I talked to a number of consumers, meeting with them one-on-one. I would also meet with other teams to get precise requirements for the projects. I would ask them very specific questions such as, How will you query the data or How will your systems integrate with the new system? I also wanted to learn about their priorities. If a particular access pattern was important to them, was it more important than the other one? I wanted to understand and establish a clear order of priorities early on. That helped me plan my milestones so that I delivered the most important features first.

Next, I had to find a balance between the most important use cases. I wanted to make sure that the use cases I was targeting in the V0 or V1 release, included use cases coming from a variety of teams and users from whom I could get feedback. V1 should be good enough to address most use cases so that people could onboard in the system and help improve it by providing feedback.

We started to build a proof of concept early on, which was somewhat unique. We would begin with a smaller project that would be easy to build but could provide users with some concrete data about what the full-scale project would look like. We built it and demoed it to our users to see what they were thinking. When one is in the early stages of a project, most of the time, one talks hypothetically: I will do this or that. Not surprisingly, users may have different interpretations of what that should be. It’s important to be on the same page from the expectations standpoint from the start.

We build a proof of concept application in a few weeks time and then host a demo for a larger audience. We explained how things would look like, how fast it would be, what data could be captured, and made available to the consumers with acceptable latencies. We asked them for their feedback explicitly: Is this the direction you had in mind? How far is this from your expectations? Based on their feedback, we formalized what V0 should mean in terms of requirements, specifications, and timeline.

We also wanted to regularly show progress to our users. Since everything was taking place after the Covid pandemic hit, demoing through a screen was the only option. I missed going to users in person and collecting their feedback chatting in the hallway. We were demoing to a larger group at least once a month, frequently doing demos to different teams weekly.

Doing demos was a great opportunity to get users interested in the project. We also built a tool that could do some visual demos. If a demo cannot be visually interpreted, it would be less appealing. When someone is producing data as a project, they should find a way to make it more attractive. Building some UI or visualization tool that could take that raw data and present it visually is critical to draw interest. We enabled users to see what was being created to be visible on their Netflix screens.

By moving from the early idea to a more mature product, we also moved from managing expectations to managing adoption. You can build something great, but your effort will be futile if no one is using it.

Lessons learned

  • We were trying out a new technology that was not mature and supported enough. Alternatives were simply not a good fit for our use case. We could have used three other available and well-supported technologies, but they were not a great fit for this particular project. We started using this new technology, and of course, as with any new technology, we expected a lot of hiccups, especially when operated on a large scale. If you are going with a less adopted solution, consider the failure domain and build tooling early on to mitigate failures. For example, when a catastrophic failure happens in your data storage, you should be able to build that state back. In this process, we helped improve the new technology around its stability at a larger scale.

  • We built the system for scale from day one. We didn’t launch this project to build it for 1 percent of users and then slowly increase to 10 and 50 percent. The reason is simple: we are building a data store that all users will use. If it gets adopted and people want to onboard, we don’t want to put them on hold for a quarter or two until we try a new system that can scale. We launched this system with 100 percent traffic from the early on. The scale is the most important requirement; if it doesn’t scale, it would be a failure no matter how good the data is.

  • Start your integrations early on. Most often, companies have multiple services and libraries that consume each other to produce a complete and cohesive system. If you are going to introduce a new system, talk to people who will be either producers or consumers of the system and make sure they can integrate. If you do this integration early on, you will create a feedback loop, which will help you understand which parts of the system you will not be able to integrate.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

Myth Busting

10 December

Supporting principles on why being data led (not driven) helps with the story telling.

Alignment
Managing Expectations
Building A Team
Leadership
Collaboration
Productivity
Feedback
Psychological Safety
Stakeholders
Vikash Chhaganlal

Vikash Chhaganlal

Head of Engineering at Xero

DevSecOps: Why, Benefits and Culture Shift

29 November

Why DevSecOps matter and what's really in it for you, the team and the organisation?

Innovation / Experiment
Building A Team
Leadership
Ownership
Stakeholders
Cross-Functional Collaboration
Vikash Chhaganlal

Vikash Chhaganlal

Head of Engineering at Xero

The Growth Mindset in Modern Product Engineering

28 November

The impact you can have with a Growth Mindset' and the factors involved in driving orchestrated change.

Building A Team
Leadership
Collaboration
Feedback
Ownership
Stakeholders
Vikash Chhaganlal

Vikash Chhaganlal

Head of Engineering at Xero

High Performance Team in Action

13 October

A high performance team refers to “ a group of goal-focused individuals with specialized expertise and complementary skills who collaborate, innovate and produce consistently superior results.”

Managing Expectations
Building A Team
Company Culture
Feedback
Coaching / Training / Mentorship
Praveen Cheruvu

Praveen Cheruvu

Senior Software Engineering Manager at Anaplan

How to Maintain Happiness: The Underrated Aspect of Creating Team Dynamic

2 August

Jonathan Ducharme, Engineering Manager at AlleyCorp Nord, encourages the importance of a workplace environment that cultivates mental wellness.

Personal Growth
Company Culture
Leadership
Internal Communication
Psychological Safety
Jonathan Ducharme

Jonathan Ducharme

Engineering Manager at AlleyCorp Nord