Back to resources

Managing Expectations on a Highly Visible Project

Managing Expectations
Internal Communication
Stakeholders

9 August, 2021

Ramayan Tiwari
Ramayan Tiwari

Senior Software Engineer at Netflix

Ramayan Tiwari, Senior Software Engineer at Netflix, details how to manage expectations on a project that attracted significant interest from a number of stakeholders even before its official start.

Problem

I started a brand new project, which was much in demand even before its official start. The problem we were trying to solve was exceedingly important to the business, and many teams were interested in using our project. The craze kicked off even before we began articulating the idea and coming up with the design.

We tried to improve the Netflix recommendation system and make it more real-time and take into account the browsing habits of our users. When not playing video content, our users would navigate the home page, engage with a few rows, play trailers, look for specific titles, using thumbs up/down to rate content, etc. All those activities were important because they could tell us what a user’s intent was. If we could further ramp up our recommendation and personalization system, it would make an even more satisfying user experience.

From a data perspective, we were collecting the data that would help us understand what users were doing in real-time and capture their interactions over a longer period of time. We could go back for a month or two to learn from their activities, personalizing the system accordingly. Apparently, that was critically important to different business units, and no wonder that a number of teams, including the search, recommendation, page construction, and ranking team, were interested. The challenge I found particularly difficult was how to manage expectations given the excitement surrounding the project from the start and furthermore, deliver the features timely without compromising on quality.

Actions taken

For starters, I had to understand what were the most common use cases. To do so, I talked to a number of consumers, meeting with them one-on-one. I would also meet with other teams to get precise requirements for the projects. I would ask them very specific questions such as, How will you query the data or How will your systems integrate with the new system? I also wanted to learn about their priorities. If a particular access pattern was important to them, was it more important than the other one? I wanted to understand and establish a clear order of priorities early on. That helped me plan my milestones so that I delivered the most important features first.

Next, I had to find a balance between the most important use cases. I wanted to make sure that the use cases I was targeting in the V0 or V1 release, included use cases coming from a variety of teams and users from whom I could get feedback. V1 should be good enough to address most use cases so that people could onboard in the system and help improve it by providing feedback.

We started to build a proof of concept early on, which was somewhat unique. We would begin with a smaller project that would be easy to build but could provide users with some concrete data about what the full-scale project would look like. We built it and demoed it to our users to see what they were thinking. When one is in the early stages of a project, most of the time, one talks hypothetically: I will do this or that. Not surprisingly, users may have different interpretations of what that should be. It’s important to be on the same page from the expectations standpoint from the start.

We build a proof of concept application in a few weeks time and then host a demo for a larger audience. We explained how things would look like, how fast it would be, what data could be captured, and made available to the consumers with acceptable latencies. We asked them for their feedback explicitly: Is this the direction you had in mind? How far is this from your expectations? Based on their feedback, we formalized what V0 should mean in terms of requirements, specifications, and timeline.

We also wanted to regularly show progress to our users. Since everything was taking place after the Covid pandemic hit, demoing through a screen was the only option. I missed going to users in person and collecting their feedback chatting in the hallway. We were demoing to a larger group at least once a month, frequently doing demos to different teams weekly.

Doing demos was a great opportunity to get users interested in the project. We also built a tool that could do some visual demos. If a demo cannot be visually interpreted, it would be less appealing. When someone is producing data as a project, they should find a way to make it more attractive. Building some UI or visualization tool that could take that raw data and present it visually is critical to draw interest. We enabled users to see what was being created to be visible on their Netflix screens.

By moving from the early idea to a more mature product, we also moved from managing expectations to managing adoption. You can build something great, but your effort will be futile if no one is using it.

Lessons learned

  • We were trying out a new technology that was not mature and supported enough. Alternatives were simply not a good fit for our use case. We could have used three other available and well-supported technologies, but they were not a great fit for this particular project. We started using this new technology, and of course, as with any new technology, we expected a lot of hiccups, especially when operated on a large scale. If you are going with a less adopted solution, consider the failure domain and build tooling early on to mitigate failures. For example, when a catastrophic failure happens in your data storage, you should be able to build that state back. In this process, we helped improve the new technology around its stability at a larger scale.

  • We built the system for scale from day one. We didn’t launch this project to build it for 1 percent of users and then slowly increase to 10 and 50 percent. The reason is simple: we are building a data store that all users will use. If it gets adopted and people want to onboard, we don’t want to put them on hold for a quarter or two until we try a new system that can scale. We launched this system with 100 percent traffic from the early on. The scale is the most important requirement; if it doesn’t scale, it would be a failure no matter how good the data is.

  • Start your integrations early on. Most often, companies have multiple services and libraries that consume each other to produce a complete and cohesive system. If you are going to introduce a new system, talk to people who will be either producers or consumers of the system and make sure they can integrate. If you do this integration early on, you will create a feedback loop, which will help you understand which parts of the system you will not be able to integrate.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

The Art of Asking Why: Narrowing the Gap Between Customers and Users

24 May

Jord Sips, Senior Product Manager at Mews, shares his expertise on a common challenge for product managers – finding root causes and solutions.

Customers
Innovation / Experiment
Product
Personal Growth
Leadership
Stakeholders
Users
Jord Sips

Jord Sips

Senior Product Manager at Mews

Managing Different Time Zones: Inclusive Collaboration Methods

19 May

Jonathan Belcher, Engineering Manager at Curative, shares an unknown side of synchronous communication tools and advises managers on how to handle a team that’s spread across the globe.

Remote
Internal Communication
Collaboration
Cross-Functional Collaboration
Jonathan Belcher

Jonathan Belcher

Engineering Manager - Patient Experience at Curative

Managing Remotely: Balancing Team Cohesion and Focus Time

26 May

Jonathan Belcher, Engineering Manager at Curative, explains how to balance team cohesion and individual focus time, tapping into his experiences of working remotely for seven years.

Remote
Micromanagement
Meetings
Internal Communication
Productivity
Psychological Safety
Performance
Jonathan Belcher

Jonathan Belcher

Engineering Manager - Patient Experience at Curative

Streamlining Product Processes After a Reorganization

16 May

Snehal Shaha, Lead Technical Program Manager at Momentive (fka SurveyMonkey), details her short-term technical strategy to unify processes among teams following an acquisition.

Acquisition / Integration
Product Team
Product
Building A Team
Leadership
Internal Communication
Collaboration
Reorganization
Strategy
Team Processes
Cross-Functional Collaboration
Snehal Shaha

Snehal Shaha

Senior EPM/TPM at Apple Inc.

Navigating Disagreements When It Comes to Priorities

9 May

Pavel Safarik, Head of Product at ROI Hunter, shares his insights on how to deal with disagreements about prioritization when building a product.

Innovation / Experiment
Product Team
Product
Dev Processes
Conflict Solving
Internal Communication
Collaboration
Convincing
Strategy
Prioritization
Pavel Safarik

Pavel Safarik

Head of Product at ROI Hunter

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.