Back to resources

How to Increase Engagement Using the Right Metrics

Alignment
Mission / Vision / Charter
Conflict Solving
Convincing
Team Processes
Data Team

11 November, 2021

Richard Maraschi
Richard Maraschi

VP Data Product Management at WarnerMedia

Richard Maraschi, VP of Data Products and Insights at WarnerMedia, speaks on how he increased user engagement using measurements and experimentation.

Problem

When I began working at one of the world’s largest multinational TV news channels, it was going through a digital transformation to become a direct-to-consumer streaming media service . At that time, the key measure of performance was based on multi-platform audience reach, a measure of how many reached globally. However, this measure was flawed in that we were unable to measure whether our audience was actually engaged with our content. Furthermore we were unaware how frequently or for how long any one audience member would read, watch, or listen to different types of content. This focus on audience reach hindered our ability to pursue our mission statement: "to inform and empower the world." Not only that but, our journalists were incentivized to write stories that would get 'clicks,' or views, rather than those that would inform and empower. My team and I implemented a system of measurements that would improve our product while shifting the mindset of those opposed to metrics.

Actions taken

The first step taken in this process was to implement active and engaged user metrics our website. I created a plan that defined different user events and measurements that we were looking to track content consumptions, such as how far scrolled on a piece of media, how long spent consuming content within a session, , how fast an article was read, and how often users frequented the site. I shared this plan with my analytics and engineering teams, and it was added as a capability to our site. This information was applied across all of our media platforms, incorporating videos, articles, and audio clips. We transferred this information into a data warehouse where our analytics and product teams analyzed the data to understand our consumers' behavior patterns.

From here, we established a term called 'active users.' An active user was a consumer that viewed or interacted with a piece of content each day and frequented our site. The idea of active users became our North Star metric, and our goal was to time and frequency of active user sessions across all of our platforms (web, mobile and TV apps). To do this, we looked into the type of content our active users were viewing. We tried to determine if this was diversified content or traffic-heavy, click-based articles. We understood that for consumers to become active users, they needed to view and interact with evergreen content or something that would inform and empower them.

We ran experiments to identify product ideas that moved the needle for active users. We were watching for steps that would increase consumption, engagement, and attention for our active users. One of the steps taken in this process was creating a tab in our app that recommended personalized content. This was an area in our app dedicated to the individual. It specialized in suggesting reading choices related to users' behaviors and media that would diversify their everyday choices. This alone increased our user engagement by 27%.

We used this success as a proof point to evangelize the measures taken with our organization. There was some doubt from executives and product leaders that active users would improve our product more than traffic volume. We began shifting the minds of our organization, and it allowed us to make significant breakthroughs for our product.

Lessons learned

  • The biggest lesson is that you can not improve a product without measurement. Creating measurements provides a metric to focus on and streamlines your organization around processes that matter. As Peter Drucker said, "If you can't measure it, you can't manage it."
  • Some people think metrics inhibit creativity and in reverse how metrics benefit creativity or intuition. As long as there are different measures and frameworks, there should be thousands of ways to drive that metric.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

(Re)Organizing Your Teams Using Domain-Driven Design

12 July

A proposal for how to create an org structure that will deliver software systems that you want, not ones you get stuck with.

Alignment
Architecture
Scaling Team
Building A Team
Internal Communication
Reorganization
Ram Singh

Ram Singh

CTO at REAL Engagement & Loyalty

Team Development Framework for new managers

26 June

Individual Contributors are familiar with a technical development framework that helps them with building products. Managers, especially new managers can leverage a parallel framework to help them build their teams while drawing analogies from an already familiar framework.

Building A Team
Team Processes
New Manager
Viswa Mani Kiran Peddinti

Viswa Mani Kiran Peddinti

Sr Engineering Manager at Instacart

Managing Through a Team Reorganization

15 June

Mugdha Myers, former Engineering Manager at Google, discusses the challenges of leading a team through the ambiguity and anxiety caused by a large-scale team restructuring.

Alignment
Changing A Company
Strategy
Changing Company
Mugdha Myers

Mugdha Myers

Engineering Manager at N/A

Promoting Interdepartmental Teamwork for More Efficient Problem-Solving

13 June

Roland Fiala, Senior Vice President of Engineering at Productsup, raises an interesting issue about autonomy in teams: does it hinder collaboration opportunities that lead to better problem-solving? He shares his system for promoting teamwork in engineering departments.

Internal Communication
Collaboration
Roadmap
Team Processes
Cross-Functional Collaboration
Roland Fiala

Roland Fiala

Senior Vice President of Engineering at Usergems

How to Motivate Your Engineers to Grow in Their Careers

13 June

Roland Fiala, Senior Vice President of Engineering at Productsup, highlights the importance of soft skills and shares how he motivates his engineers to further their careers by focusing on personal growth.

Goal Setting
Different Skillsets
Handling Promotion
Personal Growth
Coaching / Training / Mentorship
Motivation
Team Processes
Career Path
Performance
Roland Fiala

Roland Fiala

Senior Vice President of Engineering at Usergems