We've just launched plato for individuals

🔥

login


Google Sign inLinkedIn Sign in

Don't have an account? 

User Testing to Minimize Time Spent on Arguing Conceptual Ideas

Data team
Feedback
Users Feedback

10 May, 2019

Meghan Cochran, VP of Product at Tulip Cremation, discusses what she did to minimize recurring disputes that tend to occur when everyone has a slightly different opinion about product.

Problem

The problem was multiple people across the org felt that they knew what would be better for our customers on our key landing pages, but they didn't agree or have data to support. Most of the suggestions were based on theoretical assumptions of what they liked or had heard from a small sample of customers. Being a small company, we couldn't rely on quick A|B testing to help us end these debates, like Facebook can for example, so we needed to find a better way to make UX decisions.

Actions taken

It sounds obvious that the answer was to ask customers, but in our business we try not to bother customers who have often just recently lost a loved one. So, the next best option was to get a subscription to https://www.usertesting.com and run a series of studies to put the site in front of perspective customers and get their feedback directly.
In user testing, you usually do five people at a time because you're looking for errors, not sentiment. So, in order to get a broader set of users, I used the same test with a variety of user segments. I would take five men, five women, five on mobile, and five on desktop, and then ask them all to respond to, and rank the UX, on a particular measure like "trust" or "affordability", whatever the question was we were trying to solve for. From there, I was able to see enough trends in reactions, and how those reactions affected the way they ranked the UX on those attributes. From there, I could inform our stakeholders from an informed perspective. It doesn't provide the quantitative value of an A|B test, but does yield valuable qualitative data to inform the decision.

Lessons learned

  • Often, arguments about conceptual ideas don't bring people together. In fact, it can cause people to dig in their heels on their perspective. In the end, it doesn't matter who was right & who was wrong; what matters is how well we're able to predict how customers will respond. These arguments still happen, but now we have a way to keep the emotion out and shift the discussion to "what do we want to know".
  • Implementing user research takes both time and money, but it takes less of both than making bad decisions and only learning through implementation. I can launch a study on user testing at 5 p.m. on a Tuesday and by Wednesday at noon have usable data.
  • Publishing the study results helps to ground everyone in the experience of real users, rather than hoped-for results. It helps everyone to hear the words of real people who are interacting with our products and understand that those reactions are both varied and unexpected.

Related stories

Architecture Council: An Idea for Advancing Organizational Strategic Orientation
28 August

Brad Henrickson, CTO at Scoop, shares how by establishing the Architecture Council he advanced strategic thinking of the engineering team and overall strategic orientation of his organization.

Impact
Productivity
Feedback
Brad Henrickson

Brad Henrickson

CTO at Scoop

Delivering Negative Feedback Effectively
2 August

Shelly Bezanson, Director of Release at Thoughtexchange, explains how she would approach an uncomfortable situation and effectively deliver negative feedback.

Feedback
Shelly Bezanson

Shelly Bezanson

Director of Release at Thoughtexchange

Building an Efficient Data Science Team While Still Being Agile
28 July

Arun Krishnaswamy, Director of Data Science at Workday, describes how to build a data science team emphasizing the difference between software development lifecycle and data science methodology.

Agile / Scrum
Data team
Arun Krishnaswamy

Arun Krishnaswamy

Director at Workday

Why Psychological Safety Is More Important Than Ever During the Covid-19 Pandemic?
30 June

Murali Bala, Director, Software Engineering at Capital One, discusses the importance of psychological safety emphasizing its unparalleled significance during the Covid-19 pandemic.

Remote
Feedback
Team reaction
Feelings aside
Company Culture
Internal Communication
Murali Bala

Murali Bala

Director, Software Engineering at Capital One

Start With the Customer and Work Backwards
30 June

Murali Bala, Director, Software Engineering at Capital One, shares his experience of building a guest relationship management tool by embracing the customer-first mindset and working backward.

Users Feedback
Product
Murali Bala

Murali Bala

Director, Software Engineering at Capital One

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.