Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Cennydd Bowles
Exit Interview #2: Rediscovering the ethical heart of design
2025 • Rosenfeld Community
Jon Fukuda
Design Planning and Management Support
2023 • DesignOps Summit 2023
Gold
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold
Craig Villamor
Design Systems for Ethical Design
2023 • Enterprise Community
Louis Rosenfeld
Do you want to work on climate? (Climate UX Discussion Series)
2023 • Climate UX Interest Group
Mark Interrante
Collaboration Flows in Product Development
2017 • Enterprise Experience 2017
Gold
Tamara Hale
War Stories LIVE! Tamara Hale
2020 • Advancing Research 2020
Gold
Patrizia Bertini
Designing Within the Lines: How the EU AI Act Can Spark Better AI Innovation
2025 • DesignOps Community
Jennifer Kanyamibwa
Creating the Blueprint: Growing and Building Design Teams
2018 • DesignOps Summit 2018
Gold
Kit Unger
Theme 1 Intro
2022 • Design at Scale 2022
Gold
Victor Udoewa
Radical Participatory Research: Decolonizing Participatory Processes
2022 • Advancing Research 2022
Gold
Greg Petroff
Exit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian
2025 • Rosenfeld Community
Eduardo Ortiz
Day 3 Theme Panel
2025 • Advancing Research 2025
Gold
Crystal Philcox
The Many Faces of Operations
2017 • DesignOps Summit 2017
Gold
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold
Jorge Arango
Design as an Antidote to VUCA
2019 • Enterprise Community

More Videos

Sheri Byrne-Haber

"Keeping something cloud-based accessible is the most complicated because of hundreds of releases per day in continuous integration."

Sheri Byrne-Haber

Accessibility at Scale

June 9, 2021

Prayag Narula

"Feature bloat kills usability because it clouds the core value proposition."

Prayag Narula Hannah Hudson

Empowering Designers to do Good Research

March 11, 2022

Janelle Estes

"Being open and flexible to what's most helpful for the team at that moment is key."

Janelle Estes

UX Research Trends

January 28, 2021

Craig Brookes

"Finding a leadership champion and building on small wins creates momentum for integrating research and content."

Craig Brookes Andreas Huebner Morgan Quinn

"Just Make it Look Good" and Other Ways We're Misunderstood

June 11, 2021

Marc Fonteijn

"Slow and intentional communication allows you to be fully present rather than distracted by constant notifications."

Marc Fonteijn Ru Butler

Increase your confidence, influence, and impact (through a Professional Community)

December 3, 2024

Kate Towsey

"Managing vendors is a full-time job, and research ops teams often coordinate numerous vendors for tools and recruitment."

Kate Towsey

The State of ResearchOps: More Than Just Theory

June 20, 2019

Alla Weinberg

"Creating safety requires specific, behavioral agreements you can observe and practice."

Alla Weinberg

Design Teams Need Psychological Safety: Here’s How to Create It

September 9, 2022

Shawna Hein

"Slack channels and weekly meetings encourage cross-contract collaboration, avoiding duplication and sharing learnings across teams."

Shawna Hein Kevin Hoffman

Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts

November 17, 2022

Tricia Wang

"You have to find mentors and sponsors who leverage their privilege to help you get as far as possible."

Tricia Wang

SCALE: Discussion

June 15, 2018