Rosenverse
Driving Change with CX Metrics

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • •

    MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • •

    They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • •

    Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • •

    Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • •

    Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • •

    Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • •

    The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • •

    Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • •

    Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • •

    Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Bruce Gillespie
Learning from journalism: Balancing impactful communication with compassionate storytelling
2025 • Advancing Research 2025
Gold
Audrey Crane
Shadow Design–Where Else is Design Happening in Your Organization?
2023 • Enterprise Community
Louis Rosenfeld
About the Rosenverse
2026 • Rosenfeld Community
Yulya Besplemennova
[Demo] Stress-testing GenAI in user research synthesis
2024 • Designing with AI 2024
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • DesignOps Summit 2023
Gold
Peter Merholz
Customer-Centered Design Organizations
2017 • Enterprise Experience 2017
Gold
Kit Unger
Theme 3 Intro
2022 • Design at Scale 2022
Gold
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • DesignOps Summit 2021
Gold
Lukas Moro
“Feels Like Paper!”: Interfacing AI through Paper
2025 • Designing with AI 2025
Gold
Jason Mesut
Shaping design, designers and teams
2018 • DesignOps Summit 2018
Gold
Bud Caddell
DesignOps Community Sensing Session
2021 • DesignOps Community
Erin Weigel
Testing and Experimentation Tools
2026 • Advancing Research 2026
Gold
Samuel Proulx
Inclusive Research: Debunking Myths and Getting Started
2025 • Advancing Research 2025
Gold
Megan Campos
What Did I Miss? The Hidden Costs of Deprioritizing Diversity in User Research
2021 • Advancing Research 2021
Gold
Neil Barrie
Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture
2024 • Advancing Research 2024
Gold
Jen Cardello
Learning Velocity—The Insights Speedometer
2021 • Advancing Research Community

More Videos

Mark Interrante

"Prioritization work is a political process, and politics are hard to do in technology."

Mark Interrante Harry Max

AI for Prioritization (3rd of 3 seminars)

July 11, 2024

Patrizia Bertini

"Building product today is getting more and more complex, and organizations desperately need professionals who speak compliance and product."

Patrizia Bertini

The (r)evolution of designOps: It’s Time to Think (really) BIG

September 11, 2025

Shan Shen

"A tree test focuses on intuitive navigation while a prototype aims for the desired modality of an end solution."

Shan Shen

Translating UX Terms into Business Contexts

November 29, 2023

Adam Thomas

"Stop, pivot, or invest early—these are the survival metric decisions that save companies."

Adam Thomas

Survival Metrics – Making Change in a Fast, Data-Informed, and Politically Safe Way

December 6, 2022

Robin Beers

"Ubuntu means I am because we are — we don’t really know our own humaneness unless it’s in relationship to other people."

Robin Beers

Research as a Catalyst for Organizational Transformation

March 12, 2021

Sheri Byrne-Haber

"Keeping something cloud-based accessible is the most complicated because of hundreds of releases per day in continuous integration."

Sheri Byrne-Haber

Accessibility at Scale

June 9, 2021

Joseph Meersman

"People were sweating their pixels, but doing so inside of solos and personal barriers until we started critiquing transparently together."

Joseph Meersman

Sweating the Pixel: Scaling Quality through Critique

June 10, 2021

Deanna Washington

"Just because empathy isn’t always visible at the top doesn’t mean you can’t be the change."

Deanna Washington Bria Alexander Jon Fukuda Saara Kamppari-Miller Farid Sabitov

Connecting the Ops: Plenary Panel and Closing Circle

September 9, 2022

Aaron Stienstra

"We built an equity learning community with 25 virtual sessions, 4 interagency roundtables, and one-on-one consultations with 37 agencies."

Aaron Stienstra Lashanda Hodge

Leveraging Civic Design to Advance Equity and Rebuild Trust in the US Federal Government

December 8, 2021