Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Yulya Besplemennova
[Demo] Stress-testing GenAI in user research synthesis
2024 • Designing with AI 2024
Gold
Boon Yew Chew
Making Sense of Systems—and Using Systems to Make Sense of the Enterprise
2023 • Enterprise UX 2023
Gold
Malini Rao
Lessons Learned from a 4-year Product Re-platforming Journey
2021 • Design at Scale 2021
Gold
Josh Clark
Sentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)
2025 • Rosenfeld Community
Brendan Jarvis
Framing Tomorrow by Questioning Today
2022 • Design at Scale 2022
Gold
Wendy Johansson
Be a Product Boss!
2022 • Design in Product 2022
Gold
Jane Davis
Strategic Shifts and Innovations in User Research: Navigating Challenges and Opportunities
2025 • Advancing Research 2025
Gold
Noah Bond
Redefining truth and inclusivity: Navigating data ownership and ethical research in the age of disinformation
2025 • Advancing Research 2025
Gold
Nick Cochran
Growing in Enterprise Design through Making Connections
2019 • Enterprise Community
Greg Petroff
Exit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian
2025 • Rosenfeld Community
Jorge Arango
Scale Smart: AI-Powered Content Organization Strategies
2024 • DesignOps Summit 2024
Gold
Greg Petroff
Everything is About to Change: Software as Material
2016 • Enterprise UX 2016
Gold
Nathan Curtis
Beyond the Toolkit: Spreading a System Across People & Products
2016 • Enterprise UX 2016
Gold
Sara Conklin
A UXer’s 12-Month Journey from Climate Concern to Climate Credibility
2025 • Climate UX Interest Group
Marisa Bernstein
It Takes GRIT: Lessons from the Small, but Mighty World of Civic Usability Testing
2021 • Civic Design 2021
Gold
Maria Skaaden
Continuous Design: One eye on the horizon and the other on the next wave
2018 • DesignOps Summit 2018
Gold

More Videos

Daniel Gloyd

"Meisner taught actors to respond truthfully in the moment, prioritizing presence over preconceived ideas."

Daniel Gloyd

Designing From the Inside Out: How Method Acting Can Inspire Design Research

February 12, 2026

Deirdre Hirschtritt

"We’re not probing for interesting anecdotes or seating quotes for a slide deck. We’re answering real questions to provide actionable guidance."

Deirdre Hirschtritt Cesar Paredes Marie Perrot

Research is Only as Good as the Relationships You Build

November 17, 2022

Benjamin Real

"We finally instead of using the compass and the map we finally were able to build a GPS."

Benjamin Real

Maturity Models: A Core Tool for Creating a DesignOps Strategy

October 1, 2021

Max Gadney

"It’s really important to understand your own mental bandwidth and not take on too much even if the project excites you."

Max Gadney Andrea Petrucci Joshua Stehr Hannah Wickes

Assessing UX jobs for impact in climate

August 14, 2024

Lada Gorlenko

"If they can do it, so can I – that’s the takeaway from these transformation stories."

Lada Gorlenko

Theme 1: Intro

January 8, 2024

Feyikemi Akinwolemiwa

"I asked myself, will I take the leap or will I take permission? And this gamble paid off."

Feyikemi Akinwolemiwa

Play to innovate: How curiosity and experimentation transform UX

March 11, 2026

Bria Alexander

"Cohorts are a super duper fun way to engage with more of our community."

Bria Alexander

Opening Remarks

March 29, 2023

Llewyn Paine

"Ultimately, AI avatar tools offer UX teams options to keep recordings while meeting legal compliance."

Llewyn Paine

[Demo] Deploying AI doppelgangers to de-identify user research recordings

June 5, 2024

Bria Alexander

"Junior engineers often resent having to create OKRs that feel disconnected from their day-to-day work."

Bria Alexander Benson Low Natalya Pemberton Stephanie Goldthorpe

OKRs—Helpful or Harmful?

January 20, 2022