Rosenverse
How to Lead With Data, and Without Data

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Feleesha Sterling
Building a Rapid Research Program
2023 • Enterprise Community
Ben Davies
Expert Panel: The Principles of Research Repository Design
2022 • Advancing Research 2022
Gold
Alexia Cohen
Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona
2024 • Advancing Service Design 2024
Gold
Joanna Vodopivec
One Research Team for All - Influence Without Authority
2022 • Advancing Research 2022
Gold
Sam Proulx
Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
2022 • DesignOps Summit 2022
Gold
Billy Carlson
Pro-level UI Tips for Beginners
2022 • DesignOps Summit 2022
Gold
Jane Davis
Strategic Shifts and Innovations in User Research: Navigating Challenges and Opportunities
2025 • Advancing Research 2025
Gold
Jay Bustamante
Navigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation
2023 • DesignOps Summit 2023
Gold
Michael Polivka
Scaling Design through Relationship Maps
2017 • DesignOps Summit 2017
Gold
Fisayo Osilaja
[Demo] The AI edge: From researcher to strategist
2024 • Designing with AI 2024
Gold
Dan Hill
Designing for the infrastructures of everyday life
2024 • Designing with AI 2024
Gold
Sam Proulx
Mobile Accessibility and You
2022 • Design at Scale 2022
Gold
Jemma Ahmed
Convergent Research Techniques in Customer Journey Mapping
2020 • Advancing Research 2020
Gold
Noah Bond
Redefining truth and inclusivity: Navigating data ownership and ethical research in the age of disinformation
2025 • Advancing Research 2025
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Nidhi Singh Rathore
Embracing participation to unlock deeper truths in commercial research
2025 • Advancing Research 2025
Gold

More Videos

Alexia Cohen

"Prototypes were very low fidelity because too-high fidelity can intimidate participants and hinder feedback."

Alexia Cohen Adriane Ackerman

Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona

December 4, 2024

Marc Majers

"Interruptions reveal things like auto-save that you never thought about before."

Marc Majers Tony Turner

Interrupted UX - Add A Dose of Reality To Usability Testing

March 11, 2022

Jorge Arango

"Design is a third way of knowing, different from science and humanities, involving making and artifice."

Jorge Arango

Design as an Antidote to VUCA

May 9, 2019

Brenna Fallon

"Never forget the individual. People are less predictable than algorithms, but they’ll get you through change."

Brenna Fallon

Learning Over Outcomes

October 24, 2019

Daniel Gloyd

"We as humans value warmth information in others more than competence information."

Daniel Gloyd

Warming the User Experience: Lessons from America's first and most radical human-centered designers

May 9, 2024

Sam Proulx

"Confidence is critical in online shopping because nobody wants to spend money they’re unsure about losing due to unclear controls."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023

Kristin Skinner

"We are not replacing teachers; we are enhancing their ability to teach effectively."

Kristin Skinner

Five Years of DesignOps

September 29, 2021

Denise Jacobs

"It can be so tiring to be the only one constantly saying this has to be done differently."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Roberta Dombrowski

"Recruit fees from panels can escalate from $100 per participant to $600 due to international and professional targeting."

Roberta Dombrowski Lianna Aduana

5 Reasons to Bring your Recruiting in House

September 30, 2021