Rosenverse
How to Lead With Data, and Without Data

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Saara Kamppari-Miller
Theme Three Intro
2022 • DesignOps Summit 2022
Gold
Ellen Chisa
The Values of Design
2023 • Design in Product 2023
Gold
Alana Washington
(Remote) Service Design: A Transformation Case Study
2022 • Design at Scale 2022
Gold
Lisa Spitz
Building Trust Through Equitable Research Practices
2022 • Civic Design 2022
Gold
Corey Long
Hiring in DesignOps: A Critical Study on How to Hire and Get Hired
2024 • DesignOps Summit 2024
Gold
John Paul de Guzman
10k Screens Later: How We Became a Data-Driven Design Organization
2024 • DesignOps Summit 2024
Gold
George Abraham
Design Systems To-Go: Introducing a Starter Design System, and Indigo.Design Overview (Part 1)
2021 • DesignOps Summit 2021
Gold
Katie Johnson
Disrupting generative AI products with just-in-time consumer insights
2024 • Designing with AI 2024
Gold
Shivanjali M.
From Fragmented to Fluent: The Linguistic Approach to Research Knowledge
2026 • Rosenfeld Community
Gordon Ross
12 Months of COVID-19 Design and Digital Response with the British Columbia Government
2021 • Civic Design 2021
Gold
Prerna Makanawala
Achieving Balanced Design Consistency
2021 • Design at Scale 2021
Gold
Nathan Shedroff
Double Your Mileage: Use Your Research Strategically
2020 • Advancing Research 2020
Gold
Anna Nguyen
Why Our Voice of the Customer is Better Than Yours
2022 • Advancing Research 2022
Gold
Bria Alexander
Opening Remarks
2022 • Civic Design 2022
Gold
Adam Thomas
Survival Metrics – Making Change in a Fast, Data-Informed, and Politically Safe Way
2022 • Design in Product 2022
Gold
Angelos Arnis
Navigating the Rapid Shifts in Tech's Turbulent Terrain
2023 • DesignOps Summit 2023
Gold

More Videos

Jilanna Wilson

"Cameras on in Zoom meetings is a small act with a powerful impact against isolation."

Jilanna Wilson

Distributed DesignOps Management

February 26, 2019

Jaskiran Kang

"If the department is dead serious about delivering services, these design imperatives are absolutely imperative."

Jaskiran Kang

Why Community is Key to Professionalizing Design

October 28, 2022

Nicole Wright

"We had 1600 tags in our system at one point, which was overwhelming, so getting taxonomies aligned was a game changer."

Nicole Wright Ned Dwyer

Democratizing Research at HoneyBook

March 9, 2022

Sam Proulx

"Having an assistant when presenting can remove distractions and let you focus on delivering your message."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

March 10, 2022

Victor Udoewa

"The community owns the artifacts and outcomes, as well as the narratives around those outcomes."

Victor Udoewa

Radical Participatory Research: Decolonizing Participatory Processes

March 9, 2022

Jemma Ahmed

"Collaborative journey mapping is about users building out their journey together in primary research using recall and market research techniques."

Jemma Ahmed

Convergent Research Techniques in Customer Journey Mapping

March 31, 2020

Tom Armitage

"There’s a reason we’re always taught to write notes or when we revise, we write notes out by hand, the hand remembers."

Tom Armitage Carla Diana Kanene Ayo Holder

Day 2 Panel: Looking ahead: Designing with AI in 2026

June 11, 2025

Florence Okoye

"I desperately needed a specifically Black centered ethos of design to avoid being a pair of hands contributing to imperialistic white supremacist capitalist patriarchy."

Florence Okoye

AfroFuturism and UX Research

March 27, 2023

Taylor Jennings

"When everyone is taking on research as their role, you immediately have friends who understand what you do."

Taylor Jennings Alexis McNutt Unis Sydney Lawson

Research Debate Club

March 11, 2026