Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Nathan Curtis
Beyond the Toolkit: Spreading a System Across People & Products
2016 • Enterprise UX 2016
Gold
Scott Stephens
The Next Generation in DesignOps Toolsets
2022 • DesignOps Community
Mackenzie Guinon
M.C. Escher’s UX Research Career Ladder
2022 • Advancing Research 2022
Gold
Louis Rosenfeld
Becoming a Civic Designer: Making the Move from Private to Public Sector
2022 • Civic Design 2022
Gold
Joshua Graves
We Need To Talk: Navigating Conversations with Your Boss (Part 1 of 3)
2025 • Rosenfeld Community
Surya Vanka
Unleashing Swarm Creativity to Solve Enterprise Challenges
2021 • Design at Scale 2021
Gold
Natalie Dunbar
DesignOps and Content Strategy: Envisioning the Future Together
2021 • DesignOps Summit 2021
Gold
Corey Long
Hiring in DesignOps: A Critical Study on How to Hire and Get Hired
2024 • DesignOps Summit 2024
Gold
Sara Asche Anderson
Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy
2024 • Enterprise Experience 2020
Gold
Gina Mendolia
Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems
2024 • Advancing Service Design 2024
Gold
Bruce Gillespie
Learning from journalism: Balancing impactful communication with compassionate storytelling
2025 • Advancing Research 2025
Gold
Denise Jacobs
Interactive Keynote: Social Change by Design
2024 • Enterprise Experience 2020
Gold
Nicole Aleong
Future Orientations to Everyday Life: Futures Anthropology as a Methodology
2024 • Advancing Research 2024
Gold
Meghan Bausone
Systems Thinking and Design Innovation: Working with Leverage Points in Rural Maternal Health Systems
2026 • Rosenfeld Community
Deanna Smith
Leading Change with Confidence: Strategies for Optimizing Your Process
2024 • DesignOps Summit 2024
Gold
Sol Mesz
Hands or Brains? How to Hire for Strategy, Strategically
2024 • Enterprise Experience 2020
Gold

More Videos

Jim Kalbach

"There are no mistakes in jazz, just missed opportunities."

Jim Kalbach

Jazz Improvisation as a Model for Team Collaboration

November 6, 2017

Louis Rosenfeld

"We look to work with authors we like and enjoy being around because writing a book is a long, collaborative journey."

Louis Rosenfeld

Coffee with Lou: Should You Write a (UX) Book?

March 7, 2024

Catt Small

"The difference between director and principal often lies in whether you’re inspired more by leading people or focused on outputs and craft."

Catt Small Micah Bennett Brian Carr Jessica Harllee

What's Next for ICs: Exploring Staff and Principal Designer Roles

February 22, 2024

Marieke McCloskey

"Finding out more about these most active users is a key priority this year, said our CEO at the board presentation."

Marieke McCloskey

User Science: Product Analytics & User Research

March 11, 2021

Llewyn Paine

"Ultimately, AI avatar tools offer UX teams options to keep recordings while meeting legal compliance."

Llewyn Paine

[Demo] Deploying AI doppelgangers to de-identify user research recordings

June 5, 2024

Joshua Noble

"Qualitative research really poses questions; quantitative methods let you investigate those questions rigorously."

Joshua Noble

Casual Inference

October 6, 2023

Sara Logel

"Storytelling helps gain people’s attention, build empathy, and improve understanding and recall of key points."

Sara Logel

Your Colleagues are Your Users Too

March 29, 2023

Bria Alexander

"If something makes you feel unwelcome, the code of conduct explains how to engage with staff to resolve issues."

Bria Alexander Louis Rosenfeld

Welcome

January 8, 2024

Sam Proulx

"Frequent, bite-sized training is crucial so staff actually remember how to support customers with disabilities."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023