Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Jon Fukuda
Design Planning and Management Support
2023 • DesignOps Summit 2023
Gold
Sam Ladner
Data Exhaust and Personal Data: Learning from Consumer Products to Enhance Enterprise UX
2016 • Enterprise UX 2016
Gold
Sabrina Mach
How to Design Your Design Operating Model
2021 • DesignOps Summit 2021
Gold
Mandy Drew
What Role(s) Can Research Play in Responsible Design?
2021 • Advancing Research 2021
Gold
Jason Mesut
Unmasking Design Leadership: Navigating leadership without neglecting ourselves
2025 • Rosenfeld Community
Kathleen Asjes
Research Democratization: the Good, the Bad and the Ugly
2022 • Advancing Research 2022
Gold
Crystal Philcox
The Many Faces of Operations
2017 • DesignOps Summit 2017
Gold
Aras Bilgen
Who does the math: A designer’s journey in building an AI-based tutoring app
2025 • Designing with AI 2025
Gold
Chelsea Mauldin
Let's Talk About Money
2022 • Civic Design 2022
Gold
Peter Levin
Solve a Problem Here, Transform a Strategy There: Research as an Occasion for Expanding Organizational Possibility
2024 • Advancing Research 2024
Gold
Deanna Mitchell
Designing with culture: Unlocking impactful insights for Product and UX
2025 • Advancing Research 2025
Gold
Billy Carlson
Ideation tips for Product Managers
2022 • Design in Product 2022
Gold
Landon Barnes
Are My Research Findings Actually Meaningful?
2022 • Advancing Research 2022
Gold
Sarah Barrett
The "How" of Enterprise Information Architecture
2023 • Enterprise UX 2023
Gold
Emily DiLeo
Stronger Together: Lessons Learned from UX Research Ops
2024 • DesignOps Summit 2024
Gold
Maria Taylor
Knowledge is Power: Managing the Lifeblood of the Design Org
2023 • DesignOps Summit 2023
Gold

More Videos

Sam Proulx

"I don’t know anyone who uses the configuration out of the box without customizing it heavily and extensively."

Sam Proulx

Everything You Ever Wanted to Know About Screen Readers

June 11, 2021

Bria Alexander

"If anything's going wrong on your end, please reach out to help-customer-desk-service in Slack for tech help."

Bria Alexander

Opening Remarks

November 17, 2022

Corey Nelson

"Negotiating severance is absolutely possible; consider legal advice or coaching when doing so."

Corey Nelson Amy Santee

Layoffs

November 15, 2022

Milan Guenther

"Enterprises are basically people pursuing outcomes, doing activities, and using objects—this simple theory underpins the language of Edgy."

Milan Guenther

A Shared Language for Co-Creating Ambitious Endeavours

June 6, 2023

Erin May

"Without releasing control, democratizing research won’t scale. We have to empower people even if some things won’t be perfect."

Erin May Roberta Dombrowski Laura Oxenfeld Brooke Hinton

Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org

March 10, 2022

Sam Proulx

"No one uses the screen reader with out-of-the-box settings; users customize it to fit their needs."

Sam Proulx

Understanding Screen Readers on Mobile: How And Why to Learn from Native Users

June 6, 2023

Mujtaba Hameed

"We experimented with AI-powered sprints, but had lots of dead ends and false starts before finding what really works for us."

Mujtaba Hameed

The new horizon of ethnography: using AI to unlock the full potential of in-person research

March 11, 2026

Ilana Lipsett

"Whoever has been and might continue to be the most negatively impacted absolutely needs to be centered and have their voices at the table."

Ilana Lipsett

Anticipating Risk, Regulating Tech: A Playbook for Ethical Technology Governance

December 10, 2021

Samuel Proulx

"If we think about training AI on the average, then what we will get is more of the same."

Samuel Proulx

From Standards to Innovation: Why Inclusive Design Wins

September 10, 2025