Rosenverse
Experimental research: techniques for deep, psychology-driven insights

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Experimental research: techniques for deep, psychology-driven insights

Gold
Wednesday, March 12, 2025 • Advancing Research 2025
Share the love for this talk
Experimental research: techniques for deep, psychology-driven insights
Speakers: Katie Hansen
Link:

Summary

In this session, we’ll unlock the transformative power of experimental research methods, inspired by best practices in the social sciences. We’ll explore what these techniques are and how they can be used to reveal powerful insights, drive innovation. You’ll also learn how to get started, how to apply them in ethical and rigorous ways, and why these methods are essential for deep, psychology-driven insights.

Key Insights

  • Experimental research goes beyond AB testing by revealing why changes work, revealing causation instead of just correlation.

  • Testing 20 trust signals simultaneously in one experiment is far more resource-efficient than running 20 separate AB tests.

  • Trust is multi-dimensional; measuring it at both platform and individual pro levels offers a more holistic understanding.

  • Random assignment and control groups are essential for isolating effects and attributing results to manipulated variables.

  • Half of the 20 trust signals tested had statistically significant positive effects on overall trust in Thumbtack.

  • Ten trust signals demonstrated broad impact across all eight trust metrics, providing a clear prioritization roadmap for product teams.

  • Experimental research can help product teams prioritize smartly by focusing on hypotheses likely to succeed before costly AB tests or full launches.

  • Ethics in experimentation require transparency, informed consent, and minimization of harm to participants.

  • Hypothetical scenarios in experiments improve control but have limitations in ecological validity, which can be addressed by validating in live products.

  • Understanding that correlation does not equal causation is crucial for UX researchers to avoid misleading conclusions and to rely on experiments.

Notable Quotes

"When we hear experiment, most think AB testing, but experimental research goes beyond to uncover the why behind outcomes."

"The independent variable is what you manipulate, dependent variable is what you measure to see if there’s an effect."

"Randomization ensures differences between groups come from the variable tested, not external factors."

"Half of the trust signals had a statistically significant positive effect on overall trust in Thumbtack, which is a big deal."

"Testing 20 trust signals simultaneously saved significant time and resources compared to running individual AB tests."

"Trust is complex, so we measured eight different trust-related outcomes at both the platform and professional level."

"Ethics are foundational in experiments since manipulating variables can impact participants' perceptions and emotions."

"Correlation does not equal causation. To say X causes Y, you’ve gotta do an experiment."

"Experimental research bridges the gap between curiosity and action by turning ideas into evidence and evidence into impact."

"Start small with one clear testable hypothesis and build from there to make experimental research approachable."

Ask the Rosenbot
Ted Neward
Theme 4: Enterprise Organizational Journey
2019 • Enterprise Experience 2019
Gold
Ali Jeffery
How DesignOps Helped Enable Wall Street to Work Remotely
2020 • DesignOps Summit 2020
Gold
Ariel Kennan
Building a Design Culture
2017 • Enterprise Experience 2017
Gold
Chelsea Mauldin
Let's Talk About Money
2022 • Civic Design 2022
Gold
Maggie Dieringer
Creating Consistency Through Constant Change
2024 • DesignOps Summit 2020
Gold
Mariesa Lenz
What Beekeeping Taught me about Product Teams
2025 • Rosenfeld Community
Greg Petroff
Everything is About to Change: Software as Material
2016 • Enterprise UX 2016
Gold
Steve Portigal
War Stories LIVE! Steve Portigal
2020 • Advancing Research 2020
Gold
Steve Portigal
War Stories LIVE! Q&A-Discussion
2020 • Advancing Research 2020
Gold
Michelle Morrison
Practice What You Preach
2024 • DesignOps Summit 2020
Gold
David Conrad
The Feeling of Data
2023 • Enterprise Community
Fisayo Osilaja
[Demo] The AI edge: From researcher to strategist
2024 • Designing with AI 2024
Gold
JP Allen
Navigating the UX Tools Landscape
2021 • DesignOps Summit 2021
Gold
Harry Max
Priority Zero: Some Things are More Equal than Others
2016 • Enterprise UX 2016
Gold
Andrea Gallagher
The Problem Space
2019 • Advancing Research Community
Jon Fukuda
The Big Question about Innovation: A Panel Discussion
2024 • DesignOps Summit 2024
Gold

More Videos

Alexia Cohen

"We actively reduced jargon to make our process accessible for non-designers."

Alexia Cohen Adriane Ackerman

Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona

December 4, 2024

Marc Majers

"You want to let participants know it’s a bit non-typical and they are not being tricked or evaluated."

Marc Majers Tony Turner

Interrupted UX - Add A Dose of Reality To Usability Testing

March 11, 2022

Jorge Arango

"Language itself can be a powerful design object, shaping social roles and organizational interactions."

Jorge Arango

Design as an Antidote to VUCA

May 9, 2019

Brenna Fallon

"Psychological safety is far and away the key ingredient for teams being effective."

Brenna Fallon

Learning Over Outcomes

October 24, 2019

Daniel Gloyd

"Design is often framed as problem solving, but conversation is a better framework to respect users as active experts."

Daniel Gloyd

Warming the User Experience: Lessons from America's first and most radical human-centered designers

May 9, 2024

Sam Proulx

"If you build accessible shopping with these four keys, compliance with legal standards like WCAG will naturally fall into place."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023

Kristin Skinner

"We are not replacing teachers; we are enhancing their ability to teach effectively."

Kristin Skinner

Five Years of DesignOps

September 29, 2021

Denise Jacobs

"Designers are change agents. This is part of your passion and what gets you up in the morning."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Roberta Dombrowski

"You can send NDAs and informed consent forms directly through the platform, cutting down admin overhead."

Roberta Dombrowski Lianna Aduana

5 Reasons to Bring your Recruiting in House

September 30, 2021