Rosenverse
Investigating qualitative depth of AI-moderated interviews

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Investigating qualitative depth of AI-moderated interviews

Gold
Tuesday, March 10, 2026 • Advancing Research 2026
Share the love for this talk
Investigating qualitative depth of AI-moderated interviews
Speakers: Tara Tressel
Link:

Summary

When research teams are small, the hardest tradeoff is often between depth and scale. Live interviews surface rich, contextual insights, but they’re also time-consuming, resource-heavy, thus often deprioritized when bandwidth is low. In this session, I’ll share how I experimented with AI-moderated interviews to bridge that gap, using technology to recover depth and empathy without requiring live facilitation. Faced with the need to understand our customers’ decision-making (those who purchased our platform, and those who didn’t), I initially relied on surveys. However, I found they lacked the nuance that real conversations reveal. By introducing AI moderation, I created a way for participants to engage in adaptive, conversational interviews that went beyond the limits of static forms. I’ll walk through how I set up these sessions, what prompts worked (and didn’t), and how I analyzed the results. I’ll also share how I’ve used other AI tools like ChatGPT and Perplexity to assist with synthesis and bias-checking, creating a workflow that both expands my analytical reach and strengthens the rigor of my findings. As AI tools continue to enter the researcher’s toolkit, this case study illustrates how we can thoughtfully integrate them to expand—not erode—the human depth of qualitative work. It offers a model for how lean teams can maintain research quality while navigating the realities of limited time, budget, and capacity. This talk will explore the emerging space between automation and augmentation, finding opportunity for depth when time and resources are tight.”

Key Insights

  • AI moderation enables more qualitative depth than surveys due to participants speaking freely rather than typing.

  • AI moderation platforms require detailed and explicit research plans to guide effective questioning.

  • AI moderation often struggles with natural conversational flow and can produce awkward pauses or interruptions.

  • Follow-up questions from AI moderators can be vague, repetitive, or miss opportunities to dive deeper.

  • AI moderation is best suited for validation or deepening understanding of known problem spaces, not foundational discovery.

  • Participants may prefer AI moderation over surveys due to ease and flexibility, supporting higher and quicker recruitment.

  • AI moderation platforms currently do not replace human analysis or provide strong quantitative outputs.

  • High emotion or sensitive topics may work better with AI moderation if participants prefer anonymity, but human interviews excel at rapport.

  • Researchers need to invest time troubleshooting and iterating prompts in AI moderated studies.

  • AI moderated studies allow teams of one to scale qualitative research when live interviews are infeasible.

Notable Quotes

"Having one-on-calls with people is one of my favorite parts of conducting research, and it felt like this was being taken away from me just for the sake of saving business resources."

"AI moderation, in short, is a large language model that runs a moderated interview session without a human researcher present."

"I was interested in thinking of this method as a survey plus that you can use to get survey-like data but with more qualitative depth."

"When I gave the model my list of interview questions, it actually made for a really bizarre participant experience because the model just went down the line of the different questions."

"People were more likely to explain their thought process and just more context around the particular situation of their org."

"There were moments like long kind of awkward pauses where the model was processing, and sometimes it actually spoke over participants."

"Follow-up questions missed opportunities to probe deeper or were confusing, and participants had to repeat themselves."

"If I really just needed only quantitative data from a survey, then I would not go the AI moderated route."

"AI moderation did not make the research easier. It made it possible under real world constraints."

"From setup to analysis, AI moderation still requires a good understanding of what quality research looks like and how to interpret qualitative data."

Ask the Rosenbot
Shipra Kayan
How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up
2021 • DesignOps Summit 2021
Gold
Anna Avrekh
User Research, Design, and Product - A Love Story
2021 • Advancing Research 2021
Gold
Mohammad Hossein Jarrahi
Contextuality problem: Exploring the Benefits of Qualitative and Quantitative Research
2023 • QuantQual Interest Group
Brian T. O’Neill
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
2022 • QuantQual Interest Group
Jilanna Wilson
Distributed DesignOps Management
2019 • DesignOps Community
Kaitlin Tasker
Fast and Fearless Inclusive Research
2023 • Advancing Research 2023
Gold
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Ovetta Sampson
Research in the Automated Future
2022 • Advancing Research 2022
Gold
James Lang
Hopeful Futures for UX Research
2026 • Rosenfeld Community
David Cronin
Discussion
2015 • Enterprise UX 2015
Gold
Sohit Karol
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
2020 • Advancing Research 2020
Gold
Kayla Farrell
What It's Like To Be a User Researcher at Compass
2021 • Advancing Research 2021
Gold
Robin Beers
Navigating organizational systems: Rethinking researcher’s role in driving change
2025 • Advancing Research 2025
Gold
Satyam Kantamneni
Do You Have an Experience Vision?
2023 • Enterprise Community
Lija Hogan
Contexts of Use: A Framework for Connection
2021 • Civic Design 2021
Gold
James Wieselman Schulman
Research is a team sport: advancing the work when everyone does the research
2026 • Advancing Research 2026
Gold

More Videos

Erin Weigel

"Increasing page load time by three seconds can nullify all your design improvements."

Erin Weigel

UX Lessons from running more than 1,200 A/B Tests

July 10, 2024

Mike Oren

"Safe money has solid returns, so fiscally responsible companies demand higher returns from new investments especially with inflation."

Mike Oren

Why Pharmaceutical's Research Model Should Replace Design Thinking

March 28, 2023

Lija Hogan

"Now everyone’s an edge case, so research needs to focus on those edge cases, not just the average user."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

Heidi Trost

"If Charlie has been tampered with, Alice needs a clear way to be alerted that she shouldn't trust it."

Heidi Trost

When AI Becomes the User’s Point Person—and Point of Failure

August 7, 2025

Adam Cutler

"None of this matters without outcomes; we design to make people’s days better, not just to design."

Adam Cutler

People + Places + Practices = Outcomes

June 8, 2016

Rittika Basu

"Technophobia is an irrational fear and unease towards modern technology."

Rittika Basu

Age and Interfaces: Equipping Older Adults with Technological Tools

February 23, 2023

Saara Kamppari-Miller

"One superpower of ops humans is making problems and gaps visible to build awareness and drive change."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility

May 26, 2022

Bria Alexander

"This day is focused on me, my job and my specific day-to-day challenges, not on some invisible finish line or senior title."

Bria Alexander

Theme Two Intro

October 3, 2023

Laine Riley Prokay

"Design Ops teams exist in nearly all industries and for all design functions, growing rapidly year over year."

Laine Riley Prokay Lisa Gordon

Carving a Path for Early Career DesignOps Practitioners

September 9, 2022