Rosenverse

Why AI Is Bad at Research (and how to make it actually useful)

Gold
Tuesday, March 10, 2026 • Advancing Research 2026
Share the love for this talk
Why AI Is Bad at Research (and how to make it actually useful)
Speakers: Daniel Korczynski
Link:

Summary

LLMs are everywhere, but when it comes to real research, they often fall short. Generic LLMs weren’t built for continuous research workflows, and product researchers quickly see the problem: the outputs are generic, lack full context, and struggle to connect multiple data sources. Instead of surfacing meaningful insights, they can amplify noise. In this session, Daniel will break down why AI often fails research teams and what’s missing. He’ll show how to make AI actually useful for continuous product research. Accelerating analysis, connecting insights across sources, and keeping researchers at the center, equipped with a powerful tool rather than replaced by one.

Key Insights

  • AI in research struggles with large datasets, often averaging results and missing subtle but important signals.

  • Curating and filtering datasets by removing irrelevant data improves AI research output quality.

  • Scoping research into focused projects or topics helps AI deliver more precise responses.

  • Asking one question at a time significantly enhances the quality of AI-generated answers.

  • Providing detailed contextual information (personas, company background, product details) to AI boosts specificity and nuance in responses.

  • AI hallucinations and trust issues necessitate human-in-the-loop processes to verify output quality and citations.

  • Iterative refinement of AI outputs, similar to app development, is critical for achieving polished research results.

  • Spot checking AI-generated citations can be an effective and efficient way to validate research quality.

  • Context passed as embedded knowledge rather than repeated in prompts yields better AI results.

  • Using multiple specialized AI agents to critique each other’s outputs can mitigate bias and improve research accuracy.

Notable Quotes

"AI has this strange weakness that when working with a large dataset, they often miss crucial, subtle findings."

"The larger the dataset you work with, the more costly it is to run a single operation on AI models."

"Whenever possible, you should be breaking down your work into specific research projects or topics."

"When you ask a question, try to ask one at a time so the model doesn't get lost."

"Context is everything — providing AI with a folder of your company’s knowledge makes responses more detailed and useful."

"Research with AI requires as much iteration and verification as building an app or prototype."

"AI-generated research reports should always be tied to real feedback that you can verify behind every sentence."

"There's no way to deny it: every industry needs to adapt to AI, but nobody really knows how yet."

"Human in the loop means constantly interacting with AI, documenting your thoughts and assuring quality."

"Some engineers build a council of agents that debate and generate responses, which can help with bias and accuracy."

Ask the Rosenbot
Erika Flowers
Introduction to MURAL for UX
2021 • Design at Scale 2021
Gold
Bria Alexander
The Big Question about Resilience: A panel discussion
2024 • DesignOps Summit 2024
Gold
Shreya Dhawan
Making service tangible: the fastest path to higher performance
2025 • Advancing Service Design 2025
Gold
Megan Blocker
Theme 2 Intro
2025 • Advancing Research 2025
Gold
Holly Cole
Understanding Experiences: When you have to do more than work
2018 • DesignOps Summit 2018
Gold
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • DesignOps Summit 2025
Gold
Dr. Jamika D. Burge
Bridge Building across Research Disciplines
2021 • Advancing Research Community
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2017 • DesignOps Summit 2017
Gold
Dan Willis
Enterprise Storytelling Sessions
2019 • Enterprise Experience 2019
Gold
Saskia Liebenberg
Start Small for Big Impact
2019 • DesignOps Community
Tricia Wang
The most popular design thinking strategy is BS
2022 • Enterprise Community
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
Nora Tejeda
Scaling Design Capabilities at BBVA Through a Self-service Design Model
2021 • Design at Scale 2021
Gold
Renee Bouwens
Landing Product Impact: Aligning Research as a Foundational Driver for Delivering the World’s Best Products
2023 • QuantQual Interest Group
Cheryl Platz
Collaborative Creativity through Improv
2018 • DesignOps Summit 2018
Gold
Lisa Gironda
Opener: Chief of Staff–An unexpected journey
2024 • DesignOps Summit 2020
Gold

More Videos

Rebecca Gimenez

"Service design is still so new that no one really knows what maturity looks like."

Rebecca Gimenez

Work in Progress: Service Design at Airbnb

December 3, 2024

Lada Gorlenko

"Those three retailers pivoted literally overnight under intense change management pressure."

Lada Gorlenko

Theme 2 Intro

June 9, 2022

Brigette Metzler

"Mature research repositories are rare, and return on investment depends heavily on scale, governance, and organizational structure."

Brigette Metzler Dana Chrisfield

Research Repositories: A global project by the ResearchOps Community

August 27, 2020

"Delight forums helped us share stories and build a shared vernacular but didn’t lead to significant change by themselves."

Discussion

June 9, 2017

Sarah Brooks

"People implementing policy see the problems with the policy long before anyone else does."

Sarah Brooks Jennifer Pahlka

Fireside chat with Sarah Brooks and Jen Pahlka

October 21, 2021

Lisa Spitz

"Every single thing that we would plan to say to the participant is spelled out in this interview guide to build transparency and trust."

Lisa Spitz Nikki Brand

Building Trust Through Equitable Research Practices

November 18, 2022

Theresa Neil

"Preventative care UX challenges, like why men won’t come in, are deeply rooted and not solved by prettier buttons."

Theresa Neil

Designing for Wellness: Specializing in Healthcare

May 22, 2024

Maverick Chan

"Clients got used to AI-driven storytelling pretty quickly when involved in weekly progress updates."

Maverick Chan Claire Lin

From Doodle to Demo: AI as Our Storytelling Partner

October 23, 2025

Alla Weinberg

"When we’re mobilized or immobilized, our IQ literally drops by half and our peripheral vision narrows."

Alla Weinberg

Design Teams Need Psychological Safety: Here’s How to Create It

September 8, 2022