Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Don't botch the bot: Designing interactions for AI

Gold
Tuesday, June 4, 2024 • Designing with AI 2024
Share the love for this talk
Don't botch the bot: Designing interactions for AI
Speakers: Savannah Carlin
Link:

Summary

It seems like every company is adding a conversational AI chatbot to their website lately, but how do you actually go about making these experiences valuable and intuitive? Savannah Carlin will present a case study on a conversational AI chatbot—Marqeta Docs AI—that she designed for a developer documentation site in the fintech industry. She will share her insights, mistakes, and perspectives on how to use AI in a meaningful, seamless way, especially for companies like Marqeta that operate in highly regulated industries with strict compliance standards. The talk will use specific examples and visuals to show what makes conversational AI interactions uniquely challenging and the design patterns that can address those challenges. These include managing user expectations, handling errors or misunderstandings within the conversation, and ensuring that users can quickly judge the quality of a bot’s response. You’ll gain a deeper understanding of the intricacies involved in designing interactions for AI, along with practical advice you can apply in your own design processes. Take-aways What to consider before you add AI to your product to ensure it will be valuable, usable, and safe for its intended workflows The interactions that are unique to conversational AI experiences and the design patterns that work for them Common challenges in designing conversational AI experiences and how to overcome them

Key Insights

  • Defining a clear and specific primary use case is crucial before starting any generative AI chatbot project.

  • High-quality, thoroughly reviewed training data is foundational to delivering accurate and useful AI outputs.

  • Initial state messaging must clearly frame what the chatbot can and cannot help with to reduce irrelevant or off-topic queries.

  • Loading indicators for AI text responses should be subtle, with progress reflected by text appearing rather than distracting animations.

  • Supporting efficient scrolling and prompt review is vital since users frequently check and refine their inputs against often long answers.

  • Error states in AI chatbots shift from traditional fixed errors to helping users write better prompts to get more relevant results.

  • Transparency about accuracy, AI limitations, and source citations builds user trust, especially in regulated domains like FinTech.

  • Providing users with prompt engineering guidance via documentation significantly improves the quality of chatbot interactions.

  • Accessibility considerations, like keyboard navigation and screen reader compatibility, must be integrated from the start, especially given the large text outputs.

  • Chatbots can reduce customer support friction and encourage users to ask questions they might not have otherwise, enhancing user engagement with the product.

Notable Quotes

"If you have any doubts about the quality of the training data, do not proceed."

"You want to assist people in framing the interaction and setting their expectations correctly so they know how to be successful."

"The biggest difference with error states in AI bots is helping people write prompts effectively, not just recovering from simple failures."

"Loading text itself is a loading indicator; the letters appearing show progress better than jumpy animations."

"People often forget what they wrote and then want to check their prompt again before refining it."

"Every output should have at least three source links, almost like citations in a research paper."

"We had to be very careful about accuracy because we're in FinTech and compliance is critical."

"People started asking questions to the bot that they wouldn’t have taken the time to email about."

"It’s really important to be clear and transparent about what the tool is good at and what it’s not good at."

"Accessibility testing included making sure everyone could navigate it using a keyboard alone."

Ask the Rosenbot
Bria Alexander
OKRs—Helpful or Harmful?
2022 • DesignOps Community
Meghan Hellstern
The Next 100 Years of Civic Design: How Might We Better Rise to Meet the Challenges of Today and Tomorrow?
2021 • Civic Design 2021
Gold
Steve Chaparro
Bringing Into Alignment Brand, Culture and Space
2020 • DesignOps Community
Landon Barnes
Are My Research Findings Actually Meaningful?
2022 • Advancing Research 2022
Gold
Louis Rosenfeld
Opening Remarks
2022 • Civic Design 2022
Gold
Louis Rosenfeld
Coffee with Lou #3: What Makes for a Successful UX Conference Presentation?
2024 • Rosenfeld Community
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold
Harry Max
Failure Friday #5: Lessons from a SaaS Design Failure
2025 • Rosenfeld Community
Kate Towsey
The State of ResearchOps: More Than Just Theory
2019 • DesignOps Community
Phil Gilbert
A Consistent Culture of Design
2015 • Enterprise UX 2015
Gold
Sean McKay
Coexisting with non-researchers: Practical strategies for a democratized research future
2025 • Advancing Research 2025
Gold
Jorge Arango
AI as Thought Partner: How to Use LLMs to Transform Your Notes (3rd of 3 seminars)
2024 • Rosenfeld Community
Lisa Welchman
Cleaning Up Our Mess: Digital Governance for Designers
2018 • Enterprise Experience 2018
Gold
Panel Discussion: Communicating the Value of DesignOps
2018 • DesignOps Summit 2018
Gold
Bria Alexander
State of DesignOps Panel
2021 • DesignOps Summit 2021
Gold
Dan Hill
Strategic design, slowdown, and the infrastructures of everyday life
2022 • Enterprise Community

More Videos

Jim Kalbach

"Miles Davis gave the musicians the music as they entered the studio, and most first takes were the final ones."

Jim Kalbach

Jazz Improvisation as a Model for Team Collaboration

November 6, 2017

Louis Rosenfeld

"A really strong book has to be designed as a journey with a consistent voice guiding the reader."

Louis Rosenfeld

Coffee with Lou: Should You Write a (UX) Book?

March 7, 2024

Catt Small

"If you’re sitting there thinking I need to build out this project and nobody else is saying the same thing, it means you have to do it, period."

Catt Small Micah Bennett Brian Carr Jessica Harllee

What's Next for ICs: Exploring Staff and Principal Designer Roles

February 22, 2024

Marieke McCloskey

"Less than a third of respondents feel like they work somewhere with a clear investment in customer-driven decision making."

Marieke McCloskey

User Science: Product Analytics & User Research

March 11, 2021

Llewyn Paine

"Even if you’re not doing facial recognition, storing face and voice data is under growing legal scrutiny."

Llewyn Paine

[Demo] Deploying AI doppelgangers to de-identify user research recordings

June 5, 2024

Joshua Noble

"Qualitative research really poses questions; quantitative methods let you investigate those questions rigorously."

Joshua Noble

Casual Inference

October 6, 2023

Sara Logel

"Stakeholders are users too; the product we’re sharing is the research and learnings for their decision-making."

Sara Logel

Your Colleagues are Your Users Too

March 29, 2023

Bria Alexander

"This is the same conference in so many good ways — good ways are that it’s resilient and there’s a certain steadiness."

Bria Alexander Louis Rosenfeld

Welcome

January 8, 2024

Sam Proulx

"Timely interactions that log users out without saving progress cause abandonment, especially for people with disabilities."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023