Rosenverse
Managing the Human Engagement Risks of AI

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI

Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the Designing with AI 2025 playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Saara Kamppari-Miller
DesignOps for Inclusive Design and Accessibility
2022 • DesignOps Community
Kit Unger
Theme 2: Introduction
2021 • Design at Scale 2021
Gold
Jake Burghardt
Stop wasting research: Unlock more value from research insights
2025 • Rosenfeld Community
Nathan Curtis
Beyond the Toolkit: Spreading a System Across People & Products
2016 • Enterprise UX 2016
Gold
Alëna Iouguina
Designing Systems at Scale
2018 • DesignOps Summit 2018
Gold
Amy Gawronski Zuccaro
Advice for DesignOps Employee #1
2021 • DesignOps Summit 2021
Gold
Max Gadney
Assessing UX jobs for impact in climate
2024 • Climate UX Interest Group
Bianca Jefferson
From Sprints to Systems: Operationalizing Continuous Discovery Through DesignOps
2025 • DesignOps Summit 2025
Gold
Dave Gray
Liminal Thinking: Sense-making for systems in large organizations
2015 • Enterprise UX 2015
Gold
Taylor Klassman
The Evolution of UX Research Platforms
2026 • Advancing Research 2026
Gold
Alla Weinberg
How to Build and Scale Team Safety
2024 • DesignOps Summit 2020
Gold
Patrick Boehler
Fishing for Real Needs: Reimagining Journalism Needs with AI
2025 • Designing with AI 2025
Gold
Saskia Liebenberg
Start Small for Big Impact
2019 • DesignOps Community
Lada Gorlenko
Theme 2 Intro
2022 • Design at Scale 2022
Gold
Rachael Dietkus, LCSW
AI: Passionate defenses and reasoned critique [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Theresa Neil
Just Build Me a Dashboard!
2019 • Enterprise Community

More Videos

Louis Rosenfeld

"You can change the conference time zone to your local time zone in the upper right corner so there’s no confusion about start times."

Louis Rosenfeld Bria Alexander

Day 1 Welcome

September 23, 2024

Lisa Gironda

"If you don’t have 10 minutes to meditate a day, you actually need to meditate for an hour."

Lisa Gironda

Opener: Chief of Staff–An unexpected journey

January 8, 2024

Jamie Beck Alexander

"Google Maps’ Leaf visualization helps people normalize efficient driving, demonstrating the power of design in behavior change."

Jamie Beck Alexander Nina Gregg Shawn Petersen Bill DeRouchey

How can you find your role in climate?

January 17, 2024

Iram Shah

"If you’re not inspired and motivated, you cannot connect the dots across functions for a seamless customer experience."

Iram Shah

Closing Keynote: The View from the Top

June 4, 2019

Melinda Belcher

"You hire a consultant for partnership, not just to execute — you want someone thinking in terms of business results, not just projects."

Melinda Belcher

Insider preview of Enterprise Experience 2020

May 28, 2020

Natalia Radywyl

"Co-planning means building trust and relationships before planning the project itself as a blended team."

Natalia Radywyl

Co-Designing New Power in Australia's Public Sector

November 16, 2022

Uday Gajendar

"Tradecraft is the making stuff, stagecraft is a kind of cross-functional therapy, and statecraft is organizational diplomacy."

Uday Gajendar

The Rise of Meta-Design: A Starter Playbook

May 19, 2022

Jesse Zolna

"Listening is different than conversation, even though it looks and feels similar."

Jesse Zolna

Inviting the Whole Org to Come See For Yourself

March 30, 2020

Steve Baty

"Being a change agent means designing for change, which is harder than the change itself."

Steve Baty Richard Dalton Maria Giudice Harry Max

Discussion

June 9, 2016