Rosenverse
Broad Strokes: Connecting Design, Research, and AI to the World Around Us

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Broad Strokes: Connecting Design, Research, and AI to the World Around Us

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Speakers: Dr. Jamika D. Burge
Link:

Summary

AI-based modeling plays an ever-increasing role in our daily lives. Our personal data and preferences are being used to build models that serve up anything from movie recommendations to medical services with a precision designed to make life decisions easier. But, what happens when those models are not precise enough or are unusable because of incomplete data, incorrect interpretations, or questionable ethics? What's more, how do we reconcile the dynamic and iterative nature of AI models (which are developed by a few) with the expansive nature of the human experience (which is defined by many) so that we optimize and scale UX for AI applications equitably? This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?

Key Insights

  • AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.

  • Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.

  • AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.

  • Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.

  • Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.

  • Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.

  • Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.

  • Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.

  • Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.

  • Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.

Notable Quotes

"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."

"Algorithms that adjust care based on race don’t actually help and embed disparities instead."

"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."

"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."

"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."

"All commercially available facial recognition software perform worse on darker females."

"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."

"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."

"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."

"The best transformation happens by asking the right questions, even when the problem feels overwhelming."

Ask the Rosenbot
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold
Nathan Curtis
Beyond the Toolkit: Spreading a System Across People & Products
2016 • Enterprise UX 2016
Gold
Jackie Ho
Lead Effectively While Preserving Team Autonomy with Growth Boards
2024 • Enterprise Experience 2020
Gold
Luke Wroblewski
Designing AI Applications
2026 • Rosenfeld Community
Jen Cardello
Learning Velocity—The Insights Speedometer
2021 • Advancing Research Community
Sarah Auslander
Insights Panel
2022 • Civic Design 2022
Gold
Kurdin Bazaz
Culture, DIBS & Recruiting
2021 • Design at Scale 2021
Gold
Lily Aduana
5 Reasons to Bring Your Recruiting in-House (and How To Do It)
2021 • Advancing Research 2021
Gold
Sheryl Cababa
Expanding Your Design Lens with Systems Thinking
2023 • Enterprise Community
JP Allen
Navigating the UX Tools Landscape
2021 • DesignOps Summit 2021
Gold
Theresa Slate
Why Changing Hearts & Minds Doesn’t Work When Promoting DE&I Efforts, but Checklists Do
2023 • DesignOps Summit 2023
Gold
George Aye
That Quiet Little Voice: When Design and Ethics Collide
2022 • Civic Design 2022
Gold
Changying (Z) Zheng
Practical DesignOps: From Ideas to Tools That Teams Actually Use
2025 • Rosenfeld Community
Jilanna Wilson
Distributed DesignOps Management
2019 • DesignOps Community
Neema Mahdavi
Operationalizing DesignOps
2018 • DesignOps Summit 2018
Gold
Louis Rosenfeld
Becoming a Civic Designer: Making the Move from Private to Public Sector
2022 • Civic Design 2022
Gold

More Videos

Nathan Shedroff

"Kids love to play. They challenge the world through games and simulation, so the things we create need to support play and experimentation."

Nathan Shedroff Hugh Dubberly Thomas J. McLeish

How Will Design be Taught When the Schools Shut Down?

May 8, 2026

John Paul de Guzman

"Design is really a team sport. Everyone is responsible to keep the workspace up to date."

John Paul de Guzman

10k Screens Later: How We Became a Data-Driven Design Organization

September 24, 2024

Aleksandra Korczynska

"Advanced logic, matrix questioning, AI survey design creation — these are baseline capabilities you should expect from any modern survey tool."

Aleksandra Korczynska Caroline Jarrett Justyna Parmee

Survey Tools

March 12, 2026

Kristin Skinner

"We need to define success in ways that make our work meaningful and purposeful."

Kristin Skinner

Theme 2: Introduction and Provocation

January 8, 2024

Rachael Dietkus, LCSW

"The community is a big tent - welcoming diverse perspectives and geographies to define and shape civic design."

Rachael Dietkus, LCSW Victor Udoewa Jennifer Strickland

Everything You Need to Know about the Civic Design 2022 Call for Presentations

May 17, 2022

Uday Gajendar

"Creative provocation isn’t about building those crazy ideas but opening space for questioning assumptions."

Uday Gajendar

The Rise of Meta-Design: A Starter Playbook

May 19, 2022

Vanessa Varin

"Feedback is more like an investment and a contribution into not just the person's work, but also to their growth."

Vanessa Varin

Feedback: The Other F-Word

September 10, 2025

Hugh Dubberly

"Corporations are already largely self-driving often in a bad way, operating on autopilot by policies, which requires human intervention."

Hugh Dubberly

Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving

June 19, 2019

Sean McKay

"Research wasn’t meant to be a silo. It was meant to be a guiding function, a system for creating shared knowledge that drives better decisions."

Sean McKay

Coexisting with non-researchers: Practical strategies for a democratized research future

March 11, 2025