Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing inclusively with AI

Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
Designing inclusively with AI
Speakers: Kate Kalcevich
Link:

Summary

We know there’s massive opportunity for AI to be used for greater speed, growth, and insights, but how can we avoid the pitfalls? Head of Accessibility Innovation at Fable Kate Kalcevich will share how to best leverage current AI tools in your day-to-day work with inclusion in mind.

Key Insights

  • AI-generated images of people with disabilities often show racial and representation biases, such as all-white subjects and inaccurate assistive device depictions.

  • Disability representation in AI images is frequently inaccurate, e.g., guide dogs shown with leashes instead of harnesses and blindfolds over eyes.

  • AI-generated personas for users with disabilities can contain improbable feature combinations, such as a MacBook user reliant on Windows-only assistive tech like JAWS.

  • Personas created by AI lack the depth and nuance gained from real user research and testing, especially for disability accessibility considerations.

  • AI-generated mockups can have accessibility issues like poor contrast and unlabeled UI elements, requiring human accessibility review.

  • AI tools like Stark Sidekick improve design accessibility through automated issue detection, though they are not yet full replacements for experts.

  • AI-powered task management apps can enhance productivity and accommodation for neurodivergent users by organizing schedules and reducing cognitive load.

  • AI can help identify potential biases in accessibility automation projects, but requires careful prompt crafting and domain expertise.

  • Designing AI-powered interfaces comes with inclusion challenges, as users unfamiliar with prompting may struggle to get effective results.

  • Combining AI output with continuous user testing, especially involving people with disabilities, is essential to ensure accessibility and inclusion.

Notable Quotes

"If you have questions, I will take questions throughout."

"AI-generated images often have no racial diversity and show inaccurate assistive device representation."

"You are either navigating with a white cane or a service dog because you are not gonna want both of your hands full."

"Personas tend to not give you those deep insights that you would get from user research and user testing."

"Automation might not be able to capture the context of an image if you're trying to describe an image."

"AI is actually a pretty good productivity hack as a starting point for accessibility reviews."

"A lot of people will try a prompt and not get what they were hoping to get and maybe miss out on all the benefits of AI."

"For people who identify as autistic, AI tools that review writing tone can be especially helpful."

"If the code doesn’t match the label on screen, voice controls might not recognize the command."

"There always has to be a human option if you’re not getting what you need from the AI."

Ask the Rosenbot
John Paul de Guzman
10k Screens Later: How We Became a Data-Driven Design Organization
2024 • DesignOps Summit 2024
Gold
Alnie Figueroa
The Future of Design Operations: Transforming Our Craft
2025 • DesignOps Summit 2025
Gold
Kate Koch
Flex Your Super Powers: When a Design Ops Team Scales to Power CX
2021 • DesignOps Summit 2021
Gold
Ross Smith
Breaking Barriers with Empathy
2017 • Enterprise Experience 2017
Gold
Julie Norvaisas
Back to basics, or start from scratch?
2025 • Advancing Research 2025
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • DesignOps Summit 2023
Gold
Patrizia Bertini
The (r)evolution of designOps: It’s Time to Think (really) BIG
2025 • DesignOps Summit 2025
Gold
Veevi Rosenstein
Building for Scale: Creating the Zendesk UX Research Practice
2024 • Enterprise Experience 2020
Gold
Brigette Metzler
Scaling ResearchOps: Helping Researchers do Their Best Work
2020 • Advancing Research 2020
Gold
Alëna Iouguina
Designing Systems at Scale
2018 • DesignOps Summit 2018
Gold
JD Buckley
Communicating the ROI of UX within a large enterprise and out on the streets
2018 • Enterprise Experience 2018
Gold
Indi Young
Thinking styles: Mend hidden cracks in your market
2025 • Rosenfeld Community
Meredith Black
Building Community and Common Trends to Look for in 2021
2020 • DesignOps Community
Brian T. O’Neill
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
2022 • QuantQual Interest Group
Mike Brzozowski
UX in everyday products: Empowering climate conscious choices
2024 • Climate UX Interest Group
Dr. Jamika D. Burge
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
2023 • Enterprise UX 2023
Gold

More Videos

Greg Petroff

"The code starts to follow the outcomes and might even start to write itself at some level."

Greg Petroff

Software as Material—A Redux

June 6, 2023

Brendan Jarvis

"Who do you see? Who do you not see? Why do you not see them?"

Brendan Jarvis

Framing Tomorrow by Questioning Today

June 8, 2022

Prerna Makanawala

"Tracking implementation commitments after each design network group keeps momentum and accountability."

Prerna Makanawala

Achieving Balanced Design Consistency

June 9, 2021

Tricia Wang

"Language matters — calling people 'users' reflects and reinforces the reality of reduced agency."

Tricia Wang

From Users to Shapers of AI: The Future of Research

March 25, 2024

Cheryl Platz

"Not all monetization is equal; pay-to-express models are less threatening to social dynamics than pay-for-power schemes."

Cheryl Platz

Embrace Your Fun Factor: Game Development Best Practices for Product Design

January 9, 2026

Ariel Kennan

"I think we all know no matter if you’re new to this work or you’ve been doing it for years, we’re always still learning."

Ariel Kennan

Theme Two Intro

November 17, 2022

Tricia Wang

"Designers are the most insecure function in many companies because their role—to represent people's needs—is both critical and ambiguous."

Tricia Wang

The most popular design thinking strategy is BS

January 27, 2022

Kristin Wisnewski

"The state of IT is a daily reflection of what the company thinks and feels about its employees."

Kristin Wisnewski

Measuring What Matters

October 23, 2019

Scott Jensen

"They’re not living out the Judgment of Solomon every day, where the baby can be split in half."

Scott Jensen Sarah Delaney Carmen Liu

Short Take #2: UX/Product Lessons from Your Industry Peers

December 6, 2022