Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Summary
This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?
Key Insights
-
•
AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.
-
•
Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.
-
•
AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.
-
•
Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.
-
•
Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.
-
•
Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.
-
•
Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.
-
•
Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.
-
•
Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.
-
•
Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.
Notable Quotes
"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."
"Algorithms that adjust care based on race don’t actually help and embed disparities instead."
"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."
"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."
"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."
"All commercially available facial recognition software perform worse on darker females."
"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."
"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."
"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."
"The best transformation happens by asking the right questions, even when the problem feels overwhelming."
Or choose a question:
More Videos
"Treat unemployment as a full-time job; be relentless and turn over every stone to improve your chances."
Corey LongHiring in DesignOps: A Critical Study on How to Hire and Get Hired
September 23, 2024
"Prototyping is not about usability, it’s about exploring the idea."
Gretchen AndersonScaling the Human Center
June 8, 2017
"If you wait for permission, you will probably never be granted it. Just start doing what you know to be the right thing."
Lori Muszynski Peter MerholzKeeping Design Weird
October 2, 2023
"Magnification can cover the entire screen two to ten times, and users pan around to see different areas."
Sam ProulxSUS: A System Unusable for Twenty Percent of the Population
June 9, 2021
"Users are smart, innovative, and a lot of the innovation actually comes from users rather than big companies."
Prayag Narula Rida QadriHCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
March 28, 2023
"We forget a lot of details over time; documentation saves us from repeating mistakes."
Deanna SmithLeading Change with Confidence: Strategies for Optimizing Your Process
September 23, 2024
"Researchers have a lane enforcing best practice, but non-researchers can significantly contribute to creating intelligence."
Lija HoganDoing more with more: Lessons from the Front Lines of Democratization
March 9, 2022
"Whenever someone makes a game accessible today, it’s considered an inspiring hack not a built-in feature."
Sam ProulxTo Boldly Go: The New Frontiers of Accessibility
March 11, 2022
"You can also reach us quickly by going to help-desk-customer-service for any tech or support issues."
Bria AlexanderOpening Remarks
October 3, 2023
Latest Books All books
Dig deeper with the Rosenbot
How can UX practitioners build influence within healthcare organizations where clinical outcomes dominate decision-making?
What are best practices for onboarding and operationalizing UX research tools post-procurement to ensure adoption?
What role do relationships and empathy play in operational roles like design ops?