Journeying toward AI-assisted documentation in healthcare
Summary
- Developing AI design in healthcare requires close collaboration between end users and your data science team
- Piloting GenAI solutions may be more effective than traditional prototyping
- Trading accuracy for efficiency is a barrier to adopting GenAI tools in healthcare
- GenAI design in healthcare requires establishing critical boundaries as well as a good understanding of cognitive processing
- Other factors to consider when designing AI solutions for service-based industries are understanding how training might be impacted, the importance of standardization vs. personalization of data output and the need for more autonomy and control elements due to consequences of unpredictable output errors
Key Insights
-
•
Generative AI can reduce healthcare documentation time by over 60% but doesn't eliminate the need for manual editing and human review.
-
•
Focusing AI tools initially on low-risk use cases like chat encounter summaries mitigates potential clinical harm.
-
•
Large Language Models (LLMs) excel at summarizing text but struggle with capturing exact clinical details and non-verbal cues.
-
•
Designing AI tools requires managing user expectations to prevent disappointment and frustration over imperfect outputs.
-
•
A simple one-button UI with options to regenerate and manually edit notes improves user workflow efficiency and error recovery.
-
•
Implicit metrics such as the 'edit rate'—the fraction of user edits on AI-generated text—help monitor AI output quality unobtrusively.
-
•
Traditional prototyping methods are less effective for AI; incremental live pilots provide critical learning about AI's unpredictable outputs.
-
•
Operational factors like quality assurance metrics impact user attitudes when AI-generated notes lower scores despite time savings.
-
•
Human-centered collaboration between designers, data scientists, and users is essential to shape AI capabilities and discover unforeseen problems.
-
•
AI models must remain static (non-learning) in healthcare to comply with regulations, creating challenges for ongoing improvements.
Notable Quotes
"We saw documentation time reduced by 64% after six months of using the AI tool."
"Documentation is important for liability and useful for patient handoffs, so notes must be concise yet detailed."
"Healthcare is not a silver bullet for AI; a lot of context comes from non-verbal cues where generative AI doesn’t apply."
"The unpredictable black box nature of LLMs means we had to zoom in on small problems first before seeing the bigger picture."
"We emphasized a simple one-button workflow with manual editing and regeneration to handle inevitable AI errors."
"The edit rate, our metric of human-added characters over total characters, tracks AI output quality without burdening users."
"Users were initially excited but six months later expressed frustration and uncertainty about the tool’s usefulness."
"We believe cognitive biases like frequency bias and expectation bias affected how users perceived AI errors over time."
"Our AI tool doesn’t learn from ongoing use due to regulatory constraints, which led to user frustration when performance didn’t improve."
"People are the plot twist in this journey — despite promising AI, only human connection reveals new problems and solutions."
Or choose a question:
More Videos
"Respondents who feel very connected are twice as likely to say they’re making a significant impact."
Marc Fonteijn Ru ButlerIncrease your confidence, influence, and impact (through a Professional Community)
December 3, 2024
"Trust is something to build and maintain, not a box to check or a one-time achievement."
Dem Gerolemou Alexis Oh Neef RehmanClimate technology fundamentals
May 15, 2024
"Home 24’s website is not cluttered and well-structured, explaining its higher appeal and inspiration scores."
Daniela Magaña Flores Ariane Rahn Jeff Ephraim BanderAhead of Competition: Learn What UX Benchmarking Can Do for Your Business Today
March 10, 2022
"Design systems are not just components but guidelines, rules, and principles to create consistent experiences."
Abbey Smalley Sylas SouzaScaling UX Past the Size of Your Team
January 8, 2024
"The two variables that stood out were how many people the project impacts and how often they use it."
Patrick CommarfordDesign Staffing for Impact
January 8, 2024
"Engaging communities requires thinking beyond western-centric tools, considering language and cultural relevance for accessibility."
Xenia Adjoubei Sean BruceEmpowering Communities Through the Researcher in Residence Program
March 29, 2023
"Doing nothing is not a neutral act. Ignorance may be bliss, but it doesn’t absolve you from responsibility."
Craig VillamorDesign Systems for Ethical Design
January 26, 2023
"A designer working on a new component needs to understand not just that component but the templates using it and overall approach."
Jon Fukuda Ellie KryslDesign Planning and Management Support
October 3, 2023
"More frequent, open communication and direct conversations helped us avoid assumptions and deliver something usable quickly."
Iain McMaster IHan ChengDesign and Product: from Frenemy to Harmony
November 29, 2023
