Rosenverse
Building impactful AI products for design and product leaders, Part 1: The new product journey

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Building impactful AI products for design and product leaders, Part 1: The new product journey

Wednesday, July 16, 2025 • Rosenfeld Community

This video is featured in the AI and UX playlist.

Share the love for this talk
Building impactful AI products for design and product leaders, Part 1: The new product journey
Speakers: Peter Van Dijck
Link:

Summary

This talk is for product people, UX designers, researchers, and leadership preparing to work on AI products. How do we plan and build helpful and impactful AI products? What is different in this new world? Products that use AI as a core part of the experience are often very easy to prototype, but really hard to build and make actually useful and impactful. A lot of the knowledge on how to build these is locked up in foundation model companies and engineering teams in startups. Peter van Dijck, expert in both AI and IA, will provide an easy to understand framework for understanding how to think about this new world we all live in now, how to plan for effective, helpful and impactful AI products, how to think about model capabilities, AI architectures and evaluations. You will learn vocabulary and frameworks to understand what all the acronyms mean, and how to think about them.

Key Insights

  • AI product design is best understood through a model, context, and experience framework.

  • Models are stateless and require full context to be sent with each interaction.

  • Prototyping to explore model capabilities is critical before building full products.

  • Evals, automated and manual, are essential to measure model output quality and decide if it’s good enough.

  • Models gain enhanced abilities through post-training on specific capabilities like reasoning and conversational skills.

  • Synthetic data generation enables scalable training and evaluation by producing many good and bad examples.

  • Tool use and function calling empower AI agents to autonomously perform multi-step tasks.

  • Agents like Cloud Code gain product-market fit by making model reasoning and actions visible to build user trust.

  • UX principles remain vital in AI product development, especially when designing evals and assessing experience.

  • In high-risk domains, explicit risk identification and eval development are necessary, alongside clear ownership of go/no-go decisions.

Notable Quotes

"Today’s best AI products are designed by engineering first, but UX and product people have to join the party."

"Models are stateless; they take input and give output and forget everything immediately after."

"You want to trust the model, which means not overly instructing it but giving it the right context."

"Evals are the key to knowing when an AI’s output is good enough."

"Model companies post-train models on capabilities by showing them millions of good and bad examples."

"Tool use and function calling are what make agents possible, letting models call tools and reason through tasks."

"Synthetic data is exceedingly useful because you can generate it at scale for training and evals."

"The reason Cloud Code has such product market fit is because it shows what it’s doing, building trust."

"UX skills especially shine in defining realistic, user-centered evals and collaborating with experts."

"If a capability exists and is useful, it’s very likely the model will improve significantly on it over time."

Ask the Rosenbot
Sam Yen
Driving Organizational Change Through Design? Do more of this and less of that
2017 • Enterprise Experience 2017
Gold
Veevi Rosenstein
Building for Scale: Creating the Zendesk UX Research Practice
2024 • Enterprise Experience 2020
Gold
John Calhoun
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
2024 • DesignOps Summit 2024
Gold
Greg Petroff
Design is the Differentiator: Bringing New Design Innovations to a Very Antiquated and Very Large Industry
2021 • Design at Scale 2021
Gold
Ryan Rumsey
Business Influence Without Losing Your Soul
2021 • Enterprise Community
Sofía Delsordo
Public Policy for Jalisco's Designers to Make Design Matter
2021 • Civic Design 2021
Gold
Greg Petroff
Software as Material—A Redux
2023 • Enterprise UX 2023
Gold
Caroline Jarrett
Have fun with statistics?
2024 • Rosenfeld Community
Patrizia Bertini
The (r)evolution of designOps: It’s Time to Think (really) BIG
2025 • DesignOps Summit 2025
Gold
Erin May
Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org
2022 • Advancing Research 2022
Gold
Justin Entzminger
Risk and Reward: How to Diversify the Field of Civic Innovators and Designers
2022 • Civic Design 2022
Gold
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Kate Kalcevich
Integrating Accessibility in DesignOps
2024 • DesignOps Summit 2024
Gold
Victor Udoewa
Beyond Methods and Diversity: The Roots of Inclusion
2024 • Advancing Research 2024
Gold
Megan Blocker
Panel: Excellence in Impact
2024 • Advancing Research 2024
Gold
Daniel Gloyd
Designing From the Inside Out: How Method Acting Can Inspire Design Research
2026 • Rosenfeld Community

More Videos

Nathan Shedroff

"Reading books and articles is no longer central for many young people. What will replace books and articles for collective knowledge building?"

Nathan Shedroff Hugh Dubberly Thomas J. McLeish

How Will Design be Taught When the Schools Shut Down?

May 8, 2026

John Paul de Guzman

"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."

John Paul de Guzman

10k Screens Later: How We Became a Data-Driven Design Organization

September 24, 2024

Aleksandra Korczynska

"AI changes everything in unstructured data analysis; before it was painful manual work, now it's automated theme detection and clustering."

Aleksandra Korczynska Caroline Jarrett Justyna Parmee

Survey Tools

March 12, 2026

Kristin Skinner

"One hallmark of a healthy team is how well team members feel cared for so they can shine."

Kristin Skinner

Theme 2: Introduction and Provocation

January 8, 2024

Rachael Dietkus, LCSW

"Attendee cohorts meet daily in small groups with facilitators to share learning goals and deepen engagement."

Rachael Dietkus, LCSW Victor Udoewa Jennifer Strickland

Everything You Need to Know about the Civic Design 2022 Call for Presentations

May 17, 2022

Uday Gajendar

"The poetic and humanistic aspects of design are getting squeezed out by prescriptive delivery models."

Uday Gajendar

The Rise of Meta-Design: A Starter Playbook

May 19, 2022

Vanessa Varin

"Giving feedback is a skill, just like design. It takes practice and care."

Vanessa Varin

Feedback: The Other F-Word

September 10, 2025

Hugh Dubberly

"Designers are creating conditions in which systems can grow, learn, and thrive—moving beyond just solving problems to meta-design."

Hugh Dubberly

Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving

June 19, 2019

Sean McKay

"Balancing risk and confidence helps teams move at the right speed for the right decisions."

Sean McKay

Coexisting with non-researchers: Practical strategies for a democratized research future

March 11, 2025