Testing and Experimentation Tools
This video is featured in the UX metrics & analytics playlist.
Summary
Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.
Key Insights
-
•
AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.
-
•
Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.
-
•
Context is crucial for effective research moderation, which AI currently lacks.
-
•
Synthentic participants are useful for brainstorming but cannot replace real user research.
-
•
Democratization of research increases access but risks diluting research quality without expert involvement.
-
•
AI features that save researchers time by handling repetitive tasks are promising and more practical.
-
•
AI moderation should operate as a human-in-the-loop assistant, not a full replacement.
-
•
Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.
-
•
UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.
-
•
There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.
Notable Quotes
"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."
"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."
"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."
"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."
"If all tools try to be for everyone, they risk serving no one properly."
"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."
"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."
"Human moderation will always have a place because humans are unpredictable and random."
"We never train AI models on your data, and all AI features are strictly opt-in."
"There is a huge space for improvement in AI features for UX research if these have any future."
Or choose a question:
More Videos
"Sprint as theater is not going to result in a product concept that really represents human needs, behaviors, and attitudes."
Anne MamaghaniHow Your Organization's Generative Workshops Are Probably Going Wrong and How to Get Them Right
March 28, 2023
"AI can provide healthy constraints, help people focus, use a common language, and act as cognitive accelerants."
Kurt McCullochFaster alone, further together: Rebuilding collaboration in the age of AI research
March 10, 2026
"Pat Helisco funds projects that have social impact and provides designers space to exhibit and connect with stakeholders."
Sofía Delsordo Kassim VeraPublic Policy for Jalisco's Designers to Make Design Matter
December 8, 2021
"I wrote the book because people have a lot of questions about how to become a product manager or how to work with product managers or what is product."
Christian CrumlishIntroduction by our Conference Chair
December 6, 2022
"Design ops is a way to care for the people who design."
Ebru NamaldiDesigning the Designer’s Journey: Scaling Teams, Culture, and Growth Through DesignOps
September 11, 2025
"A formal critique often carries tension; focusing on people rather than work is a pitfall we must avoid."
Joseph MeersmanSweating the Pixel: Scaling Quality through Critique
June 10, 2021
"The challenge is co-developing a custom investment model that alters decision making based on portfolio logic rather than project-based thinking."
Milan Guenther Benjamin KumpfThe $212 billion ‘so what?’: unlocking impact in development cooperation
November 20, 2025
"Research democratization is all about empowering people who are not researchers to start doing research themselves."
Kathleen AsjesResearch Democratization: the Good, the Bad and the Ugly
March 10, 2022
"The future should not be self-driving. It’s up to us, not the technology, to figure out the right way to use AI."
Josh Clark Veronika KindredSentient Design, AI, and the Radically Adaptive Experience (1st of 3 seminars)
January 15, 2025
