Testing and Experimentation Tools
This video is featured in the UX metrics & analytics playlist.
Summary
Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.
Key Insights
-
•
AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.
-
•
Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.
-
•
Context is crucial for effective research moderation, which AI currently lacks.
-
•
Synthentic participants are useful for brainstorming but cannot replace real user research.
-
•
Democratization of research increases access but risks diluting research quality without expert involvement.
-
•
AI features that save researchers time by handling repetitive tasks are promising and more practical.
-
•
AI moderation should operate as a human-in-the-loop assistant, not a full replacement.
-
•
Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.
-
•
UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.
-
•
There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.
Notable Quotes
"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."
"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."
"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."
"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."
"If all tools try to be for everyone, they risk serving no one properly."
"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."
"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."
"Human moderation will always have a place because humans are unpredictable and random."
"We never train AI models on your data, and all AI features are strictly opt-in."
"There is a huge space for improvement in AI features for UX research if these have any future."
Or choose a question:
More Videos
"Critical making requires critique, theory, activity, reflection, leading back to theory, but it doesn't have to start with theory. It can start anywhere."
Nathan Shedroff Hugh Dubberly Thomas J. McLeishHow Will Design be Taught When the Schools Shut Down?
May 8, 2026
"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."
John Paul de Guzman10k Screens Later: How We Became a Data-Driven Design Organization
September 24, 2024
"There are over five hundred tools that call themselves survey software on G2, making it one of the largest software categories."
Aleksandra Korczynska Caroline Jarrett Justyna ParmeeSurvey Tools
March 12, 2026
"Sustainable and flexible work tactics are key to navigating constant change without burnout."
Kristin SkinnerTheme 2: Introduction and Provocation
January 8, 2024
"We pay all of our speakers and provide both subject matter and professional speaker coaching to polish your presentation."
Rachael Dietkus, LCSW Victor Udoewa Jennifer StricklandEverything You Need to Know about the Civic Design 2022 Call for Presentations
May 17, 2022
"Agile as trauma is a useful essay to understand how agile can harm design practices."
Uday GajendarThe Rise of Meta-Design: A Starter Playbook
May 19, 2022
"Giving feedback is a skill, just like design. It takes practice and care."
Vanessa VarinFeedback: The Other F-Word
September 10, 2025
"You don’t think of raising a child or teaching a student as solving a problem, it’s an ongoing process—and so is design today."
Hugh DubberlyProblems with Problems: Reconsidering the Frame of Designing as Problem-Solving
June 19, 2019
"Leading the future of research isn’t about control, it’s about influence and impact."
Sean McKayCoexisting with non-researchers: Practical strategies for a democratized research future
March 11, 2025
