Redefining Toolkits: Unbundling to Create a Perfect Match
Summary
"Redefining the Research Toolkit" not only applies to methods, but tools, too. The marketplace for research platforms continues growing and evolving, offerings professionals the opportunity to find a nearly perfect research match. This means that more teams are opting to "unbundle" their research stack, choosing best-in-class options for critical phases instead of an all-in-one. This session has two goals: 1) Highlight some of the of tools researchers are prioritizing for their stacks 2) Present a case study of a best-in-class stack in action Join to learn about the freedom of choice researchers have for their toolkits and some of the best-in-class options available.
Key Insights
-
•
All-in-one research platforms average $100,000 per year and may not flexibly adapt as research needs evolve.
-
•
BYO toolkits enable teams to select best-in-class tools tailored to their specific research methods and questions.
-
•
User Interviews specializes in upstream participant recruitment, maintaining an exceptionally low fraud rate of 0.3%.
-
•
AI-driven participant matching improves recruitment quality and reduces panel churn in User Interviews’ platform.
-
•
Calendly uses a BYO stack combining Sprig for in-app surveys and User Interviews for recruiting to address diverse research needs.
-
•
Flexibility in tool choice allows teams with limited headcount and budgets to scale research effectively.
-
•
BYO stacks support democratizing research by enabling controlled access for non-researchers like PMs and designers.
-
•
User Interviews offers direct integrations and flexible API partnerships, plus options to use any tool via URL links.
-
•
Effective fraud detection combines automated monitoring with researcher-controlled flags to maintain data integrity.
-
•
Focusing on a core strength—in this case, participant recruitment—and partnering with specialized tools leads to more effective research workflows.
Notable Quotes
"Budgets are tight across research, design, and product; all-in-one platforms are costly and often don’t fit those budgets."
"When you get a best in class tool, you are able to execute more rigorous research and show clearer ROI."
"We found a greater variety and depth of questions that require richer methodologies, so BYO works better for us."
"Our fraudulent participant rate is about 0.3%, compared to the industry average of 20 to 22%."
"We built AI that suggests projects to participants that pre-match what you’re asking for instead of mass blasting everyone."
"The world is your oyster with whatever tool you want to use, even if we don’t directly integrate with it."
"The only way to do research wrong is to not do it at all."
"BYO gives researchers more control over who gets access to the panel and where you want guardrails versus freedom."
"My time is more expensive than my money; fraudulent data wastes valuable time."
"We focus on getting top quality participants and let others specialize in testing or analysis tools."
Or choose a question:
More Videos
"Without a human in the loop, AI can go off and do crazy disastrous things."
Jon Fukuda Amy Evans Ignacio Martinez Joe MeersmanThe Big Question about Innovation: A Panel Discussion
September 25, 2024
"The best way is to distribute the accessibility work so it becomes light work for everyone."
Sam ProulxAccessibility: An Opportunity to Innovate
March 9, 2022
"Design and research people must report to leaders who understand their functions, or else they get assigned irrelevant tasks like social coordinator."
Anna Avrekh Amy Jiménez Márquez Morgan C. Ramsey Catarina TsangDiversity In and For Design: Building Conscious Diversity in Design and Research
June 9, 2021
"Context retention is tough; Siri and Google Assistant completely lose prior contexts after one action."
Greg NudelmanDesigning Conversational Interfaces
November 14, 2019
"We translate absolute positioning from design tools into web-based layouts using row and flex concepts familiar to developers."
George Abraham Stefan IvanovDesign Systems To-Go: Reimagining Developer Handoff, and Introducing App Builder (Part 2)
October 1, 2021
"Synthetic data created by AI—like fake personas and journeys—is super derivative and often not insightful."
Shipra KayanMake your research synthesis speedy and more collaborative using a canvas
January 24, 2025
"If something is only visible on hover, it either needs to be visible to screen readers at all times or have an alternative way to access it."
Sam ProulxDesigning For Screen Readers: Understanding the Mental Models and Techniques of Real Users
December 10, 2021
"We stuck with NPS because it was a pure metric with a long history we could trend, instead of muddying things with loyalty or revenue."
Shipra KayanHow we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up
September 30, 2021
"Gesture and embodied actions will be a key tool for understanding user experience as computing moves beyond traditional screen-based interactions."
Dane DeSutterKeeping the Body in Mind: What Gestures and Embodied Actions Tell You That Users May Not
March 26, 2024
Latest Books All books
Dig deeper with the Rosenbot
How can play help overcome the ‘messy middle’ between research insights and decision-making in organizations?
How does Rally’s research infrastructure facilitate participant management and integration with enterprise data systems?
Why is human storytelling stewardship still essential despite advances in AI and research democratization?