Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Scaling Research Via an Ops First Model at Clever

Gold
Monday, March 27, 2023 • Advancing Research 2023
Share the love for this talk
Scaling Research Via an Ops First Model at Clever
Speakers: Liza Pemstein and Jane Davis
Link:

Summary

In this Q&A session, we'll hear how Liza Pemstein, Research Operations Manager at Clever, built a robust research practice at one of the fastest-growing EdTech companies. Identifying the friction in the process experienced by design, product, engineering, and marketing, Liza has streamlined and scaled research. She'll share how she got started, what she's learned and what she's excited about in 2023!

Key Insights

  • Clever operates a research operations first model with no dedicated UX researchers, relying on designers and PMs to conduct research.

  • Liza Pennstein’s path to research operations included customer solutions roles emphasizing empathy, listening, and problem-solving.

  • Centralizing participant incentives through a shared credit card and Jira ticket system eliminated confusion and improved process efficiency.

  • Recruiting and scheduling user research participants was a major barrier that was addressed by creating shared participant databases.

  • The ROCKS program at Clever (Research Ops Knowledge Share) facilitates cross-team learning and un-silos research knowledge.

  • Initial focus was on increasing research confidence and comfort among practitioners before emphasizing quality and best practices.

  • Balancing standardization with flexibility is critical; intervention happens when research quality or participant screening falters.

  • Without dedicated UX researchers, Clever prioritizes enabling non-experts with training, templates, and external expert speakers.

  • Building a lightweight repository with Great Question software supports storing and sharing research insights effectively.

  • Future plans include broadening methods like unmoderated research, increasing visibility, and maintaining momentum through speaker series and workshops.

Notable Quotes

"Research is a sales role at its heart, delivering value to people to drive sustainable growth."

"I had a hunger for process and efficiency once I built a feedback bot to share customer insights."

"We don’t have dedicated user researchers, but designers and PMs conduct most research here."

"People were doing incentives in different ways, using their own cards, which was confusing, so I centralized it."

"The biggest struggle was recruiting and scheduling participants, so we started a shared Google sheet."

"Research is a group project and learning is a group project—you don’t have to be an expert but find those who are."

"I want to encourage excitement about research and gradually mold people into better researchers."

"Sometimes you have to make it messy before it can be clean again — disrupt to improve."

"We have the quantity of research work; now we’re focused on building up the quality."

"Unmoderated research is exciting because it lets us gather insights while freeing people up to be in two places at once."

Ask the Rosenbot
Robert Fabricant
Shifting dynamics: The evolving relationship between researchers, participants, and organizational systems
2025 • Advancing Research 2025
Gold
Joshua Graves
We Need To Talk: Navigating Conversations with Your Boss (Part 1 of 3)
2025 • Rosenfeld Community
Dr. Jamika D. Burge
The Future of Research: Bridging the Gaps
2021 • Advancing Research Community
James Rampton
The Basics of Automotive UX & Why Phones Are a Part of That Future
2024 • Rosenfeld Community
Kristen Honey
"Let’s Talk About Data and Crisis”: Public Digital Service Delivery = Open Data + Human Centered Design
2021 • Civic Design Community
Gina Mendolia
Coordinated collaboration: a Service Design & DesignOps love story
2025 • Advancing Service Design 2025
Gold
Allison Sanders
Operating with Purpose
2024 • DesignOps Summit 2020
Gold
Peter Merholz
Design at Scale is People!
2021 • Design at Scale 2021
Gold
Mike Brzozowski
UX in everyday products: Empowering climate conscious choices
2024 • Climate UX Interest Group
Richard Buchanan
Creativity and Principles in the Flourishing Enterprise
2018 • Enterprise Experience 2018
Gold
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • Enterprise UX 2023
Gold
Marc Rettig
Discussion
2015 • Enterprise UX 2015
Gold
How to Identify and Increase your "Experience Quotient"
2018 • Enterprise Experience 2018
Gold
Eric Shumake
Diagnosis UX: Building Influence in Healthcare Design
2026 • Rosenfeld Community
Tara Tressel
Investigating qualitative depth of AI-moderated interviews
2026 • Advancing Research 2026
Conference
Sheri Byrne-Haber
The Importance of Accessible Design Systems
2024 • DesignOps Summit 2020
Gold

More Videos

Daniel Gloyd

"Design research often acts like scientists with clipboards rather than humans in conversation."

Daniel Gloyd

Designing From the Inside Out: How Method Acting Can Inspire Design Research

February 12, 2026

Deirdre Hirschtritt

"I just feel like you’re talking to robots in the system a lot. They don’t have any awareness of other people’s cultures or worldviews."

Deirdre Hirschtritt Cesar Paredes Marie Perrot

Research is Only as Good as the Relationships You Build

November 17, 2022

Benjamin Real

"It was like arriving to your home after a long day and just right to rest but at the same time you know you could not wait to continue working."

Benjamin Real

Maturity Models: A Core Tool for Creating a DesignOps Strategy

October 1, 2021

Max Gadney

"Sometimes you need to view your career like an investor, where you invest your time for the kind of impact you want."

Max Gadney Andrea Petrucci Joshua Stehr Hannah Wickes

Assessing UX jobs for impact in climate

August 14, 2024

Lada Gorlenko

"What is it like to be a new kid on an old block?"

Lada Gorlenko

Theme 1: Intro

January 8, 2024

Feyikemi Akinwolemiwa

"C-FRE stands for curiosity, freedom, reframing, exploration, and engagement—the soft qualities we tend to overlook in play."

Feyikemi Akinwolemiwa

Play to innovate: How curiosity and experimentation transform UX

March 11, 2026

Bria Alexander

"We’re obligated to treat each other with kindness and respect."

Bria Alexander

Opening Remarks

March 29, 2023

Llewyn Paine

"If you can’t see a human in a video, how do you know the entire conversation wasn’t fabricated?"

Llewyn Paine

[Demo] Deploying AI doppelgangers to de-identify user research recordings

June 5, 2024

Bria Alexander

"When teams are connected to real strategic missions and own their key results, OKRs become positive."

Bria Alexander Benson Low Natalya Pemberton Stephanie Goldthorpe

OKRs—Helpful or Harmful?

January 20, 2022