Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Alexia Cohen
Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona
2024 • Advancing Service Design 2024
Gold
Rachael Dietkus, LCSW
The power to heal and harm
2025 • Advancing Research 2025
Gold
Jessamyn Edwards
Surviving Your UX Career in Enterprise Design
2021 • Enterprise Community
Bria Alexander
Opening Remarks
2023 • DesignOps Summit 2023
Gold
Jemma Ahmed
Collaboration: learning from other fields beyond our own [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Ian Swinson
Designing and Driving UX Careers
2016 • Enterprise UX 2016
Gold
Anna Avrekh
Expert Panel: Leading in and with Research
2022 • Advancing Research 2022
Gold
Robert Reimann
Taming Design Complexity with UX Models
2017 • Enterprise Experience 2017
Gold
Sandra Camacho
Creating More Bias-Proof Designs
2025 • Rosenfeld Community
Saara Kamppari-Miller
DesignOps for Inclusive Design and Accessibility
2022 • DesignOps Community
Jilanna Wilson
Distributed Design Operations Management
2019 • DesignOps Summit 2019
Gold
Lada Gorlenko
Theme 1: Intro
2024 • Enterprise Experience 2020
Gold
Yasmine Khan
Checking Bias and Listening to Financially Vulnerable Americans
2020 • Advancing Research 2020
Gold
Kevin Bethune
Reimagining Design: Unlocking Strategic Innovation
2022 • Design at Scale 2022
Gold
Indra Klavins
A Design Ops Girl in a Dev Ops World
2019 • DesignOps Summit 2019
Gold
Chris Chapo
Data Science and Design: A Tale of Two Tribes
2015 • Enterprise UX 2015
Gold

More Videos

Sheri Byrne-Haber

"Accessibility at scale can put you into the VUCA world – volatile, uncertain, complex, ambiguous."

Sheri Byrne-Haber

Accessibility at Scale

June 9, 2021

Prayag Narula

"When UX researchers and product managers collaborate, the product speaks to users’ hearts."

Prayag Narula Hannah Hudson

Empowering Designers to do Good Research

March 11, 2022

Janelle Estes

"The hardest skill to recruit for is someone who can consult and ask the right questions to uncover what the request really is."

Janelle Estes

UX Research Trends

January 28, 2021

Craig Brookes

"Collaboration without ego is key to building the right product, not just what one's own interests are."

Craig Brookes Andreas Huebner Morgan Quinn

"Just Make it Look Good" and Other Ways We're Misunderstood

June 11, 2021

Marc Fonteijn

"Slow and intentional communication allows you to be fully present rather than distracted by constant notifications."

Marc Fonteijn Ru Butler

Increase your confidence, influence, and impact (through a Professional Community)

December 3, 2024

Kate Towsey

"Managing vendors is a full-time job, and research ops teams often coordinate numerous vendors for tools and recruitment."

Kate Towsey

The State of ResearchOps: More Than Just Theory

June 20, 2019

Alla Weinberg

"Co-regulation means someone who is safe can help bring another out of a triggered state."

Alla Weinberg

Design Teams Need Psychological Safety: Here’s How to Create It

September 9, 2022

Shawna Hein

"The collaboration cycle requires following standards for accessibility, content, design system usage, and IA or you don’t launch your product."

Shawna Hein Kevin Hoffman

Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts

November 17, 2022

Tricia Wang

"You need to build credibility. It's not enough to just state a problem, you have to show you'll do the work and push for it."

Tricia Wang

SCALE: Discussion

June 15, 2018