Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Laureen Kattan
Centering Patients and Clinicians in a Complex Government Ecosystem
2023 • Design in Product 2023
Gold
Sha Hwang
The Lost Year
2021 • Design at Scale 2021
Gold
Ron Bronson
Design, Consequences & Everyday Life
2022 • Civic Design 2022
Gold
Steve Turbek
Designing Interactive Graphics with AI Code Help
2026 • Rosenfeld Community
John Maeda
Making Sense of Enterprise UX
2016 • Enterprise UX 2016
Gold
Jacqui Frey
Setting the Table for Dynamic Change
2019 • DesignOps Summit 2019
Gold
Cennydd Bowles
Exit Interview #2: Rediscovering the ethical heart of design
2025 • Rosenfeld Community
Catt Small
What's Next for ICs: Exploring Staff and Principal Designer Roles
2024 • Rosenfeld Community
Jake Burghardt
Stop wasting research: Unlock more value from research insights
2025 • Rosenfeld Community
Mark Boulton
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Cassini Nazir
The Dangers of Empathy: Toward More Responsible Design Research
2023 • Advancing Research 2023
Gold
Dave Hoffer
UX Job Search AMA #2 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Joseph Williams
Unlocking impact and influence through inclusive hiring in research
2021 • Advancing Research Community
Nathan Curtis
Beyond the Toolkit: Spreading a System Across People & Products
2016 • Enterprise UX 2016
Gold
Jon Fukuda
All the Ops: Successful cross-functional collaboration
2025 • DesignOps Summit 2025
Gold
Dr. Nikki Smith
Research Strategy: Connecting Insights to Outcomes
2025 • Advancing Research 2025
Gold

More Videos

Megan Blocker

"Right sizing research respects communities by matching the scope to what participants can reasonably engage with."

Megan Blocker Amy Bucher Katie Hansen Ricardo Martins Nidhi Singh Rathore

Day 2 Theme Panel

March 12, 2025

Dave Hoffer

"F them if they don’t see your skills or fit; the market is huge with varying needs and companies."

Dave Hoffer Joanne Weaver

UX Job Search AMA #3 with Joanne Weaver and Dave Hoffer

July 16, 2025

Sarah Brooks

"Healthcare.gov was a blow, but it opened the door to new ways to work in government technology."

Sarah Brooks Jennifer Pahlka

Fireside chat with Sarah Brooks and Jen Pahlka

October 21, 2021

Jen Briselli

"You can’t drive learning like a project plan, but you can create conditions where learning becomes more likely."

Jen Briselli

Learning Is The Engine: Designing & Adapting in a World We Can’t Predict

April 16, 2025

Nidhi Singh Rathore

"Too often research is something we do to people, but participatory research invites participants as co-creators of knowledge."

Nidhi Singh Rathore Amber Davis

Embracing participation to unlock deeper truths in commercial research

March 12, 2025

Jamie Beck Alexander

"Two UX designers at Amazon built a movement of 8,700 employees that led to Amazon announcing their climate pledge."

Jamie Beck Alexander Nina Gregg Shawn Petersen Bill DeRouchey

How can you find your role in climate?

January 17, 2024

Sheryl Cababa

"Systems thinking requires constant engagement with stakeholders; working in isolation will not lead to successful outcomes."

Sheryl Cababa

Expanding Your Design Lens with Systems Thinking

February 23, 2023

Andrew Custage

"While technical errors matter, broader issues like content relevance and navigation create more mission failures."

Andrew Custage Michael Mallett

The Digital Journey: Research on Consumer Frustration and Loyalty

March 29, 2023

Cheryl Platz

"Embrace chaos as a gift, a constraint that fuels innovation and growth mindset."

Cheryl Platz

Collaborative Creativity through Improv

November 7, 2018