Rosenverse
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Lean Engineering: Engineering for Learning and Experimentation in the Enterprise

Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Speakers: Bill Scott
Link:

Summary

Bill recounts his journey from Netflix, where he witnessed the shift from DVDs to streaming, to PayPal, where he led transformative engineering reforms. At Netflix, he realized that designing software as a throwaway experimentation layer rather than permanent code was key, with multiple concurrent experiments driving user-focused learning. He stresses that engineering should enable learning rather than just code stability. At PayPal, Bill faced legacy technical debt and cultural inertia (organizational antibodies) but pushed for a culture of rapid iteration, collaboration, and customer immersion. He implemented a new technology stack based on Node.js and GitHub, democratized innovation through an internal open source model, and emphasized the need to give Agile a 'brain' by embedding continuous user feedback deeply into the backlog and process. Bill highlights the importance of shared vocabulary between disciplines, collaboration, and continuous customer feedback to keep teams aligned and focused on solving real user problems rather than defending solutions. Drawing on examples from Netflix, PayPal, and Meetup, he underscores that successful teams embrace failure in small increments, enable rapid prototyping, and design for volatility.

Key Insights

  • •

    At Netflix, 95% of the UI layer was thrown away within a year, reframing UI as an experimentation layer rather than durable software.

  • •

    Engineering's primary goal should be enabling learning, partnering closely with design and product teams.

  • •

    Different parts of a software stack have different risk profiles; applying the concept of shearing layers helps accept more risk on the user interface layer to enable faster learning.

  • •

    Organizations contain 'antibodies'—cultural and organizational forces resistant to change—that must be understood and navigated to drive transformation.

  • •

    PayPal's transformation was accelerated by a top-down mandate combined with intense cross-functional collaboration and frequent user testing.

  • •

    Using internal GitHub and open source paradigms democratizes code access and innovation, allowing anyone to contribute and experiment.

  • •

    Prototyping should be considered a first-class engineering activity, not a separate or lesser process.

  • •

    Agile methods lack an inherent ‘brain’; embedding continuous customer feedback and real user context into the backlog gives Agile teams direction and purpose.

  • •

    Engineering teams that share vocabulary and deeply collaborate with designers and product managers produce better outcomes.

  • •

    Embracing small incremental failures enables faster learning and avoids the risk of large-scale failures that can paralyze organizations.

Notable Quotes

"I started thinking of the UI layer as the experimentation layer."

"Engineering’s number one goal should be to enable learning."

"You have to design for throw away ability because the majority of experience gets thrown away."

"Organizations contain antibodies—cultural forces that resist change."

"If you don’t soak teams in real customer context, they do dumb things not because they’re dumb, but because they lack context."

"Features eventually become barnacles that are impossible to scrape off the boat."

"Agile needs a brain, and that brain is the continuous customer feedback loop."

"Prototyping isn’t a second-class citizen, it should be a first-class citizen."

"Democratizing code with an internal open source model accelerates innovation."

"I know I will fail, but I will fail in small increments rather than bet everything and fail big."

Ask the Rosenbot
Tatyana Mamut
Opening Keynote: Breaking Conway's Law--or How to Work Differently and Not Ship Your Org Chart
2019 • Enterprise Experience 2019
Gold
Mike Oren
Why Pharmaceutical's Research Model Should Replace Design Thinking
2023 • Advancing Research 2023
Gold
Peter Van Dijck
Building the Rosenbot
2024 • Designing with AI 2024
Gold
Nathan Shedroff
Double Your Mileage: Use Your Research Strategically
2020 • Advancing Research 2020
Gold
Luke Wroblewski
Designing AI Applications
2026 • Rosenfeld Community
James Rampton
The Basics of Automotive UX & Why Phones Are a Part of That Future
2024 • Rosenfeld Community
Kim Holt
A Salesforce Panel Discussion on Values-Driven DesignOps
2022 • DesignOps Summit 2022
Gold
Andy Barraclough
From Costly Complexity to Efficient Insights: Why UX Teams Are Switching To Voxpopme
2024 • DesignOps Summit 2024
Gold
Crystal Philcox
The Many Faces of Operations
2017 • DesignOps Summit 2017
Gold
Megan Blocker
Day 2 Theme Panel
2025 • Advancing Research 2025
Gold
Prabhas Pokharel
Order and Chaos: New Ways of Collaborating on Synthesis and Storytelling
2022 • Advancing Research 2022
Gold
Chris Hammond
Embedding sustainability into enterprise design and development: A journey towards "sustainability consciousness"
2025 • Climate UX Interest Group
Matt Bernius
Trauma-informed Research: A Panel Discussion
2021 • Advancing Research Community
Alla Weinberg
People Are Sick of Change: Psychological Safety is the Cure
2023 • DesignOps Community
Xenia Adjoubei
Empowering Communities Through the Researcher in Residence Program
2023 • Advancing Research 2023
Gold
Chloe Amos-Edkins
A Cultural Approach: Research in the Context of Glocalisation
2023 • Advancing Research 2023
Gold

More Videos

Mark Interrante

"AI can accelerate prioritization tasks, but the final judgment must come from humans who understand context."

Mark Interrante Harry Max

AI for Prioritization (3rd of 3 seminars)

July 11, 2024

Patrizia Bertini

"The accessibility expert was simply lost into shouting, we must be accessible without creating a strategy."

Patrizia Bertini

The (r)evolution of designOps: It’s Time to Think (really) BIG

September 11, 2025

Shan Shen

"Start small, try to learn as you go, and set realistic expectations to avoid being humbled when things go live."

Shan Shen

Translating UX Terms into Business Contexts

November 29, 2023

Adam Thomas

"Stop, pivot, or invest early—these are the survival metric decisions that save companies."

Adam Thomas

Survival Metrics – Making Change in a Fast, Data-Informed, and Politically Safe Way

December 6, 2022

Robin Beers

"Researchers have superpowers: deep listening, empathy, making connections, synthesis — capabilities everyone in business needs today."

Robin Beers

Research as a Catalyst for Organizational Transformation

March 12, 2021

Sheri Byrne-Haber

"Assistive technology has to communicate with the destination, that’s your website or product."

Sheri Byrne-Haber

Accessibility at Scale

June 9, 2021

Joseph Meersman

"There is no formula for good critiques, but mindset is behavior over time: humility, active listening, gratitude, owning blind spots, and acknowledgment."

Joseph Meersman

Sweating the Pixel: Scaling Quality through Critique

June 10, 2021

Deanna Washington

"Go meet someone new at this conference and set up a coffee chat next week."

Deanna Washington Bria Alexander Jon Fukuda Saara Kamppari-Miller Farid Sabitov

Connecting the Ops: Plenary Panel and Closing Circle

September 9, 2022

Aaron Stienstra

"Citizen participation is like eating spinach—everyone agrees it’s good, but few actually do it meaningfully."

Aaron Stienstra Lashanda Hodge

Leveraging Civic Design to Advance Equity and Rebuild Trust in the US Federal Government

December 8, 2021