Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Louis Rosenfeld
About the Rosenverse
2026 • Rosenfeld Community
Tess Dixon
C'mon Get Happy
2021 • DesignOps Summit 2021
Gold
Darian Davis
Lessons from a Toxic Work Relationship
2024 • Enterprise Experience 2020
Gold
Megan Nipe
Human-Centered Design for Engagement: Maturing from Newsletterville to Personalized, One-to-One Messaging
2021 • Civic Design 2021
Gold
Clemens Janssen
Efficiently Scaling Research as a Team of One
2023 • Advancing Research 2023
Gold
Mackenzie Guinon
M.C. Escher’s UX Research Career Ladder
2022 • Advancing Research 2022
Gold
Jake Burghardt
Finding More Inroads into Research Impact
2026 • Rosenfeld Community
Peter Van Dijck
Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!
2025 • Rosenfeld Community
Catt Small
Craft a Vision that Actually Gets Shipped
2026 • Rosenfeld Community
Shahrzad Samadzadeh
What Is My Value? Two Takes and Some Mistakes
2024 • Enterprise Experience 2020
Gold
Brigette Metzler
Scaling ResearchOps: Helping Researchers do Their Best Work
2020 • Advancing Research 2020
Gold
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2017 • DesignOps Summit 2017
Gold
Chris Engledowl
A Mixed Method Approach to Validity to Help Build Trust
2023 • QuantQual Interest Group
Llewyn Paine
Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration
2025 • Advancing Research 2025
Gold
Christian Rohrer
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold

More Videos

Rebecca Gimenez

"Cycles of successive approximation help us get closer and closer to great design through iteration."

Rebecca Gimenez

Work in Progress: Service Design at Airbnb

December 3, 2024

Lada Gorlenko

"Designers can be first responders, not just in digital but in real disaster situations."

Lada Gorlenko

Theme 2 Intro

June 9, 2022

Brigette Metzler

"Research repositories and libraries are social things — many teams look for best practices beyond just building a library."

Brigette Metzler Dana Chrisfield

Research Repositories: A global project by the ResearchOps Community

August 27, 2020

"Delight forums helped us share stories and build a shared vernacular but didn’t lead to significant change by themselves."

Discussion

June 9, 2017

Sarah Brooks

"Community and ‘the phone a friend’ are what get you through the hardest parts of working in government."

Sarah Brooks Jennifer Pahlka

Fireside chat with Sarah Brooks and Jen Pahlka

October 21, 2021

Lisa Spitz

"We actively coach Department of PFML staff to shadow and eventually run their own research sessions so they’re empowered long-term."

Lisa Spitz Nikki Brand

Building Trust Through Equitable Research Practices

November 18, 2022

Theresa Neil

"We were doing all the things until we got the advice to specialize; that moment changed everything."

Theresa Neil

Designing for Wellness: Specializing in Healthcare

May 22, 2024

Maverick Chan

"Clients got used to AI-driven storytelling pretty quickly when involved in weekly progress updates."

Maverick Chan Claire Lin

From Doodle to Demo: AI as Our Storytelling Partner

October 23, 2025

Alla Weinberg

"When we’re mobilized or immobilized, our IQ literally drops by half and our peripheral vision narrows."

Alla Weinberg

Design Teams Need Psychological Safety: Here’s How to Create It

September 8, 2022