What I learned as a user research intern at Spotify
During my internship at Spotify, I practiced a broad range of research skills and learned important lessons about product development. By doing research that supported several different products, I gained experience in:
- Conducting guerrilla interviews
- Planning and moderating in-person interviews
- Conducting and co-facilitating remote interviews
- Eliciting deeper insights from participants
- Moderating usability tests and analyzing findings
- Analyzing, annotating and consolidating insights from user testing videos
- Generating hypotheses from preliminary research
- Collaboratively designing and analyzing an A/B test
- Designing, piloting, and distributing surveys
Analyzing qualitative data
- Thematically analyzing open-ended survey responses
- Working with data scientists to triangulate research questions
- Getting designers, data scientists and product owners involved in research sessions
- Presenting actionable insights to designers, product owners and other stakeholders
Communicating research insights
- Curating research insights with video
- Turning research into a story
Here's an example of how I presented research findings to my product team at Spotify. (These slides are based on the intro section of a slide deck that I co-designed with another intern. Product details are redacted and photos are taken from diverseui.com):
Below are some of the broader principles of product development that I learned:
Hospitality is a major 🔑
Spotify's culture of hospitality is part of what makes it an effective company and a great place to work. There is an implicit (and explicit) understanding, for example, that diversity is a strength and a priority for the company. Senior executives, mid-level managers and product owners all embody this commitment, and it has become a social norm in the company. Because of this, people from traditionally underrepresented backgrounds can thrive.
A second dimension of hospitality has to do with knowledge sharing. Spotifiers understand that insights and expertise don't belong in silos. They are generous with their time and always open to fielding questions from people beyond their team. This openness creates lines of communication that allow knowledge and insights to circulate through the company. As a result, product teams can avoid repeating each other's mistakes, capitalize on each other's breakthroughs, and build products that naturally complement each other.
User research + data science = 🔥
Working closely with data scientists at Spotify has changed the way I think about user research and opened my mind to new ways of gathering information and prioritizing research questions.
Imagine that your product team just released a new feature, but users are slow to adopt it. Half of the users see an in-app announcement telling them about the new feature, while others did not. What is causing the issue? Are people not noticing the new feature? Is something about it confusing?
Before jumping into usability testing, it would be wise to sit down with a data scientist and find answers the following questions:
- Who is consistently using the feature? What's distinctive about them, in terms of demographics or behavior?
- When are people using this feature? What activity are they doing in the system at the time they use the feature?
- When people try the new feature for the first time, how likely are they to keep using it? Is there a common falloff point?
- The people who use the new feature for a short period but stop -- what's distinctive about them, in terms of demographics or behavior?
- Do people on different platforms (iphone vs. android vs. desktop) use this feature at the same rate?
- Are people who got the announcement significantly more likely to try the new feature?
The answers to these questions will guide the next research steps.
Imagine, for example, that the data scientist finds that once people try the feature, they tend to keep using it. But she also discovers that very few users are trying the feature -- even the ones who got the announcement about it. This would suggest that the announcement may not be working. It might not be well structured, or maybe people are just ignoring it. In this case, it would make the most sense to design and test new announcements.
Or maybe the data scientist finds that lots of people are trying the feature, but few continue using it. This would suggest that the feature itself may be confusing, frustrating, or unhelpful. In this situation, the data scientist could identify some of these "early quitters," and the user researcher could contact them for interviews and usability tests. This would help the team understand why they stopped using the feature.
By thinking through questions like this with a data scientist, user researchers can make more informed hypotheses about what's happening, and better recommendations about how to address it.
Communicate across boundaries 💬
A team can solve problems much more quickly and effectively when its members have a good understanding of each other's work. One of the strengths of my team at Spotify was our ability to "speak each other's language." Designers and product owners understood user research, for example, while product owners and user researchers understood how data scientists ran A/B tests. This allowed us to communicate seamlessly and think through challenges together.
A team that plays together stays together 😜💯
It's a lot easier to collaborate with people when you've had coffee with them, shared a meal, or played an intense game together. By organizing fun outings for parts of the product team, our managers created an environment where people trusted each other and were comfortable giving candid feedback.
Experiment, learn, improve 👩🔬🤔📈
When most people think of innovation, they think of novel products and services. But real innovation happens when you iterate on your process, not just your prototype. As user researchers at Spotify, we were constantly exploring new strategies for:
- recruiting users;
- simulating product experiences for usability testing;
- recording and distributing our research data, and
- packaging our research insights for the broader product team.
This culture of innovation was largely driven by three words: "What if we...?"
The things we tried didn't always work, but the point was to always be learning. When we did find success, we shared our insights with the wider team, and made it available to other user researchers in the company as well.