Real Talk: Proving Value through a Scrappy Playbook

Dianne Que
Sr. Design Program Manager, Pandora

From the DesignOps Summit website:
Most DesignOps practices, whether new or established, tackle a handful of common areas: hiring, workflows & process, culture & morale, among others. While you can find plenty of tools and best practices for these areas on- and offline, there are no one-size-fits-all solutions. Every company, team, toolset, and timeline is different, and the road to solutions is often, if not always, messy.

Based on my experiences co-building DesignOps and Design Management practices at Pandora and Capital One, this talk will pull back the curtain on common problems we’ve been asked to solve and the scrappy, yet effective ways we’ve delivered early solutions, value, and measurable outcomes.

She joined Pandora about six months ago. Their Chief Product Officer asks them what they have been jamming to.

The topic of re-orgs, leadership changes, attrition, and new hires are familiar to many of us. These things lead to ambiguity, instability, and uncertainty. That was the state of things when she first joined Pandora, and it was compounded by the acquisiton by SiriusXM. There was fear and uncertainty in the air.

They assessed their teams’ challenges – how the handbook might help and committed to measuring impact and outcomes over time. Taking a scrappy approach … what does she mean by ‘scrappy’? From Urban Dictionary: “Seemingly small and threaning, but shockingly able to kick your ass.” Her definition … “don’t let perfect get in the way of better”.

She did a listening tour and learned that there was a lack of clarity around processes, roles and responsiblities. There were processes were in wikis, in heads, in hard to find places … but not in a concise, digestible way. In order to prove value, we needed to understand their issues and what they valued.

Their handbook started in Google Slides, and it was based on Richard Hackman’s Five Factor Model for high-functioning teams:

She is going to focus on #2 and #3 and how it helped with their challenges. They knew the team needed to have a common sense of vision and purpose.

For the #2 compelling direction, they collected the following:

The last one was not well documented and clear, which was part of the problem. But it presented an opportunity for leadership to ensure they were articulating their vision and alignment with the other parts of the pyramid.

#3 was about enabling structure. This included frameworks, workflow, processes. They documented the following areas:

As part of the playbook they needed to clarify the role of Design Ops. They built a good part of this based on what they learned at the 2018 DesignOps Summit:

Within feature teams, roles and responsiblities also needed to be better defined. The workflow was simplified for stakeholder discussions, and then referred to detailed views in the Wiki over time.

In addition, rather than describing work in an inconsistent way, they started using McKinsey’s Horizons framework so people began using consistent language to describe whether the work was on the roadmap, on a “future roadmap”, or an “innovation”, for example.

They also started conversations about how they could be better organized. So that helped to demonstrate a focus on processes of today, but how we could organize and adapt to address their future aspirations.

Dianne reminded us to consider Visibility, Visualizaton, and Vocabulary. Speaking the same language alone add lots of value. But the playbook also became a springboard, and it was a forcing function to understand the as-is so improvements could be identified. It also hepled their teams feel better.

We’re not always proving value upwards, but also across and down.

It took them about two months to prepare the playbook, and their boss is the VP of Design and Product Management. He is very busy and they didn’t want his schedule to hold them back, so they rolled it out without his approval.

Intentionally, they chose clarity as a measure. Lack of clarity was causing confusion, spin, anxiety. Without clarity some of the smallest bumps were getting harder and harder to overcome. They measured clarity before (3.1) and after (4.6). They also received very positive feedback from stakeholders.

Dianne encouraged us to turn feelings into data. It makes people feel better is a valid metric. So much of what we provide is qualitative, so this is something they are trying to move forward in their initiatives.

Less intentionally – content has been extended and repurposed in other ways – like onboarding for new hires, for new product kickoff, executive share-outs, and randomly in others’ presentations.

These changes are slowly influencing how they work, both inside and outside their organization. It has also changed how they collaborate – how to incorporate that thinking into future initiatives, or into roadmapping with engineering. They have also seen some acceleration – less time focused on aligning up front. But they still have not determined the best way to measure those types of outcomes for future initiatives.

Outcomes emerge over time – now that they know what they know, she realizes the importance of measuring baselines.

The process of communication, information sharing, and how it could be used and evolved in different scenarios – ultimately it was not just about the playbook itself. But, they re-orged a month after rolling out the playbook.

The reality is that there is no end-state in DesignOps. Let’s not let perfect get in the way of better. The framework is there – the content will need to continue to be refreshed, and other teams need to hear the roadshow. They are also going to continue to refine their DesignOps metrics.