Measuring the Designer Experience

Kim Fellman Cohen
Design Experience Lead, Pinterest

From the DesignOps Summit website:
DesignOps teams are increasingly being leveraged to solve ambiguous organizational problems like career development, culture or belonging—they’re cornerstones for any successful team. Creating a measurable strategy for these problems is complex, and it can be hard to prove the value of dedicating full-time resources to maintain this work in the long-run. Based on our learnings at Pinterest, I’ll share tactical approaches you can take to craft programming for this problem space, define what ideal states look like and how you can measure and prove value.

Kim works on Design Experience at Pinterest. How did we end up in operations? They had people-related problems – career development, skill building, team events, morale and more – in addition to the work they do to drive the products forward.

What if you had more time to do this on your own? It’s not just solving complex people problems, and it’s not just about demonstrating value. It’s about showing measurable impact so you get the time and resources you need.

She is going to cover these areas: stay connected and learn, define problems and ideate, formula for measurement, test ideas like a product, and report / look forward.

They are 100 people, spanning 8 disciplines – product designers, UX writers, UX engineers, creative technologists, designops, systems designers, researchers, production designers.

People and Cuture efforts were often put aside when product demands were high. As a result, employee satisfaction survey results were concerning – especially along the dimensions of not being able to do their best work, employee retention, and a sense of belonging.

How to measure a tidal wave of feelings emerging from this company survey? A time sensitive issue may result in reactive solutions instead of proactive ones. Risks can be high, or there are intractable problems – in their case, missing the budgeting period and having to operate without funding. Feedback can be vague or subjective, and the rate of change is rapid. So, these problems require some innovation on our side.

They made the decision to use Design Thinking, and to work with other groups like HR or external consultants. The identified clear metrics, and proved them out in pilots with a clear beginning and end.

Her recommendations:

Stay connected and learn. Frame a set of questions that help you learn from murky feedback. What are 1-2 things keeping you from doing your best work this week? Focus groups were awesome, especially with good data as starting point for conversation. Regular surveys … even a whiteboard where people (especially introverts or those who need time to reflect) can contribute post-its. Making people feel heard builds credibility. Those learnings can also help you determine how to measure success.

Take time to define solvable problems. Use some basic criteria – is this problem the right scope and size? Does this problem provide enough context for ideas? Is is broad enough room to explore new ideas? Their key issues were related to doing their best work, making career paths and skills more clear.

Ideate together. They collected a diverse group of people to brainstorm with a diverse group. Mix levels of experience, background, roles, and tenure. Ensure people who are dealing with the problems are involved. But also bring in experts with a new perspective, and engage stakeholders. They uncovered some big ideas, and small ones too.

Synthesize using an idea criteria. Could you prototypie and test this in the next month? Do you have the resources to try this? Can you measure the outcome clearly? Green light ones that meet the criteria. Tackle just one or two things so you are not overwhelmed, and the quick wins are well though out and sustainable.

Define success with a formula.

Give yourself a timeframe that is aligned with the scope of work – they had about six months. The challenge in goal-setting is to be realistic with the time you have, and also to leave room for failure. They targeted 60% success, and 40% learning. They did that because of the health of their team at the time – a healthier team might strive for a higher success percentage.

Build and test your ideas like a product. Prototyping helps you figure out if your idea has merit. Don’t build it alone if you don’t have to, and then b e sure to pilot to learn early and often. Rather than thinking of those pilots as failure, they tried to reframe them as learning opportunities. Make sure that stakeholders are engaged.

Report results and think forward. They made improvements on all their prioritized dimensions. Make sure you provide context, and the ‘what was’, Share metrics and the results.

And then think a bit about the future – what’s next. Sharing a little qualitative feedback goes a long way as well. How might we shape the future? They improved, but they knew they weren’t in their final state. They wanted to reframe their aspirations. Their goal was to get into the 80-90% range – a big leap from the 50% where they started.

Desiging Thinking is not revolutionary, but it is new for us to apply it to these people problems in a measurable way. We are uniquely qualified to design these experiences for our teams.