Natalie Hanson Operating in Context
Opening Keynote: Operating in Context
Leisa Reichelt, Head of Research and Insights, Atlassian
From the Design Operations Summit website:
You know that thing where you start a new job and suddenly realize that all those great rules of thumb you thought were almost universally true are either impossible or ineffective in your new organization. Welcome to Leisa’s life. In this talk Leisa will share her experience of completely resetting her idea of best practice, implementing a strategy that is the opposite of what everyone expected, and why so few people do what they think is right. You’ll also get bonus thoughts on how to best set up your research team for maximize effectiveness.
With her boys, she sees and appreciates certainty you have when you’re young. “I know, Mom.” In contrast, she feels uncertain about so many things – and she feels that humility keeps the door open to learning. She was a promoter of false certainty, trying to move past that.
We know that context is super important. In her most recent job move, she felt like she was starting from scratch, and that was the inspiration for this talk. Just like when desinging products and services … organizing teams and ways of working varies with context. There is no silver bullet. She is going to give us a tour through her last three jobs.
In London she was part of the Government Digital Services (GDS). Before that she was deliberately freelancing; this was her first in-house job. She realized that the needed to change the organization to make sustainable change, not just their products and services. The GDS were right in the center of the government.
They had seven founding digital principles, and there was a lot of thinking about users, but not much observing of or talking to or meeting users. Usability Testing was considered a psuedo-science. There were four researchers on staff, covering 25 product teams. It was an impossible setup for them to succeed.
Compared to other work she had done, it was not the most complicated. But you do have to make it work for everybody. She was working as part of a colocated, experienced team.
Her challenge was how to set up a research team to be successful, how to overcome the skeptism. Through legacy budget, one product manager was able to hire a researcher and didn’t have to share. That helped to showcase what good looked like, which the government adopted over time.
She established the following targets:
- One researcher per team, at least three days a week (so they couldn’t be shared across teams and lose focus). It was considered a huge luxury at the time.
- Every sprint, the goal was to conduct reserach with at least 5 users, including at least one participant with special access needs.
- Everyone in the team should get a least two hours every six weeks of ‘exposure hours’.
They eventually build a lab with a huge observation space. They wrote about it, made posters. In retrospect, there is only one rule she would keep – “find what works, not what’s popular”.
How did she get budget for 20 people? She asked for what she needed, even if she knew the organization would think it was outrageous. We often back down from our own needs and assumptions.
Don’t negotiate before you get to the table!
People will not say yes if you don’t bring Plan A to the table. She asked for 20 and she got 12. The way she addressed the gap was that she followed her model, and left some teams without research support. She did eventually get the 20. At one point there were more researchers than engineers.
Over time, she and her husband realized that they wanted to live closer to family. An opportunity came up that looked really familiar, and so she joined the Digital Transformation Office (DTA) in the Australian government. It had inherited similar structure and programs. There were some small differences, like having a Federal, State and Local structure, where in the UK it was just Central and Local. She wasn’t sure that it was going to make her practice different. Her head office was in a town nothing like London. Because Australia couldn’t agree on which city to make the capital, Canberra is the capital city, in between Sydney and Melbourne. It’s filled with public servants. Technologists and designers who are born there leave because of the governemen’ts terrible reputation with technology. So, she set up her office in Sydney, but that meant she had to travel a lot.
Like the UK, they were simple transactions using websites and forms – not that hard. But teams were distributed (Sydney, Canberra), and teams were significantly less experienced. People that weren’t exposed to agile, had never worked in cross functional teams. And no experienced researchers, no designers. They had to ship in service designers from other agencies to try to make things happen. Her role became about unifying how they worked so they could run a coherent program.
People in Canberra are not representative of the county at large – over educated, over paid. Much harder in a small town, so another difference from the UK is that they had to travel for research to get diverse input. So, there was no lab, no observation room. But, there was a lot more hands on experience (across functional teams) with research. There wasn’t a rigid research cycle, because they had to find the research where they could.
She developed a new doctrine to support this context, including:
- Embedding researchers in product teams, augmenting especially in cases where there were no formally trained researchers.
- Geographic distribution of their audience required that everyone participated in research to some degree, with some standardization of processes and tools.
- Rather than the Agile cadence, they had continuous research efforts punctuated by readouts every two weeks.
Research findings were shared onsite through a Showcase (readout) approach, which helped teams see each others’ work. It also resulted in new kinds of artifacts like empathy walls – visercal ways to bring insights back from the field.
In one project, they eliminated a six page form required to get newborns access to health services. That project brought together a service designer, lawyer, and technical architect in a room, and the outcome was to eliminate the form entirely.
Most recently she joined Atlassian – they make JIRA, Confluence, and other software. People thought that getting out of government would be easier. But from the outset, people told her they weren’t doing to stop doing the research they were doing (which included lots of surveys and interviews). She did a discovery project, which gave her a good landscape of what was going on, where the pain points were – that helped her decide what to prioritize.
Unlike government services, Atlassian applications are complex. Product teams and platform teams – it’s not always clear where to put the researcher to have an impact. The team was distributed, five locations and three timezones. She ‘confident amateurs’, not always justified.
If you are doing multiple things in your role, you think you’re expert at lots of things. But early on in your learning journey you feel super confident. You have to learn what you don’t know – but there isn’t necessarily justification to do so,
How to keep that enthusiasm, but make sure we’re moving along this knowledge axis? The only thing she knew for sure is that what she had done in the past would not work here. This time around, she has established different priorities that reflect her new context:
- Align to corporate goals. She mapped team activitiy to company OKRs, which gave them a powerful rationale for her decisions.
- Build capability. Pair researchers for professional development – even if that meant doing less work. Hire for breadth of experience.
- Encourage a user-centric not product-centric mindset. Contribute inisghts that illuminate product-centric blind spots, and synthesize with other types of data.
All her prior work had been with embedded resarchers. She had to pull them out – it was pretty contentious. The familar path wasn’t gonig to work. But most of the prior research was feature and product validation, and she knew that needed to change.
They are currently using Top Tasks methodology, adding depth using Google HaTS survey, and usability benchmaking. She is working closely with the teams, but outside of them. And combined with Product Analytics from support, product management insights, etc,
It is yet again a very different way of working.
Which hill to die on? So many hills, so many monkeys! In her experience, problems that really matter will come around again.
Silver bullets are only for werewolf problems. 🙂
Think about the context, and the characteristics of your team; embrace context dependency. Solve problems with agility, creativity, and experimentation – as a designer would.
Natalie Hanson 
Pingback: Design Ops 2018 – Recap | Natalie Hanson