Natalie Hanson Agile – Implications for UX Research
This Part 4 of a 4-Part series on Agile.
As I mentioned earlier, one of the pleasures of working in both Lean and Agile working environments is that it’s extremely flexible, adaptable, and emergent. There is lots of room for experimentation! However, that also means that the summary in my prior posts is by no means exhaustive.
It has been shown over and over again that the greatest Return on Investment (ROI) for User Experience is in early stages of product development. You can see one such study in my presentation at The Qualitative Report annual conference. The goal is to ensure that the right thing is being built / aligned with the needs of markets and users. That means that research plays a critical role very early on in product direction and definition. Ideally – but rarely! – you’ll be involved as a market is being identified and a product is just materializing.
But at the same time, as sites like Little Big Details reinforce, the devil is in the details when it comes to delivering a good experience through software. That means UX team members have to be engaged throughout, with strong transitions between research and design resources. Of course it depends on what the team is doing when you join. Are you there at inception, or do you join two years into the development of a maturing product?
In reality, you’ll probably land smack in the middle of a project, and be thrown into the deep end. The team will be towards the nearing the end of a sprint for one release, planning for the first sprint of the next release. You’ll have to figure out what is known, what is not, and where you can best contribute. In the following section, I’d like to provide some specific examples about ways that researchers can engage at each of the major stages in the development lifecycle of a software product.
In a controversial article entitled Act First, Do the Research Later, Don Norman argues that a project may not plan the time or make a commitment to the research that is required. Researchers should be engaged in shaping those decisions, and that it requires the ability to quantify the impact of our recommendations, much as market researchers do. But it is common for teams to be formed around a given market or product opportunity, which means that some significant decisions have already been made once that team is formed. Norman argues that researchers are often engaged relatively late in the planning and decision-making process, which limits the strategic impact of our findings. In other words, building a research plan when the market, users, or sales potential have already been defined is too late in the game. Norman argues that research skills need to change in order to earn a seat at the Boardroom table where those early and most critical decisions are made.
The only way to achieve this is to develop knowledge beyond your own discipline and methods and into the business issues that your research is intended to address. Only in this way will you truly establish trust with executives to earn a seat at that decision-making table. The reality is that you are most likely to make an impact with your findings when the research is sponsored by senior members of the organization. I have experienced over and over again that small, critical contributions in one project lead to repeat work and an increasingly strategic working relationship over time. In some cases, I was able to re-purpose or repackage earlier findings as a relationship matured. For example, what started as user research for a series of web pages expanded over a several year period into ethnographic research, and my team was eventually asked by senior management to summarize 2-3 years of findings to help with that group’s strategic direction.
The challenge is always to balance the effort between the forward-thinking research in advance of a release, and doing some quick, iterative testing to inform adjustments the product as it’s going out the door. The way I would characterize the former is build the right thing, and the latter is build the thing right. I documented my perspective on this topic for The Qualitative Report annual conference, but here it is again:
Again, both have value, though different methods and skills will surely be needed at different stages. Ethnographic methods might be desirable early on due to the richness and complexity of the findings, but when your team is just trying to ship a non-buggy product on time, some quick prototype or usability testing will do!
Below, please find specific examples about the role UX team members can play at these different stages. Note the waves at bottom, which represent the areas of highest engagement (and value) for UX Research, UX Design, and UX (or front-end) Development respectively.
![ScreenClip [1]](https://nataliehanson.files.wordpress.com/2013/02/screenclip-1.png)
So, although you may have a preference about where you want to be engaged (as I do) Building the Right Thing and Building the Thing Right have value, but you can’t always be in both places! So, where to begin? Well, it depends on who hired you and how much they trust the value of user insights! Here are some considerations:
- I think that it takes more knowledge of (and trust in) UX to spend the time doing research up front … because it takes longer for the impact of those findings to influence what ships. That type of research is also best suited for ethnographic methods, and perhaps also for innies (researchers that are members of the organization) rather than outies, as the delay in time-to-value may be tough for an external consulting team that’s trying to sell ongoing research.
- In an organization that is really committed to Agile, they may be unwilling to support Big Upfront Design (BUFD). In fact, BUFD is a four-letter word with hard-core Agile teams! Those teams are very likely working with UX generalists (not dedicated research and design experts) if they have a dedicated UX function at all. See for example this blog post called UX for Devs Manifesto, which seeks to explain to developers how to take responsibility for engaging users. Or this value proposition for the Lean UX NYC conference in April 2013:
The days of relying on a UX Designer for the whole experience are done. Collaboration has won out, and every team member needs to know how to contribute to product development. At LeanUX NYC, entrepreneurs, designers, developers and user experience professionals will learn about the entire product development lifecycle from accomplished practitioners and leaders in the LeanUX Design community.
- UX team members are strongly encouraged to find ways of working more efficiently than they have been historically, e.g. no paper prototypes. Furthermore, the team expects that all of the UX tasks will be planned and executed incrementally as the developments tasks are. Depending on the level of maturity in your UX organization and in your UX standards/libraries (as well as in your development organization), this is extremely challenging, but some practitioners swear by it!
- Some teams working in larger organizations recognize that it is not possible to orchestrate complex (e.g. enterprise-level) software development in this way. As I mentioned elsewhere in this series of posts, it make sense in those cases for UX practitioners to align with others thinking about the big picture – for example, product managers, systems architects, and business analysts.
The challenges you face and the way you most effectively engage with the project team has everything to do with how they are practicing Agile, how much they know (or have experience in) working with UX, and many other factors that are not (and will not!) within your control. Your best chance for success is to understand the forces at play and adapt. To quote Charles Darwin:
It’s not the strongest in the species that survives, nor the most intelligent that survives. It is the one that is most adaptable to change.
In Closing
If you’ve read my posts on Lean and Agile, I hope by now it’s clear why Lean and Agile are sometimes conflated. Both are driven by some common underlying concerns, including a focus on efficiency and transparency, a focus on the impact of work outside the project team, and a people orientation – within the team but also with customers. On the other hand, these practices are so very different. Lean is a means for driving change in organizations of all sizes, and – at least for the moment – my knowledge of Agile is limited to it’s use in software development, though I know it has applications in lots of other areas (e.g. manufacturing).
Agile also deeply shapes the organization in which it is implemented, and role of the hiring manager is critical here. In an Agile context, I feel that research is best aligned with senior management or with product management, where findings are delivered before development work gets underway so that research can inform what is built, not just how it should work and behave. Although both add value to the product in the long run, the former is more strategic (and also better suited to ethnographic methods). If the UX professional is hired by a development manager, there may be an expectation that research and design are deeply embedded in engineering activities. It is possible to deliver value, but the rapid cycles and very specific feedback require rethinking methods and historically lengthy research project timelines. On the design side, some efficiency will be critical, and can be achieved through re-use of via templates, design standards, and so on.
Both Lean and Agile contexts require similar things from researchers. Engaging in these settings means having some awareness of what external pressures and perceptions the company may be contending with. It’s important to understand those factors as they will have a profound influence on the organization. Take the initiative to read project artifacts and white papers, talk to product managers, learn how to speak a language that is accessible to executive decision makers. In other words, be a thoughtful participant observer!
Keep a critical eye and ear on the discourse of your work environment; whether you’re there on a temporary basis or as an employee, that awareness will be critical. Agile without stakeholder or user involvement misses the mark and cannot deliver its true value or potential. More and more (especially in the software development arena) these Lean and Agile may come together (as in Eric Ries’ book The Lean Startup and Dean Leffingwell’s Agile Software Requirements). Nonetheless, it can beneficial to understand the historical roots of these trends and why they are drawn upon as industries evolve and change.
The biggest challenge, for researchers, I think, is to recognize these shifts in the business setting, learn about the implications for their work, and to actively explore new ways to engage and contribute. I hope this series of blog posts will help achieve those goals! I would welcome your questions and feedback in the comments.
Natalie Hanson 
This is a great article. I appreciate in particular your realistic outlook. UX cannot just stroll into a product development environment and say “these are the things we are going to do and you are going to listen to us”. Executing the UX activities are easy — getting anyone to listen to you is the hard part!
Thank you for the article, earlier this year I attended an Agile meetup discussion group that covered this topic. It was from the scrum practitioners point of view and they asked ‘when should we include this?’ The conclusion was that it should be considered at the earliest point. This was great news for me as I am working on a product aimed at agile practitioners to help them ‘discover’ requirements using an approach very similar to user stories. We recommend this early discovery stage includes UX people as you need to consider the user and their goals so you can figure out what the heck you need to build! We’re still in early stages but you can find out more at the-skore.com.
@Sam Thanks for your feedback. Yes, being integral to and valued by the team you’re supporting with your UX skills is definitely a critical success factor!
@Craig I look forward to learning more about The Skore – thanks for the tip!
Hello Natalie,
Thank you so much for posting 1-4 series on Agile. I must say that post one really made me laugh out of that shared complicity that comes with being an anthropologist learning a foreign language expressed in “Sprints”, “Scrum” or any other ‘exotic’ terminology, knowing that one will get the local lingo at some point. As someone who just did their first work with Agile this is useful. I share your feeling about ‘user stories’ and the shift to “outcomes” that you point to is important and a good contribution, in my view. It’s like going from problem-focused to solution-focused.
I was particularly keen in learning a bit more on the situations when you think quick “testing” is more adequate than ethnography, what kind of tests you think are useful and how to marry that up with “ethnography”,as I am working on an article at the moment that tries to think through that question. Anyway, that would probably make too long of a post, so I can fully understand why you didn’t go deeper on that front. Thanks for sharing all this. Pedro
Hi Pedro – thanks for taking the time to read the posts and share your feedback – I’m glad you found them both funny and useful!
I have said before and I continue to believe that ethnographic research is really better for forming an understanding of a new domain or a new market or a new group of users. Or … gaining a deeper understanding of how users are engaged in a particular set of tasks (perhaps with a competitive solution). The challenge with Usability Testing is always that it can only answer questions that you’ve thought to ask!
But please also keep in mind that there are many types of usability testing, and it will be important to understand what kinds of tests to use and when – formative testing for early stages of product development, and summative testing for products nearing completion. Understanding these nuances will allow you to help shape the testing protocol that makes the most sense for where you are in the project and the types of insights you need.
All best,
Natalie
Hi! I could have sworn I’ve visited your blog before but after
going through many of the posts I realized it’s new to me.
Regardless, I’m definitely happy I came across it and I’ll
be bookmarking it and checking back frequently!