How user research changes during design and delivery

The kind of user research you do changes as you move through the service design and delivery process.

The needs and goals of the users define the service. You will uncover these needs and goals through your research in the Discovery stage and build on them through the other stages.

In Alpha and Beta stages you'll be testing prototypes, and in Live stage you will be researching to find out how you can improve the working service.

You will go from doing more generative research (making sure you are designing the right thing) in Discovery to more evaluative research (making sure you are designing it the right way) in the later stages.

In all stages:

do research with a broad range of users, including those with access needs and low digital skills
think about the whole user experience (including offline), not just the product you are delivering

[toc]

Meeting the Digital Service Standard

You must do user research when designing and building your service to meet the following criteria:

The Digital Service Standard guides teams to build services that are simpler, clearer and faster.

User research in Discovery stage

Before you start to design anything, find out about your users. This will help you to understand how the service you’re designing can meet their needs.

Observe users

Watch how users do things now and what problems or barriers they face. This is called contextual qualitative research. Don't use market research for this: find the users' real problems by observation.

Interview users

Interview and observe frontline service staff (if your agency has them).

This will help you to understand:

limitations of the current service
how staff explain some of the complexities
undocumented workarounds that people use to get things done

Map the user journey

Create a user journey map of the experience of the people who want to do the task that brings them to your service.

Do research to understand the real user journey. It's more important than the journey the business would like to think users have.

Examine existing data

Look at existing data. This might include analytics, back-office workflow and support logs.

This can reveal new issues and quantitative evidence that supports qualitative insights.

Review previous user research

To avoid bias, review previous user research after your own primary research.

No matter how much research has been done before, the team needs to do its own research. This will help team members build a deeper understanding as well as empathy for the user and their needs.

What you'll find out in Discovery

From these activities you should get:

an understanding of the problem your service is trying to solve for the user
a list of users’ touchpoints as they try to achieve their goal — often expressed as a journey map, service map, mental model or alignment diagram
sets of needs and task models for different types of users
insights about users that you’ve gained from analysing the research
a list of research gaps or limitations of the current research and opportunities for further research
descriptions of user groups and the characteristics that are significant for this service

You’ll have done enough research when you understand:

the different kinds of people who use your service
what users needs from your service
how people with support and access needs interact with your service

Combine profiles with real stories

Descriptions of users are often expressed as user profiles or personas.

We prefer to use profiles combined with actual stories from users the team has met. This tends to be more effective than developing personas. Empathy for an actual user is likely to be more authentic than empathy for a persona.

Keep on discovering

Discovery research doesn't just happen in the Discovery phase. Ongoing research helps teams to understand the evolving needs of their service.

Empathy wall

This represents all of the empathy we built through Discovery with the real users. It’s a great way to introduce new people to the project. We used red post-its to flag pain points in user journeys and blue for where our research validated existing research.
— Service designer

A wall with a lot of pictures, post-it notes and drawings — Caption: A wall with user profiles and user research findings. It helps you build a picture of the users.

User research in Alpha stage

In the Alpha stage, try lots of different approaches to meet user needs and find out which approach works best.

This is an important stage and you should make sure you have budget for it. The Alpha stage will help you reduce the risks around:

design — making sure you get the right scope for the service
business processes — finding out if government can build the service
technical capability — being sure you can integrate the service and make it secure

Work out what works, not what's popular

Be prepared to park any ideas that do not meet user needs. You may also find that different elements of an idea works, sotake what is working and combine it with other ideas. Remember to document the different ideas you have tested to refer to later. This helps the team remember what has been tested and what has not.

Do task-based testing so that you can understand which version is most effective. We care about what works, not what people tell us they prefer.

You will know that something works best because people will find it easy to complete the tasks. They will understand what they are doing and won't make mistakes.

Keep iterating and testing

Once you have a prototype (even if it’s a paper one) you can start to test it with users. You can see whether your ideas will meet user needs and find out if the prototype is usable. You can then use this insight to iterate your design and test again with users.

Keep on interviewing users

You can combine testing your prototypes with user research interviews. This will help you to continue to deepen your understanding of user needs.

Park any ideas and prototypes if they don't meet user needs.

User research in Beta stage

There are two different research stages in Beta:

researching as you build the beta service
researching with users of a working beta service

1. Researching as you build the beta service

You can do a similar style of research as you were doing in Alpha as you build the beta service. But you are aiming to refine a solution for launch.

Do task-based usability testing with a wide range of users.

You should decide on hypotheses in your research. These are ideas about how you think the design will meet a user need.

Use a structured approach to evaluation. This will help you to be clear about what is working well and where there is more work to be done.

Make sure you do accessibility testing with people who have access needs. Commission an accessibility review before you put the service into a working beta.

2. Researching with users of a working beta service

Keep doing what you were doing before but add:

analytics (including reporting KPIs to dashboard)
interviews / shadowing of real users
live intercepts
A/B testing (comparing 2 versions of a web page to see which performs better) or multivariate testing — think about whether the sample size will be useful to test your beta service
feedback from frontline and operational staff
face-to-face and remote usability tests to find usability and accessibility issues
an accessibility audit to uncover accessibility issues and get recommended fixes
private or public beta tests of the end-to-end service with real users, including support options
web analytics and back-office data to measure service performance
surveys or follow-up interviews to collect detailed feedback from service users

What you'll find out researching in Beta

From these activities you’ll typically learn:

more about how different kinds of users experience your services
the usability and accessibility issues you need to fix
ways to improve your service

User research in Live stage

When your service reaches the Live stage, research helps you to make sure it is still meeting user needs. It also helps you make decisions on improving the service.

You can learn more about your users’ needs through:

reviewing web analytics and back-office data to measure service performance
surveys or follow-up interviews to collect detailed feedback on your service
interviews and visits to get a deeper understanding of any problems users tell you about
face-to-face and remote usability tests to find issues with new or changed features
A/B testing on new and changed features

What you'll find out researching in Live

These research activities will usually give you:

deeper understanding of how different kinds of users experience your service
insights into usability and accessibility issues and how to fix them
ideas for ways to improve your service in future

How often to research in Live

The frequency and depth of your research once you reach Live stage depends on what you are trying to find out. For example, if you were looking to add a new component to your service, you would start Discovery again.

If you are going to decommission your service, find out how people use it first. This will help you to understand the gap that will be created when the service is gone.