Discovery — understand the problem

Discovery allows you to get a deep understanding of the problems users are trying to solve. It helps the team challenge their existing ideas of what the problem and solution might be.

[toc]

Understand user needs

It is important to research your users and understand why they are using the service. Using this research, create a measurement plan so you can understand and improve how you are meeting your user’s needs.

Why it's in the guide

Analytics works best when it has direction. Understanding how people use your service lets you develop clear goals to meet their needs.

How you can meet this

Part of researching your user’s needs sometimes means you need to understand areas that are not ‘in scope’. Contact other teams, such as policy or communications teams, who provide content for online services. These teams may have different kinds of information about your users that will provide new insights and measurement methods. It will also connect your quantitative research with qualitative methods. You will also expand your product’s stakeholders, which will help you communicate clearly with data

Consider completing a Measurement Plan (.XLSX, 36KB), starting with core objectives for your website or service.This is an example and template supplied by the DTA and their GA360 Partner Data Runs Deep.

Define success

Define what success looks like and understand how you might measure it. Think beyond pageviews and sessions and use your user research to let people’s needs guide your service’s definition of success. 

Why it's in the guide

For a service to improve, product managers need to accurately identify how users are currently engaging with a service. Defining success requires you to create a baseline you can build on. From this baseline, teams can decide where and how to improve their service.

Defining success for your products and services

Defining what success means for your products and services is not always about how many people use your service. These numbers (often referred to as ‘vanity metrics’) may look impressive, but don’t help you learn about your users’ behaviour. 

For example, the purpose of your service might be to communicate an important message to the public. You could define success as how many people used the service and saw the message, but it you shouldn’t stop there. Is success how the message spread over a period of time. Did more or less people use the site over time? Did they share the message? Did they find the message the way you expected? You can also define success as how much of the message people absorbed. Did they read all the way through or lose interest? This can be a powerful indicator of whether the number of people using the service actually received the message.

Describe success in the form of questions. This keeps you and your team focused on the experience of people using the service, but also makes reporting, dashboards, and communicating the results much easier. For example, 'how many people in each state were able to complete an online form?' is a more human-centred way of saying 'form completions by state.' Start with a small set of questions you can answer with no more than 6 to 10 metrics. This will help you measure what matters

Describing success as questions also opens up the conversation. Using the example above, we can ask:

• How many people couldn’t complete the form?
• Is that number more or less than expected?
• What might be the reasons form completion is different from one state to another?

These questions can show you where you can improve your service, or where it isn’t working.

Set a baseline

Before we can identify success, we need to understand what the current state looks like now.

Why it's in the guide

If you are developing a new feature for an existing service, it is important to quantify how users navigate and engage with your service so you can identify if your changes have been successful. If you are developing a new service identify how your users are currently meeting their needs through other channels.

This baseline will be the fixed reference point for the start of your product’s development or improvement, allowing you to measure your changes and identify its value.  If your project frequently ‘re-baselines’ without a formal process, it will be difficult to know if you are meeting your original hypothesis. Changing baseline may create uncertainty about your project’s likelihood of success.

How you can meet this

Start by recording the initial metrics identified when you are defining success and visualising in your business intelligence tool. You should also consider using webmaster tools and search consoles to identify who are your main referrers and where their traffic predominantly lands. These tools also provides key search terms used by your users, which can help you understand and analyse the intent of people using your service.