[toc]

The analytics for developing a service from scratch will be different to the analytics used for creating a new feature for an existing product.

Discovery

Discovery allows you to get a deep understanding of the problems users are trying to solve. It helps the team challenge their existing ideas of what the problem and solution might be.

Understand user needs

Research to develop a deep knowledge of the users and their context for using the service. Build a measurement plan around the needs that you're addressing.

Define success

Define what success looks like and understand how you might measure it. Think beyond pageviews and sessions to what will help you learn about how people use your service. 

Set a baseline

Before we can identify success, and before we can truly understand and measure improvement, we need to understand what the current state looks like now. A longitudinal set of metrics establishes what normal looks like, and gives context for measured improvement. 

Alpha

At the start of the Alpha stage, you should have a good understanding of the user journey and know what the major stages of the user journey are.

Design with measurement in mind

Design and development decisions can make effective measurement difficult or impossible. Consider performance measurement early during design.

Measure what matters

Learn how to measure what really matters. Set key performance indicators (KPIs) and track them over time to know if your service is meeting the needs of your users and improving.

Segment key audiences

Different groups of users may have different needs. Break your audience into ‘segments’ to understand how your service is performing for each group.

Private Beta

At the start of Beta stage, you take the prototype you agreed on at the end of Alpha stage and build a minimum viable product. You can then release this as a private Beta. Use the private Beta stage to make sure your measurements are working before scaling up your data acquisition and analysis for the public Beta stage.

Keep data consistent

Clean and consistent data make it simple for you to compare performance across services. 

Communicate clearly with data 

Understand who you need to communicate with and what is important for them to know. Use the right communication tools to reach different audiences. 

Public Beta

Keep data anonymous

Privacy is important. Anonymous data is all you need to understand how people use your service. Understand how to avoid capturing personally identifiable information

Live

After you move to the Live stage you should keep improving your service based on user feedback, analytics and further user research.

Work in the open

Share your learnings and insights between and across teams. Publish your key performance indicators on dashboard.gov.au

Success is finding ways to improve

You will always find areas of your service to improve. This is what success looks like for performance measurement. If you can not see any areas to improve, ask why. Evaluate performance regularly and ask how you might improve. People and systems change over time. The analytic methods you established at the start of the project may no longer accurately represent what is happening on your service and with your users.