The Department of Health’s, Health Products Regulation Group, includes the Therapeutic Goods Administration (TGA), which is responsible for safeguarding the health of all Australians through effective, timely and risk proportionate regulation of therapeutic goods, such as medicines. Regulation includes pre-market approval and post market monitoring of therapeutic goods that sponsors (usually pharmaceutical companies) want to sell in Australia.

This project focuses on post-market monitoring for medicines.

The objective of the service—reporting adverse events—allows the TGA to identify issues with medicines that that have been approved for use in Australia. Issues with a medicine may include:

• it has unintended side effects (safety issues)
• it is not efficacious

This will be achieved by:

• having multiple channels which are simple, intuitive, easy to use to enable easy reporting of adverse events
• enabling timely and efficient analysis of rich data sets so the TGA can effectively act in the interest of public safety

The focus of Alpha was to improve the reporting mechanisms for sponsors who are required to report certain types of adverse events, and to improve the analytics capability of the TGA.

The analytics capability will include better integration and analysis of available datasets to identify safety issues. The reporting mechanism will include:

• enhancing the TGA reporting web page to make it simpler and easier to find
• delivering an improved portal experience to Sponsors, Health Professionals and Consumers, including the ability to register and access previous reports
• enhancing the online reporting forms to make it easier to report, and align better with international reporting standards
• introduce integrated reporting channels, including ability for sponsors to report via their own systems, and harmonisation with international standards

Each year the TGA receives more than 18,500 reports of suspected adverse events to medicines and vaccines. Most adverse event reports are made by sponsors (such as pharmaceutical companies) who are required to report serious adverse events. Many reports are also made by state and territory health departments, health professionals and consumers.

Criterion 1: Understand user needs

The service team demonstrated a good awareness about their users’ needs. They classified them into significant categories and have applied approaches to user research that matched the category of user well. 

The sponsors as the major user source of data category (60+ %) were well researched, and the delivery team is demonstrating an excellent understanding of their needs. The approach to learning about their needs stemmed from leveraging off previous research plus other major surveys and touch points. The team has applied many of those learnings and augmented them with more direct involvement in subsequent prototyping sessions to validate and update the earlier learnings. The approach employed used both facilitated and unfacilitated collection techniques. The second major data source group, the health professionals, has also been well researched using the same techniques and touch points as outlined for sponsors. 

The user research has been centered mainly on the higher volume data sources of sponsors and health professionals. Lower volume data sources such as state health departments (via vaccines reports) and direct from consumers have also been collected. This approach is supported by the fact that state health departments are collecting adverse event information directly associated with vaccines only, and consumers generally will report events through their prescribing doctor. 

One user group, being the World Health Organisation (WHO), did not seem to be specifically targeted in user research and the existing reporting mechanisms assumed as maintaining the status quo. Given they are a significant recipient of content from the service, perhaps some further research into any possible improvements in outputs to the WHO could improve the service in the longer term.    

The delivery team has performed well under this criterion but could benefit from additional user research targeted at outputs to the WHO or data inputs from consumers directly as the project moves into Beta. 

Criterion 2: Have a multidisciplinary team

The team is working very well together and has the appropriate mix of digital professionals and subject matter expert (SME) end users. The scrum team is composed of information technology positions augmented with subject matter experts and user-centric designers.  A separate scrum team had been allocated for data analysis, with appropriate ties into the main scrum team, but recently has been merged into one scrum team. A number of project staff have been on DTA assessor training which has helped to embed the principles of the Digital Service Standard (DSS) into other team members not trained in DSS delivery. Team change overall has been handled well with the key roles of scrum master and change manager being absorbed into other team member’s functions.

The product owner is fully engaged and across the project deliverables and approach, with and decisions being made at the correct level resulting in the decision making processes being quick and efficient.

The team is a highly motivated, multi-disciplinary and performing very well in adhering to the DSS such as the encouraged principles, agile rituals and agile practices, but is encouraged to document this in a team charter.

Criterion 3: Agile and user-centred process

The team has good overall sprint process and has excellent business involvement throughout the process. The product owner articulated the approach as “water scrum fall” thus demonstrating a complete understanding of the objectives in use on the project that align with agile principles and where they deviate based on organisational constraints. Business staff have direct access and use the electronic kanban to update product backlog items (PBIs) as they travel through the backlog. 

The project has demonstrated the use of backlog refinement through to sprint reviews, retrospectives and showcases. The results of retrospectives are published on the team website and proactive changes made from the feedback received. 

The minimum viable product (MVP) is clearly documented and articulated as part of the features schedule tying directly to sprint schedules and delivery. A wall has been dedicated to design and user empathy to further demonstrate full use of user-centric design-based approaches.  

Hypothesis statements and validating approaches have been captured as part of the confirmation process to ensure a user-centric design can be delivered upon in the Beta stage.

The team is efficiently and effectively using agile (Scrum) processes, embedding the business staff in the process and delivering well under the Service Design and Delivery Process. 

Criterion 4: Understand tools and systems

The team is re-using and leveraging a technology stack used successfully in other areas of health. The procurement costs and ongoing support costs in infrastructure and technical support will be minimised. It also aligned extremely well with future frameworks and overall technology directions in Health. The team is planning on leveraging the whole of government software purchasing power via the DTA to reduce costs involved in procurement on one element of the technology stack. The addition of this software product has demonstrated the project’s willingness to adapt and adopt the user needs once the requirement was identified during Alpha testing. 

Criterion 5: Make it secure

The team has completed a privacy impact statement and is implementing practices consistent with Information Security Manual. Legal advice has been sought to ensure all correct legal instruments are in place to support caching of private personal data and aligned with privacy disclaimer text on the site. The team is also working with the department’s IT Security Advisor to obtain an accreditation and certification for the system.

Criterion 6: Consistent and responsive design

The selected technology platform has a web responsive design that will cater for most types of devices and user digital interactions. 

Criterion 7: Use open standards and common platforms

The service is re-using and leveraging a technology stack used successfully in other areas at the Department of Health. Open standards for HTML, CSS, and javascript have been employed and the existing standards for the TGA authentication mechanisms have been employed. The team has used the agreed data information sharing standards for this type of transaction (ICH E2B R2/R3, AS2, SOAP, REST).

Criterion 8: Make source code open

The service is considering API solutions for the next phase and has demonstrated the use of open source repository such as GitHub. The public facing portal is based on an open source Community Edition of the ADX Studio portals by Microsoft.

Criterion 9: Make it accessible

The service will follow the standard Department of Health governance process for ensuring adherence to WCAG 2.0 in both development and design protocols. The service will also take advantage of the assisted digital service available to all TGA users.

Criterion 10: Test the service

The team has a comprehensive multi-layered test strategy in place. It covers usability, accessibility, security, developer (unit), system and user acceptance testing. Performance testing, is not available in an automated fashion at the Department of Health. Where possible, the project has planned to test the service manually with high numbers of concurrent users.

Criterion 11: Measure performance

The team has a number of performance measurement targets in progress. 

Criterion 12: Don’t forget the non-digital experience

The existing service channels (telephone, email, fax), will remain available. 

Criterion 13: Encourage everyone to use the digital service

The TGA has robust ongoing communications in place to encourage the use of digital service channels. The team is currently encouraging the adoption of business to business electronic transactions, especially with sponsors, via the new web service. 

Recommendations

The assessor panel congratulates the service team on very good progress on, and adherence to the digital service standard. 

Ongoing user research targeted at outputs to the World Health Organisation (WHO) or data inputs from consumers directly could inform improvements to the service. 

The team is a highly motivated, multi-disciplinary and performing very well in adhering to the DSS such as the encouraged principles, agile rituals and agile practices, but is encouraged to document this in a team charter.