About
PHES-EF (Public Health Environmental Surveillance Evaluation Framework) is an international consensus study to develop a framework for evaluating public health environmental surveillance systems, using wastewater-based surveillance as an example.
Why PHES-EF is needed
Public health environmental surveillance, which monitors health threats using samples from the environment, expanded rapidly during the coronavirus disease (COVID-19) pandemic. Wastewater-based surveillance is the most prominent example. As these systems diversify and become part of routine public health practice, there is still no agreed way to evaluate them, and standard methods for analysing and reporting their data remain unclear (1).
A shared evaluation framework will help public health agencies, decision-makers and researchers to:
- Judge whether a surveillance system is effective and provides a return on investment
- Strengthen accountability, transparency and public trust
- Plan for sustainability
- Compare system performance across settings using consistent methods
What the framework will look like
Evaluation framework definition:
An evaluation framework is a structured system designed to systematically assess the effectiveness and impact of a program (2,3). It typically outlines key components (such as domains, concepts, and indicators or measures) that should be included in an evaluation. It also provides a logical organization of these components, explains the process by which they were identified and prioritized, and may offer guidance on how to measure them. By offering a comprehensive and transparent basis for evaluation, such frameworks support informed decision-making and help identify areas for improvement in public health programs.
PHES-EF organizes these components into four levels, from broad to specific:
Theme
A high-level grouping of related concepts that define broad areas of public health evaluation.
Example: Surveillance effectiveness
Domain
A major area of focus within a theme.
Example: Ascertainment performance
Subdomain
A specific aspect or perspective within a domain.
None needed for this example
Attribute
A measurable property or characteristic.
Example: Sensitivity
How PHES-EF is being developed
PHES-EF is being developed through an international e-Delphi consensus study. Candidate items come from two scoping reviews of how surveillance systems are evaluated. A panel of content experts, knowledge users and engaged members of the public will rate these items over two anonymous online survey rounds, followed by a consensus meeting. See Study design for the five study steps and timeline, and the Delphi study page for how the survey works.
Our principles
Open science
All data and materials shared under an open licence.
Equity and inclusion
Guided by EDI and OCAP principles.
Public involvement
Knowledge user and public involvement reported with GRIPP2.
Open science
This project is committed to an open science approach, ensuring transparency and accessibility at every stage. All project data and materials will be made available, including the search strategy, search results, initial curation of evaluation measures, executive group discussion summaries, e-Delphi process documentation, and consensus procedures.
PHES-EF will be made available under an open licence (CC BY-SA 4.0), enabling reuse, adaptation, and further development. A comprehensive research data management plan will outline the data and materials shared, along with guidance on how to access them, to support reproducibility and facilitate collaboration.
For licensing details, please refer to the Licence section of this page.
Equity, diversity, and inclusion (EDI)
“Equity is defined as the removal of systemic barriers and biases enabling all individuals to have equal opportunity to access and benefit from the program” (4). To achieve this, members of the research team are committed to developing a strong understanding of the systemic barriers faced by individuals from underrepresented groups.
“Diversity is defined as differences in race, colour, place of origin, religion, immigrant and newcomer status, ethnic origin, ability, sex, sexual orientation, gender identity, gender expression and age” (4). Panellists will be asked to provide characteristic information (e.g., country of residence, gender, primary discipline) to assess and help ensure (through more targeted recruitment) a large diversity in perspectives.
“Inclusion is defined as the practice of ensuring that all individuals are valued and respected for their contributions and are equally supported” (4). To help ensure that all research team members are integrated and supported, we will have a code of conduct statement in the executive group Terms of Reference.
All study working group members will complete EDI-related Canadian Institutes of Health Research (CIHR) Training, and the First Nations principles of Ownership, Control, Access and Possession (OCAP©) training for the research (5). The executive group will receive an overview of EDI and OCAP and will be encouraged and supported to complete such training. Furthermore, staff trained in EDI and OCAP principles will review the framework from an EDI perspective to ensure appropriate language.
Knowledge user and public involvement
To ensure accurate and transparent reporting of knowledge users and public involvement throughout the study, we will refer to the Guidance for Reporting Involvement of Patients and the Public (GRIPP2) checklist (6). We will document the methods used to engage knowledge users, report the impacts and outcomes of their engagement, and report on lessons learned from the experience.
Declarations
Funding
This project is funded by the Canadian Institutes of Health Research-funded network, CoVaRR-Net (Coronavirus Variants Rapid Response Network. FRN: 175622), and Health Canada (through the Safe Restart Agreement Contribution Program. Arrangement #: 2223-HQ-000098). The Canadian Institutes of Health Research and Health Canada have not been involved in the design or conduct of the study and the views expressed herein do not necessarily represent the views of either funding organization.
Ethics approval and consent to participate
This study has received ethical approval (20230428-01H) from the Ottawa Health Science Network Research Ethics Board (OHSN-REB). Implied consent will be obtained from all panellists prior to participating in the e-Delphi survey. Information about the survey and consent practices will be provided on the first page of the survey. Panellists will be informed that by providing their demographic information and proceeding to the next page, they are providing their implied consent to participate in the study. Only anonymous panellist responses will be used for analyses. All panellist data and responses will be password protected and only REB-approved study working group members from the Ottawa Hospital will have access to the data.
If you have questions about ethical issues related to this study, you can talk to someone who is not involved in the study at the Ottawa Health Science Network Research Ethics Board, Chairperson at +1-613-798-5555 ext. 16719.
Competing interests
All participants (i.e., study working group, executive group, and e-Delphi panellists) will be asked to declare any financial or personal relationships that may influence either the conduct or presentation of this research. Conflict-of-interest disclosure forms for executive group members are available on the Team page.
Licence
This website hosts a range of content, including references to journal articles, research papers, and other educational materials.
The majority of the content on this website is licensed under two Creative Commons licences: CC-BY-4.0 (Creative Commons Attribution 4.0 International) and CC-BY-SA-4.0 (Creative Commons Attribution-ShareAlike 4.0 International).
Please see the licence page for more information.
Privacy
Anonymous aggregate data will be made available on our Open Science Framework (OSF) project page upon study completion. Individual responses to e-Delphi rounds will be anonymous at the source level by the platform used.