e-Delphi Study

A Delphi technique will be used to develop an internationally accepted evaluation framework for public health environmental surveillance. The Delphi method is an iterative multi-round approach that uses a series of sequential surveys, interspersed by controlled feedback, to elicit consensus among a group of individuals while maintaining anonymity (1,2). An electronic Delphi (e-Delphi) method will be used to overcome geographic barriers and allow us to engage panellists internationally across various time zones.

The full Delphi study protocol is prespecified and available {add link}. The protocol has been submitted for peer-review.

Recruitment of Panelists

We will recruit a multinational, multidisciplinary panel of wastewater-based surveillance (wastewater surveillance) experts, knowledge users, and engaged public to complete the e-Delphi survey. Panellists will be selected to capture the multiple perspectives of the design, implementation, evaluation, reporting, and use, of wastewater surveillance.

We will aim to recruit at least 50 panellists, preferably, with at least 8-10 people per discipline group. The study working group will monitor the distribution of registered panellists based on their demographic information and will try to have appropriate distribution across disciplines and other demographic markers.

Eligibility Criteria for e-Delphi Survey Panellists

Discipline subgroup	Inclusion criteria
Public health, infectious disease, epidemiology Environmental and physical science Mathematical sciences Social sciences Communication, knowledge translation and Exchange	An adult¹ who is proficient in English and has a graduate degree in one of the listed specializations², or ≥ 3 years of professional experience, or ≥ 2 peer-reviewed publications relating to wastewater surveillance.
Knowledge users	An adult¹ who is proficient in English and who is a professional who does not have specialized training or qualifications in wastewater-based surveillance, but who uses surveillance to inform policy and action in their workplace.
Engaged public	An adult¹ who is proficient in English and has relevant lived experience.
¹ See the full protocol, Appendix 2 for the specializations associated with each discipline.
² Adult: ≥ 18 years of age.

Definitions

Professional experience: Paid employment or professional practice in a listed specialization (current or former). Graduate or professional degree: Master or PhD in a listed discipline/specialization.

Procedure

We will conduct a two-round e-Delphi survey to generate consensus on evaluation criteria (Figure 2). Summaries of Round 1 will be compiled for the subsequent round. Custom survey pathways will be generated for each stakeholder group (i.e., panellists from different stakeholder groups will be shown a different collection of candidate items). Within each custom stakeholder survey pathway, panellists will also have the option to skip items or self-declare that they are not qualified to assess certain candidate items.

Round 1

Panellists will be invited to rate their level of agreement with candidate items generated from scoping review results and consultation with the study executive group.

Round 2

Panellists will rate their level of agreement with newly identified candidate items from the first e-Delphi round and to re-rate their level of agreement for items that did not reach consensus during the previous round. Anonymous feedback from Round 1 panellists will also be compiled and presented during Round 2. Any newly suggested items during Round 2 will be deliberated on during the consensus meeting.

Defining Consensus

Consensus for each item is defined as ≥ 70% of the panelist votes falling within the same grouping of three categories within a 7-point Likert scale:

Core Item: Highly or moderately relevant
Further Discussion: Slightly relevant, neutral, slightly irrelevant
Excluded Item: Moderately or highly irrelevant

The survey will include a free-text box for panellist to provide comments or suggest new items. Newly suggested items, as well as non-consensus items, will be listed in the second round.

Figure 2. E-Delphi onsensus process for the Public Health Environmental Surveillance Evaluation Framework (PHES-EF)

References

Shang Z. Use of Delphi in health sciences research: A narrative review. Medicine [Internet]. 2023 Feb 17;102(7). Available from: http://dx.doi.org/10.1097/MD.0000000000032829

Niederberger M, Spranger JS. Delphi Technique in Health Sciences: A Map. Frontiers in Public Health [Internet]. 2020 Sep 22;8. Available from: http://dx.doi.org/10.3389/fpubh.2020.00457