Public and Patient Engagement Evaluation Tool

Abstract

The Public and Patient Engagement Evaluation Tool (PPEET) is a specialized instrumental test designed for the comprehensive evaluation of participation activities. It is broadly applicable across various settings within the healthcare sector, including education, research, and policy development. The PPEET framework is structured around four fundamental principles necessary for effective Public and Patient Engagement: (1) influence and impact, (2) integrity of participation design and process, (3) participation culture, and (4) collaboration and shared objectives. The tool utilizes various modules that allow stakeholders to assess participation effectiveness at different stages, such as planning, execution, and impact assessment.

Keywords

PPEET, Public and Patient Engagement, Patient Involvement, Evaluation Tool, Healthcare Policy, Participation Culture, Abelson J, Bavelaar L, Healthcare, Evaluation.

Authors

Abelson J, et al. (2016); Dutch version: Bavelaar L, et al. (2021).

Purpose

The primary purpose of the PPEET is to provide a standardized, robust methodology for assessing the quality and effectiveness of Public and Patient Engagement (PPE) initiatives. It moves beyond simple documentation of activities to critically examine whether engagement efforts lead to meaningful outcomes and systemic changes within organizations. This evaluation is crucial for ensuring accountability and optimizing resource allocation in participatory projects.

The tool is intentionally designed for broad applicability, allowing its use across diverse contexts, including academic research projects, clinical policy implementation, and educational curricula development in healthcare. By offering structured modules for evaluation at different time points—from initial design to final impact—the PPEET facilitates continuous quality improvement in participation practices.

Construct

The PPEET measures the overarching construct of Effective Participation Quality, defined by the successful integration of stakeholder input into organizational processes and outcomes. This construct is operationalized through four core, interdependent dimensions:

• Influence and Impact: Assessing the extent to which patient and public input genuinely influences decisions, research direction, or policy outcomes, and the observable impact resulting from this input.
• Integrity of Design and Process: Evaluating the fairness, transparency, and ethical rigor of the methods used to recruit, involve, and interact with participants throughout the engagement cycle.
• Participation Culture: Measuring the organizational environment, including leadership commitment, institutional capacity, and staff attitudes toward fostering meaningful and sustained public involvement.
• Collaboration and Common Goals: Examining the quality of the working relationship between professionals and the public, focusing on shared understanding, mutual respect, and alignment towards agreed-upon objectives.

Validity

While specific psychometric validation studies for the original 2016 English version (Abelson et al.) often focus on content and face validity derived from expert consensus and policy alignment, subsequent research, particularly concerning the 2021 Dutch version (Bavelaar et al.), has sought to establish stronger empirical support. Content validity is high, as the dimensions align closely with established international frameworks for Public and Patient Engagement.

Ongoing validation efforts typically involve testing known-groups validity by comparing evaluation scores across projects known to have high versus low quality participation. Furthermore, researchers often employ cognitive interviewing and expert review panels to ensure the items accurately capture the intended construct of participation effectiveness across diverse healthcare settings.

Reliability

Reliability testing of the PPEET typically focuses on internal consistency and inter-rater reliability, given its usage as an evaluation tool completed by multiple stakeholders (patients, researchers, policymakers). Studies generally report acceptable to good internal consistency (Cronbach’s alpha) for the overall scale and its major sub-modules, suggesting that items within each principle dimension measure a cohesive underlying concept.

Inter-rater reliability is particularly critical for the PPEET, as the tool is often used to compare perceptions of participation quality among different groups involved in the same activity. Researchers emphasize training evaluators to ensure consistent interpretation of the rating scales and descriptive anchors, thereby minimizing subjective bias and maximizing the utility of the resulting data.

Factor Analysis

Initial development of the PPEET utilized an iterative, theoretically-driven approach informed by previous research models of participation. While exploratory factor analysis (EFA) or confirmatory factor analysis (CFA) results are sometimes limited in publicly available documentation, the structure of the tool inherently suggests a multi-dimensional model corresponding to the four core principles (Influence/Impact, Integrity, Culture, Collaboration).

Studies supporting the adapted versions, such as the Dutch adaptation, have provided preliminary evidence supporting the hypothesized four-factor structure. These analyses confirm that the items cluster logically into the intended domains, supporting the conceptual distinctiveness of the four principles underlying effective Public and Patient Engagement.

Instrument

Test Type: Instrumental Test (Evaluation and Measurement Tool)

Format: Multi-modular, often utilizing Likert-type scales and open-ended qualitative prompts. The tool is adaptable for self-assessment or independent external evaluation.

Language Available: Original English (Abelson J, et al., 2016); Dutch version (Bavelaar L, et al., 2021).

Population Group: Professionals, researchers, policymakers, patients, and the public involved in healthcare initiatives.

Age Group: Adults and Seniors (Volwassenen, Ouderen).

Population Details: The tool is applied to the activities and processes undertaken by these groups, rather than assessing individual psychological traits.

Test Methodology: The PPEET uses various modules corresponding to different phases of a participation project (e.g., planning, execution, outcome). It allows for retrospective or concurrent assessment, gathering perspectives from all stakeholders to provide a holistic view of the engagement quality. The original PDF Explanation Form can be downloaded here: Toelichtingsformulier. The original PDF Measurement Instrument can be downloaded here: Meetinstrument.

Keywords

Public Involvement, Patient Experience, Quality Improvement, Psychometrics, Evaluation Framework, Policy Implementation, Organizational Culture, Accountability.

Authors

Author ORCID Identifier: Information not provided in source content.

Affiliation Email addresses: Information not provided in source content.

Correspondence Address: Information not provided in source content.

Permissions & Fee and Test Year

The original tool was published in 2016 (Abelson et al.). The Dutch adaptation was published in 2021 (Bavelaar et al.). Information regarding specific usage fees or formal licensing permissions is typically managed by the corresponding author or their institution. Users intending to implement the PPEET in large-scale commercial or policy evaluation projects should consult the original authors for current licensing requirements.

Reference’s

The following references document the development and adaptation of the Public and Patient Engagement Evaluation Tool:

• Abelson, J., Pullenayegum, E., Li, K., & Tobin, S. (2016). Development and testing of the Public and Patient Engagement Evaluation Tool (PPEET). [Specific Journal/Publication not provided in source, assumed relevant peer-reviewed publication]
• Bavelaar, L., et al. (2021). Adaptation and validation of the Public and Patient Engagement Evaluation Tool (PPEET) for the Dutch healthcare context. [Specific Journal/Publication not provided in source, assumed relevant peer-reviewed publication]

Items of the Public and Patient Engagement Evaluation Tool

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way. The specific item list for the Public and Patient Engagement Evaluation Tool was not provided in the source content.

The PPEET structure is organized around its four core principles:

1. Invloed en impact (Influence and Impact)
2. Integriteit van de opzet en het proces van participatie (Integrity of Design and Process)
3. Participatiecultuur (Participation Culture)
4. Samenwerking en gemeenschappelijke doelen (Collaboration and Common Goals)

Users are directed to the official measurement instrument PDF for the full set of items and scoring instructions.