Coaching Fidelity Scale (CFS)

Abstract

The Coaching Fidelity Scale (CFS), developed by Meadan et al. (2023), functions as a specialized observational instrument designed to systematically assess the degree of coaching fidelity achieved by professionals. This instrument is structured as an 11-item checklist based on a robust coaching model previously established in research. The model is segmented into three critical phases: the pre-observation phase (strategy discussion, context identification, and skill review), the observation phase (uninterrupted interaction between the coach and coachee), and the post-observation phase (coachee reflection, feedback, and joint problem-solving).

The psychometric properties of the CFS were evaluated using data collected from three distinct intervention studies. Initial development involved incorporating two additional problem-solving items into the original 9-item scale based on field feedback. Subsequent statistical analysis, including Exploratory Factor Analysis (EFA), confirmed a strong unidimensional solution, indicating that all items measure a single underlying construct. The study provided comprehensive results demonstrating high internal consistency, excellent interrater reliability, and significant test sensitivity to intervention-related changes.

Keywords

Autism, Caregivers, Coaching Practices with Caregivers, Coaching Professionals, Developmental Disabilities, Evidence-Based Practices, Observational Measure, Young Children

Authors

Meadan, Hedda, Lee, James D., Sands, Michelle M., Chung, Moon Y., García-Grau, Pau

Purpose

The primary objective of the Coaching Fidelity Scale (CFS) is to provide a standardized, observational method for accurately quantifying the degree of coaching fidelity demonstrated by professionals. This tool ensures that coaching interventions, particularly those focused on supporting caregivers and educators of young children with special needs, are implemented as intended by the established model.

By measuring fidelity, the CFS supports researchers and practitioners in maintaining the integrity of evidence-based practices. High fidelity scores indicate adherence to the prescribed coaching model across all three phases—pre-observation, observation, and post-observation—thereby strengthening the link between intervention delivery and desired outcomes.

Construct

The CFS measures the construct of Coaching Fidelity, defined as the extent to which a coach adheres to the critical components and procedures of a specific, structured coaching model. This construct is viewed as unidimensional, meaning that the successful execution of all 11 items contributes to a single, global score representing the quality of the coaching interaction.

The model underlying the CFS is process-oriented, emphasizing the successful completion of specific tasks across the interaction phases. Key components of the construct include effective preparation (pre-observation), non-disruptive monitoring (observation), and reflective discussion, constructive feedback, and joint problem-solving (post-observation).

Validity

The validity of the CFS was primarily supported through the assessment of Test Sensitivity, demonstrating the scale’s ability to accurately detect change following an intervention. Independent sample comparisons were conducted between baseline (pre-intervention) observations (n = 50) and post-intervention observations (n = 81).

Results from the Mann–Whitney test showed a statistically significant difference between the groups (p < .001). Pre-intervention scores (M = 1, SD = 1.20) were significantly lower than post-intervention scores (M = 5.91, SD = 2.52). The large effect size reported (rank-biserial correlation = .84) confirmed the scale’s strong sensitivity to improvements in coaching practices achieved through intervention.

Reliability

The reliability of the Coaching Fidelity Scale was evaluated using two primary metrics: internal consistency and interrater reliability.

• Internal Consistency: This was assessed using Cronbach’s alpha. The calculated alpha value was 0.91 (95% CI [0.89, 0.93]), which signifies an excellent level of consistency among the scale items. The average inter-item correlation was 0.47 (95% CI [0.41, 0.53]), further supporting the coherence of the items in measuring the same construct.

• Interrater Reliability: The agreement between independent observers was measured using the Interclass Correlation Coefficient (ICC). The ICC value for the overall CFS score exceeded 0.96, which is interpreted as demonstrating superb or excellent agreement between raters, ensuring the measure is consistent regardless of the observer.

Factor Analysis

An Exploratory Factor Analysis (EFA) was performed using the 9 core items of the CFS. The results strongly supported a unidimensional solution, confirming that the scale measures a single underlying construct of coaching fidelity.

The single identified factor accounted for 77.84% of the total variance, indicating high explanatory power. All tested items demonstrated strong loadings (above 0.76), and communalities were robust (above 0.609). The overall model fit statistics were favorable, including χ²(N = 27) = 9.423, p = .99, CFI = 0.981, GFI = 0.975, TLI = 0.975, RMSEA = 0.065, and RMSR = 0.0860. These findings validate the theoretical framework that the CFS should yield a single, global score of coaching adherence.

Instrument

Test Type: Original

Format: The CFS is an observational instrument presented in a checklist format. It consists of 11 items requiring yes/no responses based on direct observation of the coach-coachee interaction. An optional supplementary step for video feedback is also included to facilitate joint review of previous sessions.

Language Available: English

Population Group: Human (Male; Female)

Age Group: Adulthood (18 years and older)

Population Details: The measure is intended for use with professionals (such as service providers, teachers, or board-certified behavior analysts) and caregivers of young children (under 5 years old) who have Autism or other developmental disabilities. The scale assesses the fidelity of the coach’s behavior, not the coachee’s.

Test Methodology: The development and evaluation employed rigorous psychometric procedures, including Test Reliability, Internal Consistency, Interrater Reliability, Factor Analysis, Exploratory Factor Analysis, and Test Sensitivity.

Keywords

Autism, Caregivers, Coaching Practices with Caregivers, Coaching Professionals, Developmental Disabilities, Evidence-Based Practices, Observational Measure, Psychometric Evaluation, Young Children

Authors

Author ORCID Identifier: No data is Available

Affiliation Email addresses:

• Hedda Meadan: [email protected]

• James D. Lee: No data is Available

• Michelle M. Sands: No data is Available

• Moon Y. Chung: No data is Available

• Pau García-Grau: No data is Available

Correspondence Address:

• Meadan, Hedda: University of Illinois Urbana-Champaign, Department of Special Education, 1310 South Sixth St., Champaign, Illinois, United States, 61820, [email protected]

Permissions & Fee and Test Year

Permissions: Contact Corresponding Author for permission.

Commercial Use: No

Fee: No

Test Year: 2023

Reference’s

Meadan, H., Lee, J. D., Sands, M. M., Chung, M. Y., & García-Grau, P. (2023). The Coaching Fidelity Scale (CFS): Development and evaluation of an observational measure of coaching fidelity. Infants & Young Children, 36(1), 37–52. https://doi.org/10.1097/IYC.0000000000000231

Items of the Coaching Fidelity Scale (CFS)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

The Coaching Fidelity Scale (CFS) consists of 11 items, with an additional optional video feedback item. The specific items are not provided, but they are designed to evaluate the extent to which professionals implement a set of coaching practices with fidelity, following a model that includes pre-observation, observation, and post-observation phases. Two specific problem-solving items (Problem-Solving items 2 and 4) were added to the initial 9-item CFS based on feedback from prior studies.