Engagement with Criticism Scale (ECS)

Abstract

The Engagement with Criticism Scale (ECS), developed by Orosz et al. (2023), is a psychometric instrument designed to evaluate the multifaceted dimensions of how individuals respond to negative feedback. Specifically, the ECS measures three distinct engagement styles: Criticism Over-Engagement, Criticism Disengagement, and Criticism Constructive Engagement. The instrument was initially created to support research investigating the unique contributions of growth mindset and trait mindfulness in fostering the motivation to learn from criticism.

The scale was validated using a nine-item structure on an online sample consisting of Hungarian adolescents and adults. The development process involved established scale construction principles and included robust analyses of factor structure, internal consistency, and various forms of validity, ensuring its suitability for use in psychological and educational research settings.

Keywords

Criticism Over-Engagement, Criticism Disengagement, Criticism Constructive Engagement, Criticism, Psychological Engagement, State Trait Level Measures

Authors

Orosz, Gábor, Evans, Kristopher M., Török, Lilla, Bőthe, Beáta, Tóth-Király, István, Sik, Kata, Gál, Éva

[quads id=5]

Purpose

The primary purpose of the Engagement with Criticism Scale (ECS) is to provide researchers and practitioners with a nuanced tool capable of quantifying multiple psychological and behavioral responses to criticism. Unlike instruments that only measure general defensiveness or avoidance, the ECS differentiates between adaptive learning strategies and maladaptive coping mechanisms.

By assessing the three core dimensions—constructive engagement, over-engagement (e.g., rumination), and disengagement (e.g., avoidance)—the scale facilitates a deeper understanding of the individual differences that modulate the impact of negative feedback on motivation, learning, and psychological well-being.

Construct

The central psychological construct measured by this scale is Criticism Engagement. This construct is defined by the three primary subscales which represent different patterns of response to feedback:

• Criticism Constructive Engagement: Reflects the adaptive utilization of criticism as a tool for personal or professional growth.
• Criticism Over-Engagement: Characterized by excessive rumination, preoccupation, and emotional distress following criticism.
• Criticism Disengagement: Involves actively ignoring, dismissing, or avoiding critical feedback.

Validity

Evidence for the validity of the ECS was established through several psychometric analyses. For Convergent Validity, the scale demonstrated theoretically sound correlations with other self-reported constructs. Specifically, components of the ECS correlated reasonably with measures of rumination, positive reappraisal, and depression, indicating that the scale accurately captures related constructs within the emotional and cognitive processing domain.

To establish Discriminant Validity, which ensures that the scale measures a unique construct and is not merely capturing response bias, the authors assessed its relationship with social desirability. Using linear logistic regression, the findings indicated that correlations between ECS scores and social desirability were non-significant, suggesting that the scale is relatively free from this common source of measurement error.

Reliability

The internal consistency of the Engagement with Criticism Scale was evaluated using Cronbach’s alpha coefficients. Across the three distinct subscales, the resulting alpha coefficients ranged from .69 to .85. This range indicates acceptable to good internal consistency, confirming that the items within each subscale reliably measure their respective facets of criticism engagement.

Factor Analysis

The structural integrity of the ECS was confirmed through a Confirmatory Factor Analysis (CFA). The analysis tested a first-order, three-factor, nine-item model, which demonstrated strong goodness-of-fit indices (χ2 = 127.078, df=24, CFI=0.969, TLI=0.954, RMSEA=0.054, 90% CI 0.045-0.064).

These fit indices support the theoretical model that criticism engagement is best understood as a construct composed of three correlated, yet distinct, dimensions. Furthermore, the standardized factor loadings were robust, ranging from 0.670 to 0.848, confirming that each item strongly contributes to its intended factor. The three empirically supported subscales are: Criticism constructive engagement, Criticism disengagement, and Criticism over-engagement.

Instrument

Test Type: This is an original inventory/questionnaire.

Format: The instrument consists of 9 items. Responses are captured using a six-point Likert scale, with options ranging from “Almost never” to “Almost always”. The test is administered electronically.

Language Available: The scale is available in English and Hungarian.

Population Group: The instrument is designed for human populations, including males and females.

Age Group: The scale is suitable for a wide range of ages, including Adolescence (13-17 years), Adulthood (18 years & older), Young Adulthood (18-29 years), Thirties (30-39 years), Middle Age (40-64 years), and Aged (65 years & older).

Population Details: The development study included adolescent and adult participants from Hungary, with ages ranging from 13 to 71 years old.

Test Methodology: The development and validation process involved assessments of Test Validity, Convergent Validity, Discriminant Validity, Test Reliability, Internal Consistency, Factor Analysis, and Confirmatory Factor Analysis.

Keywords

Criticism Over-Engagement, Criticism Disengagement, Criticism Constructive Engagement, Criticism, Psychological Engagement, State Trait Level Measures

[quads id=5]

Authors

Author ORCID Identifier: Orosz, Gábor: http://orcid.org/0000-0001-5883-6861; Bőthe, Beáta: http://orcid.org/0000-0003-2718-4703; Sik, Kata: http://orcid.org/0000-0002-0897-258X; Gál, Éva: http://orcid.org/0000-0002-7515-7024.

Affiliation Email addresses: Orosz, Gábor: [email protected]

Correspondence Address: Orosz, Gábor: Université du Littoral Côte d’Opale, Université d’Artois, Université de Lille, ULR 7369-URePSSS-Unité de Recherche Pluridisciplinaire Sport Santé Société, Liévin, France, [email protected]

Permissions & Fee and Test Year

The ECS is available for use without a fee. Interested parties seeking permission to use the scale for research or practical application should contact the corresponding author, Gábor Orosz. The scale was developed and published in 2023.

Reference’s

Orosz, G., Evans, K. M., Török, L., Bőthe, B., Tóth-Király, I., Sik, K., & Gál, É. (2023). The differential role of growth mindset and trait mindfulness in the motivation of learning from criticism. Mindfulness, 14(4), 868–879. https://doi.org/10.1007/s12671-023-02117-4

[quads id=5]

Items of the Engagement with Criticism Scale (ECS)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

The provided text indicates that the specific test items are not available within the document itself. However, it is noted that the items can be found in the supplemental material of the source reference article.