Feedback Tool for Rescue Force Development – Basic Training (FIRE-B)

Abstract

The feedback tool for Rescue Force Development – Basic Training (FIRE-B) is a psychological scale designed to measure the perceived quality of foundational training programs for firefighter personnel. This instrument is specifically targeted at training delivered at the community and district levels in Germany for both voluntary and professional fire brigades.

The development process was rigorous, commencing with a qualitative analysis of key characteristics defining excellent basic training, involving 21 experts and trainees. This informed the adaptation and creation of items based on existing scales used for management training and university teaching evaluation. The FIRE-B was subsequently tested and validated across two independent samples, totaling 708 participants from various basic fire brigade courses.

Factor analysis established a robust six-factor structure: Structure & Didactics, Support & Promotion, Group Dynamics, Practice Relevance, Material & Premises, and Competence Acquisition. The scales have demonstrated high internal consistency (reliability) and provide evidence supporting content, construct validity, and criterion validity.

Keywords

Fire brigade, evaluation, training, rescue force, basic training, competence acquisition, organizational psychology, didactic quality.

Authors

Meinald T. Thielsch, Lena Kläpker, Lisa Streppel

Purpose

The primary objective of the FIRE-B instrument is to provide a standardized and reliable feedback mechanism for the evaluation of basic technical training courses within the fire service sector. By collecting data on trainees’ perceptions immediately following a course, the scale facilitates the objective assessment of instructional quality and training effectiveness at the grassroots level (municipal and district).

The instrument is applicable to foundational training programs, such as the Brandmeisterausbildung (Professional Fire Brigade) and the Truppmann- or Truppführerausbildung (Voluntary Fire Brigade), as well as other introductory courses not conducted at state fire schools. The feedback generated is intended to support instructors in reflecting on their teaching practices and enables organizational leaders to identify areas for improvement in training content, delivery, and resources.

Construct

The FIRE-B measures the perceived quality of basic rescue force training, conceptualized as six distinct but interrelated dimensions (subscales) derived from qualitative research on successful firefighter training characteristics:

1. Structure & Didactics: Measures the clarity, organization, and instructional quality provided by the trainers.
2. Support & Promotion: Reflects the trainers’ engagement, motivation, and responsiveness to trainees’ needs and difficulties.
3. Group: Assesses the cohesion and active participation dynamics within the trainee group.
4. Practice: Evaluates the realism of practical exercises and the adequacy of post-exercise review and feedback.
5. Material & Premises: Gauges the quality and sufficiency of training resources, including vehicles, equipment, and training facilities.
6. Kompetenzerwerb (Competence Acquisition): Acts as an outcome scale, measuring the trainees’ self-assessed learning gains, improved safety awareness, and preparedness for deployment.

Validity

Extensive validation studies (Studies II and III, total N=708) were conducted to establish the psychometric properties of the FIRE-B. Evidence supports content, construct validity, and criterion validity. Content validity was initially established through a qualitative research approach (Study I), utilizing the Critical Incident Technique (CIT) and open-ended questions with 21 subject matter experts and trainees to ensure comprehensive coverage of essential training aspects.

Confirmatory Factor Analysis (CFA) in Study III verified the proposed six-factor structure of the final 29-item scale. The model fit indices demonstrated excellent to acceptable fit with the data (e.g., RMSEA = .05; SRMR = .04; CFI = .95; TLI = .95), confirming the theoretical structure of the instrument.

Reliability

The internal consistency of the FIRE-B scales is consistently high across all six subscales. Reliability coefficients calculated using both Cronbach’s Alpha and McDonald’s Omega were reported to be consistently greater than or equal to .76 (Alpha ≥ .76; Omega ≥ .76). These values indicate that the items within each factor measure their respective dimensions consistently.

Factor Analysis

The structure of the FIRE-B was developed and confirmed across two validation studies. An initial Exploratory Factor Analysis (EFA) using 44 items (Study II, N=257) indicated a strong case for a six-factor solution (Kaiser-Meyer-Olkin Test = .94). This six-factor model was selected over a five-factor alternative due to superior conceptual clarity, successfully separating the broad factor concerning support, promotion, structure, and didactics into two distinct factors.

The final 29-item structure was confirmed by a Confirmatory Factor Analysis (CFA) in Study III (N=451). This analysis confirmed the six-factor structure, which includes one outcome scale (Competence Acquisition) and five process scales (Structure & Didactics, Support & Promotion, Group, Practice, and Material & Premises), providing a detailed framework for evaluating the multi-faceted nature of basic firefighter training.

Instrument

Test Type: Feedback Tool / Evaluation Instrument

Format: Self-report questionnaire, consisting of 29 core items distributed across six scales, plus four optional items (Items 30-33). Items 1-31 utilize a seven-point Likert scale (1 = strongly disagree to 7 = strongly agree). Item 32 uses a six-point grading scale (1 = very good to 6 = insufficient). Item 33 is an open-ended format for qualitative feedback.

Language Available: German (deutsch)

Population Group: Active members of rescue forces, specifically firefighters in both professional (Berufsfeuerwehr) and voluntary (Freiwillige Feuerwehr) services.

Age Group: Adults (Validation samples ranged from 16 to 63 years old).

Population Details: Primarily used for evaluating basic training courses (e.g., Truppmann, Truppführer, Brandmeisterausbildung) conducted at the community and district level in Germany. The development and validation samples were predominantly male (approx. 90%) and primarily drawn from North Rhine-Westphalia.

Test Methodology: The instrument is typically administered post-training or post-course section. Scoring involves calculating the mean response value for each of the six scales. Higher mean scores indicate a more positive evaluation of the training quality. Guidelines recommend a minimum of eight completed questionnaires for aggregation and anonymized evaluation results.

Keywords

Firefighter training, training evaluation, basic rescue operations, German fire service, instructional quality, organizational development, scale validation.

Authors

Author ORCID Identifier: Not provided.

Affiliation Email addresses:

• Meinald T. Thielsch: [email protected]
• Lena Kläpker: [email protected]
• Lisa Streppel: [email protected]

Correspondence Address: Westfälische Wilhelms-Universität Münster, Institut für Psychologie, Germany.

Permissions & Fee and Test Year

The scale was developed through a cooperative research project between the Westfälische Wilhelms-Universität Münster and the Institut der Feuerwehr NRW (IdF NRW). The main empirical validation studies (Studies I, II, and III) were conducted between 2017 and 2018. The paper detailing item construction and selection (Streppel, 2018) can be made available as a PDF by the authors upon request.

Reference’s

• Babiel, S. (2016). Evaluation der Führungsausbildung bei der Feuerwehr: Entwicklung und Validierung eines Fragebogens. Unveröffentlichte Masterarbeit, Westfälische Wilhelms-Universität Münster.
• Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. Q., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research & Evaluation, 18(6).
• BHKG (2015). Gesetz über den Brandschutz, die Hilfeleistung und den Katastrophenschutz in Nordrhein-Westfalen.
• Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276.
• Döring, N., & Bortz, J. (2016). Forschungsmethoden und Evaluation in den Sozial- und Humanwissenschaften (5. Aufl.). Springer.
• Epskamp, S. (2017). semPlot: Path diagrams and visual analysis of various SEM packages’ output. R package version 1.1.
• Feuerwehr-Dienstvorschrift 2 [FwDV 2] (2012). Ausbildung der Freiwilligen Feuerwehren. (Herausgegeben vom Ausschuss für Feuerwehrangelegenheiten, Katastrophenschutz und zivile Verteidigung).
• Feuerwehr-Dienstvorschrift 3 [FwDV 3] (2008). Einheiten im Lösch- und Hilfeleistungseinsatz. (Herausgegeben vom Ausschuss für Feuerwehrangelegenheiten, Katastrophenschutz und zivile Verteidigung).
• Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358.
• Gläßer, B., Eickelmann, B., & Gläßer, S. (2002). Der Test zur Erfassung von Lehrqualität im Hochschulbereich (TRIL). Universitätsverlag Göttingen.
• Grabbe, C. (2003). Evaluation der Lehrveranstaltungsqualität an der Hochschule der Bundeswehr Hamburg. Unveröffentlichte Diplomarbeit, Helmut Schmidt Universität/Universität der Bundeswehr Hamburg.
• Grötemeier, S., & Thielsch, M. T. (2010a). MFE-ZGr: Fragebogen zur Messung des Feedbacks zu Gruppenarbeiten in der Lehre. Westfälische Wilhelms-Universität Münster.
• Grötemeier, S., & Thielsch, M. T. (2010b). MFE-ZHa: Fragebogen zur Messung des Feedbacks zu Hausaufgaben in der Lehre. Westfälische Wilhelms-Universität Münster.
• Guttman, L. (1954). Some necessary conditions for common-factor analysis. Psychometrika, 19(2), 149–161.
• Hirschfeld, G., & Thielsch, M. T. (2009). MFE-V: Fragebogen zur Messung des Feedbacks zu Vorlesungen. Westfälische Wilhelms-Universität Münster.
• Kaiser, H. F., & Dickman, K. (1959). Analytic determination of common factors. American Psychologist, 14, 425.
• Kläpker, L. (2018). Evaluation der Basisausbildung bei der Feuerwehr: Entwicklung und Validierung des Feedback-Instruments zur Rettungskräfteentwicklung – Basisausbildung (FIRE-B). Unveröffentlichte Masterarbeit, Westfälische Wilhelms-Universität Münster.
• Moosbrugger, H., & Schermelleh-Engel, K. (2008). Exploratorische Faktorenanalyse. In: H. Moosbrugger & A. Kelava (Hrsg.), Testtheorie und Fragebogenkonstruktion (S. 305–342). Springer.
• Olde lütke Beverborg, T. (2017). Entwicklung und Validierung des Feedback-Instruments zur Rettungskräfteentwicklung – Dozenten (FIRE-Dozent). Unveröffentlichte Bachelorarbeit, Westfälische Wilhelms-Universität Münster.
• Revelle, W. (2018). psych: Procedures for psychological, psychometric, and personality research. R package version 1.8.10.
• Rindermann, H., & Amelang, M. (1994). HILVE (Hamburger Inventar zur Lehrveranstaltungsevaluation). Diagnostica, 40(3), 263–280.
• Robertz, P. (2015). Evaluation der Führungskräfteausbildung bei der Feuerwehr: Entwicklung und Validierung eines Fragebogens. Unveröffentlichte Masterarbeit, Westfälische Wilhelms-Universität Münster.
• Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.
• Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.
• Schulte, H. S., & Thielsch, M. T. (in press). Feedback-Instrument zur Rettungskräfte-Entwicklung (FIRE). In: M. T. Thielsch (Hrsg.), Psychologische Eignungsdiagnostik für die Feuerwehr. Springer.
• Staufenbiel, T. (2000). FEVOR (Fragebogen zur Evaluation von Vorlesungen und Veranstaltungen). Hogrefe.
• Steinmetz, H. (2015). Analyzing observed composite variables and unobserved common factors with latent variable modeling. Behavior Research Methods, 47(1), 17–29.
• Streppel, L. (2018). Entwicklung und Validierung des Feedback-Instruments zur Rettungskräfteentwicklung – Basisausbildung (FIRE-B). Unveröffentlichte Bachelorarbeit, Westfälische Wilhelms-Universität Münster.
• Taber, C. J., Plumb, M. J., & Jolemore, E. (2008). The fire service: Psychological stressors and their effects. International Journal of Emergency Mental Health, 10(3), 161–171.
• Tanguma, J. (2001). Conducting and interpreting factor analysis. ERIC Clearinghouse on Assessment and Evaluation.
• Thielsch, M. T., Busjan, S., & Frerichs, A. (2018). FIRE-P: Feedback-Instrument zur Rettungskräfte-Entwicklung – Prüfung. Westfälische Wilhelms-Universität Münster.
• Thielsch, M. T., & Hirschfeld, G. (2012). MFE-Sr: Fragebogen zur Messung des Feedbacks zu Seminaren. Westfälische Wilhelms-Universität Münster.
• VAP1.2-Feu (2015). Verordnung über die Ausbildung und Prüfung der Beamtinnen und Beamten der Laufbahngruppe 1, 2. Einstiegsamt, des feuerwehrtechnischen Dienstes (VAP1.2-Feu). Nordrhein-Westfalen.
• Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321–327.
• Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A review and comparison of 20 procedures. Psychological Methods, 5(4), 498–521.
• Wickham, H. (2011). The split-apply-combine strategy for data analysis. Journal of Statistical Software, 40(1), 1–29.

Items of the feedback tool for Rescue Force Development – Basic Training (FIRE-B)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

Tabelle 2

Je nach Evaluationskontext optionale Items