Münster questionnaire for the evaluation of exams (MFE-K)

Abstract

The Münster questionnaire for the evaluation of exams (MFE-K) is a validated psychological scale designed for the systematic evaluation of written university examinations (Klausuren). Functioning as an additional module to the comprehensive “Münster Questionnaire for Evaluation,” the MFE-K addresses the critical need for quality assurance in examination procedures within higher education, a necessity highlighted by the structural changes imposed by the Bologna Process.

Developed primarily at the Westfälische Wilhelms-Universität Münster, the instrument consists of 27 items, focusing on three core psychometric dimensions: perceived Student Burden, Transparency of examination requirements, and the quality of Exam Design. The scale has undergone extensive revision based on qualitative studies to enhance its theoretical foundation and content validity, making it a reliable tool for gathering student feedback immediately following an examination.

Keywords

Examination Evaluation, Student Stress, Teaching Evaluation, Higher Education, Psychological Scale, Test Transparency, Written Exams, University Assessment, Quality Assurance.

Authors

Benjamin Froncek, Meinald Thielsch.

Purpose

The primary purpose of the MFE-K is to provide a standardized, psychometrically sound instrument for evaluating university examinations, which are often neglected by generic teaching evaluation instruments. The scale was initiated in response to the increased frequency and intensity of examinations resulting from the shift towards continuous assessment models mandated by the Bologna Process.

The MFE-K allows academic departments and examiners to gather anonymous, immediate feedback on key operational and pedagogical aspects of an examination. This information is used centrally to identify strengths and weaknesses in examination practice, support quality assurance initiatives, and guide improvements concerning clarity, fairness, and student workload management.

Construct

The MFE-K measures the overall construct of Examination Evaluation (Prüfungsevaluation) by operationalizing three distinct, yet correlated, sub-constructs:

• Student Burden (Belastung der Studierenden): This subscale captures the subjective strain experienced by students due to the workload, time management challenges, and the density of examinations within a semester.
• Transparency: This factor measures the clarity and availability of information regarding exam requirements, content scope, key topics, required literature, and the overall format (type and length of questions) prior to the examination.
• Exam Design (Klausurgestaltung): This subscale assesses the quality of the exam paper itself, including the clarity and precision of item formulation, unambiguous instructions, adequate spatial organization, and suitability of the chosen response format.

Validity

Validation efforts for the MFE-K have demonstrated evidence supporting its content and factorial validity, addressing the inherent challenge in defining universally accepted criteria for high-quality examinations.

Content Validity: The current version of the MFE-K is strongly anchored in empirical analyses regarding the characteristics of good examinations (Froncek & Thielsch, 2011). The instrument’s structure was refined following a qualitative study (Winter 2009/10) involving guideline interviews with both students and examiners, ensuring that the items adequately cover all relevant aspects of written assessments from the student perspective, thereby enhancing content validity significantly compared to initial versions.

Factorial Validity: A Confirmatory Factor Analysis (CFA) performed on data from Winter Semester 2010/11 (N~680) confirmed the three-dimensional structure (Student Burden, Transparency, and Exam Design) with acceptable model fit (Chi-Square = 107.83, df = 41; TLI = .96, CFI = .97, RMSEA = .06). Furthermore, the scale demonstrated sensitivity to differences across various assessed examinations in a multivariate analysis (F = 11.9; df = 48; p < .01), providing evidence for criterion-related validity.

Reliability

The internal consistency of the MFE-K subscales is considered sufficient to good, particularly given the small number of items composing each scale. Reliability estimates, calculated using Cronbach’s Alpha, were derived from a sample of 688 evaluation forms collected during the Winter Semester 2010/11.

• The three-item Student Burden subscale achieved a Cronbach’s Alpha of .79.
• The four-item Transparency subscale achieved a Cronbach’s Alpha of .84.
• The four-item Exam Design subscale achieved a Cronbach’s Alpha of .76.

Factor Analysis

The scale development utilized both exploratory and Confirmatory Factor Analysis (CFA) techniques across several semesters of data collection. Initial exploratory work suggested a two-factor model, but subsequent CFA indicated poor fit. This led to a major theoretical revision based on qualitative data identifying key aspects of good examinations (Froncek & Thielsch, 2011).

The revised structure resulted in 11 core psychometric items loading onto three correlated factors: Student Burden, Transparency, and Exam Design. The final CFA model successfully confirmed this three-dimensional structure using the Maximum Likelihood estimator in AMOS, achieving robust fit indices (CFI = .97; RMSEA = .06). The inter-factor correlations ranged from -.30 (Transparency and Burden) to .62 (Exam Design and Transparency).

Instrument

Test Type: Paper-and-Pencil Questionnaire (Self-Report)

Format: The instrument contains 27 items in total. The 11 core items relating to the three main psychometric subscales employ a 7-point Likert scale (1 = “stimme gar nicht zu” to 7 = “stimme vollkommen zu”). The remaining items use open-ended, dichotomous (Yes/No), or multiple-choice formats to capture demographic data, preparation methods, and specific problems encountered by students.

Language Available: German (Deutsch)

Population Group: University Students (Specifically those taking written examinations in academic degree programs).

Age Group: 18 to 48 years (Sample Mean: 23.08 years; SD = 3.41)

Population Details: The validation sample (N=688) consisted primarily of Psychology students (76.3% Bachelor, 20.7% Master) at the Westfälische Wilhelms-Universität Münster, evaluated across 17 different examinations in Winter Semester 2010/11.

Test Methodology: The questionnaire is administered immediately following the completion of the written examination (Klausur). Students are instructed to complete the anonymous form and submit it within a specified timeframe (typically two weeks).

Keywords

Examination Evaluation, Student Burden, Test Design, Transparency, Higher Education Quality Assurance, Bologna Reform, University Pedagogy, Teaching Evaluation, Psychometric Scale Development.

Authors

Author ORCID Identifier: Not provided in source content.

Affiliation Email addresses: [email protected], [email protected]

Correspondence Address: Benjamin Froncek, FernUniversität in Hagen, Institut für Psychologie, Universitätsstr. 33, 58097 Hagen; Dr. Dipl.-Psych. Meinald Thielsch, Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster.

Permissions & Fee and Test Year

Test Year: Initial development began in Winter Semester 2007/8; the finalized, three-factor structure was validated using data from Winter Semester 2010/11.

Permissions and Fee: The scale was developed for internal academic evaluation and quality control within the Department of Psychology at WWU Münster. Specific information regarding external usage permissions or associated fees is not detailed in the source documentation.

Reference’s

Dany, Szczyrba & Wildt (2008). Cited regarding the increase in examinations due to the Bologna process.

Eugster & Lutz (2004). Leitfaden für das Planen, Durchführen und Auswerten von Prüfungen an der ETHZ. Used as a source for item construction.

Froncek (2010). Cited regarding confirmatory testing and the need for theoretical basis.

Froncek & Thielsch (2011). Cited regarding empirical analyses of good examination characteristics, forming the basis for scale modification.

Kultusministerkonferenz (2005). Cited regarding the recognized need for evaluation of examination practices.

Marsh, H. W. (1984). Cited in comparison to validation problems in teaching evaluation.

Mayring, P. (2003). Cited regarding the qualitative content analysis methodology used in item revision.

Rindermann (1996). Cited regarding the benefits and purposes of teaching evaluation.

Thielsch et al. (2008). Cited regarding the detailed procedure of examination evaluation.

Thielsch et al. (2010). Cited regarding earlier revisions and standardization of the response format.

Items of the Münster questionnaire for the evaluation of exams (MFE-K)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

Items zu Belastung der Studierenden

Anmerkung: Die Nummerierung der Items entspricht ihrer Position im Fragebogen.

Items zu Transparenz und Organisation

Items zu Klausurgestaltung

Zusatzfragen (Demographie)

Ergänzend werden folgende Items vorgegeben:

Zusatzfragen (Vorbereitung und Information)

Zu Beginn des Fragebogens werden die Informationsquellen und die Vorbereitung für die Klausur mit folgenden Items erfasst:

Zusätzlich vorgegebene, hier aber nicht weiter behandelte Items (Am Ende des Fragebogens)