Münster questionnaire for the evaluation of seminars (MFE-S)

Abstract

The Münster Questionnaire for the Evaluation of Seminars (MFE-S) is a validated, concise psychometric instrument designed for the efficient teaching evaluation of university seminars. This scale comprises three distinct item batteries which function as subscales, originally developed as a shortened version of a more comprehensive evaluation module utilized by the Department of Psychology at the University of Münster. The MFE-S focuses on three core dimensions: Didactics, Difficulty, and Self-Study. It was specifically optimized for high resource efficiency (Ökonomie) and deployment via a web-based platform to maximize participation rates and streamline data collection and feedback processes.

The MFE-S includes 14 total items, with nine core items contributing to the three main subscales. Psychometric properties, including internal consistency (ranging from Cronbach’s Alpha = .57 to .86) and structural validity, were confirmed using new data through Confirmatory Factor Analysis (CFA), supporting the hypothesized three-dimensional structure.

Keywords

Teaching evaluation, Seminar assessment, Didactics, Student self-study, Scale development, Psychometrics, Higher education, University of Münster.

Authors

Dr. Dipl.-Psych. Meinald Thielsch, Dr. Dipl.-Psych. Gerrit Hirschfeld (and collaborators, including Haaser, Moeck, and Grabbe).

Purpose

The primary purpose of the MFE-S is to provide an economical and psychometrically sound instrument for the systematic evaluation of seminar quality in a university setting. It was constructed specifically for online administration to overcome the length limitations often associated with traditional evaluation instruments, which can negatively impact response rates and efficiency.

By concentrating on central aspects of lecturer success—namely **Didactics** and teaching presence—the scale aims to secure high voluntary participation while still delivering meaningful data. This feedback is essential for continuous quality assurance, enabling instructors to identify strengths and weaknesses, improve their teaching qualifications, and facilitating discussions between faculty and students.

Construct

The MFE-S measures the construct of **Teaching Evaluation** (Lehrevaluation) structured around three distinct dimensions of seminar quality:

• Didactics: This subscale assesses the lecturer’s pedagogical skill, clarity in explaining complex material, and the perceived suitability of teaching methods for conveying course content.
• Difficulty: This dimension evaluates whether the instructor appropriately adjusts the seminar level to the students’ prior knowledge and addresses the student perception of the course content being overly challenging.
• Self-Study: This subscale measures the provision of adequate and helpful supplementary materials (e.g., literature, scripts) and gauges the student’s personal commitment to preparing for seminar sessions.

Validity

The validation of teaching evaluation instruments is challenging, as teaching success is influenced by numerous variables, including student prerequisites and course type (Marsh, 1984). The MFE-S was developed from a comprehensive item pool (Grabbe, 2003) but was deliberately shortened to focus on the most critical aspects of teaching success, consistent with models like Rindermann’s (2001) multimodal condition model.

This concentrated focus on core areas such as instructional appearance and **Didactics** provides strong evidence for the instrument’s **Content Validity** (Inhaltsvalidität) within the domain of student seminar assessments. Furthermore, the high voluntary response rates observed (averaging > 60% for core seminars) suggest positive acceptance of the instrument by both students and faculty, supporting its ecological validity.

Reliability

The internal consistency of the MFE-S subscales was assessed using Cronbach’s Alpha. The reliability coefficients demonstrated acceptable to good consistency, particularly given the instrument’s brevity (only three items per subscale):

• Didactics: Alpha = .86
• Difficulty: Alpha = .62
• Self-Study: Alpha = .57

The slightly lower Alpha values for the Difficulty and Self-Study subscales are acknowledged by the developers, as the items were intentionally selected to capture diverse facets of their respective dimensions rather than highly redundant content. These reliability figures are comparable to those reported for other, often more extensive, German-language teaching evaluation instruments (e.g., FEVOR, HILVE, KIEL, TRIL).

Factor Analysis

The dimensional structure of the MFE-S, previously suggested by exploratory principal components analysis (Haaser, 2006), was confirmed using **Confirmatory Factor Analysis** (CFA). This utilized a linear measurement model with three correlated factors (Didactics, Difficulty, Self-Study).

The CFA was conducted using the conventional Maximum Likelihood estimator within the AMOS software package. The analysis demonstrated a good fit for the hypothesized three-factor model, confirming the expected structure of the nine core items. Key fit indices provided robust support for the model: Chi-Square = 31 (df = 24); TLI = .97, CFI = .98, and RMSEA = .05.

Instrument

Test Type: Psychological Scale / Standardized Questionnaire (Validated)

Format: Web-based self-report questionnaire. Core items (3-11) use a 7-point Likert scale (1 = completely inaccurate, 7 = completely accurate).

Language Available: German (Deutsch)

Population Group: University Students

Age Group: 18 to 67 years (M = 23.1, SD = 3.6)

Population Details: Data was collected from students, primarily psychology majors, at the University of Münster between Winter Semester 2002/3 and Summer Semester 2008 (N = 9757 individual ratings). The psychometric validation analysis utilized a subsample of N = 103 guaranteed non-identical respondents from the Winter Semester 2007/08 cohort.

Test Methodology: The MFE-S is administered online via a platform based on PHP and MySQL databases. Instructions and demographic information (age, gender, semester, field of study) are collected upon initial access to the system. Scoring involves summing or averaging the response values of the items within the three subscales, and the online system provides comparative feedback against other courses at the university.

Keywords

Confirmatory Factor Analysis, Psychometrics, Didaktik, Online survey, Cronbach’s Alpha, University of Münster, Maximum Likelihood Estimation, Content Validity.

Authors

Author ORCID Identifier: Not provided in source content.

Affiliation Email addresses: [email protected] (Dr. Meinald Thielsch); [email protected] (Dr. Gerrit Hirschfeld).

Correspondence Address: Dr. Dipl.-Psych. Meinald Thielsch, Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster, Germany.

Permissions & Fee and Test Year

Test Year: Developed starting Winter Semester 2002/3. Psychometric validation published based on data collected up to Summer Semester 2008.

Permissions & Fee: The scale was developed internally for use in the Department of Psychology at the University of Münster. Information regarding external licensing or fees is not provided in the source documentation.

Reference’s

The scale development and validation reference several existing studies and instruments:

• Gediga, G., Schlegl, M., & Schuler, J. (2000). KIEL: Kieler Evaluationsbogen für Lehrveranstaltungen.
• Gollwitzer, M., & Schlotz, W. (2003). TRIL: Trierer Rating-Skala zur Evaluation von Lehrveranstaltungen.
• Grabbe, S. (2003). Entwicklung und Validierung eines Evaluationsfragebogens für Lehrveranstaltungen im Fach Psychologie. Unveröffentlichte Diplomarbeit, Fachbereich Psychologie, Westfälische Wilhelms-Universität Münster.
• Haaser, J. (2006). Zur psychometrischen Qualität von Online-Lehrevaluation. Unveröffentlichte Diplomarbeit, Fachbereich Psychologie, Westfälische Wilhelms-Universität Münster.
• Haaser, J., Thielsch, M. T., & Moeck, P. (2007). Webbasierte Lehrevaluation im Fach Psychologie an der Universität Münster. Zeitschrift für Evaluation, 6(1), 107-124.
• Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76(5), 707-754.
• Rindermann, H. (1996). Evaluation von Hochschullehre. Göttingen: Hogrefe.
• Rindermann, H. (2001). HILVE: Hallescher Inventar zur Lehrevaluation.
• Staufenbiel, T. (2000). FEVOR: Fragebogen zur Evaluation von Vorlesungen.
• Thielsch, M. T., Haaser, J., & Moeck, P. (2008). Akzeptanz und Nutzung von Lehrevaluation. Vortrag auf der 10. Tagung der Fachgruppe Methoden und Evaluation der Deutschen Gesellschaft für Psychologie, 24.-26. September 2008, Lüneburg.

Items of the Münster questionnaire for the evaluation of seminars (MFE-S)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

The MFE-S consists of three core item batteries (Didaktik, Schwierigkeit, Selbststudium), plus supplementary items (1, 2, 12, 13, 14) that were administered but not included in the psychometric analysis of the subscales.

Items zu Didaktik

Items zu Schwierigkeit

Items zu Selbststudium

Zusätzlich vorgegebene, hier aber nicht weiter behandelte Items