Additional Module Moderation – Münster Evaluation Questionnaire (MFE-ZMo)

Abstract

The additional module moderation (MFE-ZMo) is a specific item battery developed to assess the quality of an instructor’s moderation skills within university courses. This module serves as a supplement to the main “Münster Questionnaire for Evaluation” (MFE), designed by the Department of Psychology at the University of Münster. The core objective of the MFE-ZMo is to provide a flexible and economical instrument for customized teaching evaluation, allowing instructors to focus specifically on the interactive elements and presentation style used during their seminars or lectures.

Developed primarily for online administration, the scale prioritizes high efficiency and low temporal burden on students. The instrument is validated and demonstrates good internal consistency (Cronbach’s Alpha = .86) and strong discriminative validity, making it suitable for evaluating moderation quality at the course level.

Keywords

Teaching Evaluation, Moderation Skills, Higher Education, MFE, Psychological Measurement, Questionnaire, Psychometrics, University of Münster.

Authors

Dr. Dipl.-Psych. Meinald Thielsch, B. Sc. Ina Stegemöller

Purpose

The primary purpose of the MFE-ZMo module is to enable a focused and flexible assessment of an instructor’s specific skills in moderating teaching sessions, separate from general didactics or course content. The development of this supplementary module was driven by the need for short, economical scales suitable for web-based administration, especially given the time constraints students face near the end of the semester.

By separating specific thematic areas, such as moderation, into optional add-on modules, instructors can tailor the evaluation to their individual course design and ensure that students only answer questions relevant to the teaching methods employed, thereby increasing the quality and relevance of the feedback received.

Construct

The core construct measured by the MFE-ZMo is the quality of Moderation in university teaching. This falls under the broader category of Teaching Evaluation (Lehrevaluation). The scale assesses various aspects related to the instructor’s delivery and interactive competence, including organizational clarity, engagement, motivational impact, verbal expression, speaking confidence, ability to summarize complex facts, and provision of constructive feedback to students.

The item battery is designed to be unidimensional, meaning all items contribute to a single, overall score representing the perceived quality of the instructor’s moderation style.

Validity

The validity of the MFE-ZMo was investigated by examining its relationship with other indicators within the broader MFE framework, focusing on convergent, divergent, and discriminative validity.

• Convergent and Divergent Validity: The module demonstrated expected positive correlations (convergent) with related scales from the core MFE evaluation modules (MFE-Sr for seminars and MFE-Vr for lectures), such as ‘Instructor & Didactics’ (r ≈ .77–.78), ‘Materials’ (r ≈ .61–.75), self-assessed learning success (r ≈ .62–.63), and overall course judgment (r ≈ .60–.74). Divergent validity was supported by low to moderate negative correlations with the ‘Overburdening’ scale (r ≈ -.23 to -.34).
• Discriminative Validity: An ANOVA testing the ability of the module score to differentiate between various courses showed a significant difference among evaluated events (F = 4.59; df = 22; p < .01; Eta-Squared = .20). This result confirms that the MFE-ZMo is sensitive enough to detect differences in moderation quality across different teaching events.

Reliability

The internal consistency of the “Moderation” module is judged to be good for typical evaluation purposes at the course level. Based on the data collected from the revised seven-item version, the reliability, measured using Cronbach’s Alpha, was calculated at .86. Item analysis further supports the strong coherence of the scale, with item-level statistics (M, SD, T, FL) confirming the psychometric suitability of the individual items.

Factor Analysis

The data collected (N = 438) between Winter Semester 2010/11 and Summer Semester 2012 was subjected to item and factor analysis. The Kaiser-Meyer-Olkin (KMO) test indicated that the dataset was highly suitable for factor analysis, yielding an MSA value of .90.

An exploratory Factor Analysis (Principal Axis Factoring with oblique Promax rotation) was performed. Based on the Eigenvalue > 1 criterion and inspection of the Scree plot, a clear **one-factor solution** was identified. This single factor successfully explained 54.2% of the variance (Eigenvalue = 3.80). All seven items loaded significantly and strongly onto this common factor, with factor loadings (FL) ranging from .65 to .82, confirming the unidimensional structure of the scale.

Instrument

Test Type: Self-report questionnaire, Supplementary Module for Teaching Evaluation

Format: 7 items utilizing a 7-point Likert scale (1 = “stimme gar nicht zu” [strongly disagree] to 7 = “stimme vollkommen zu” [strongly agree]). An additional option, “nicht sinnvoll beantwortbar” (not meaningfully answerable), is provided.

Language Available: German (deutsch)

Population Group: University students enrolled in seminars and lectures.

Age Group: 18 to 55 years (Mean = 22.7, SD = 4.0)

Population Details: The validation sample consisted of 483 evaluations collected between Winter Semester 2010/11 and Summer Semester 2012. The majority of respondents (60.7%) were Psychology Bachelor students at the University of Münster.

Test Methodology: Online administration via a dedicated web-based platform (PHP/MySQL), typically presented optionally immediately following the core evaluation modules (MFE-Sr or MFE-Vr).

Keywords

Psychometrics, Online Survey, Student Feedback, Higher Education Quality, Internal Consistency, Discriminative Validity, Factor Analysis.

Authors

Author ORCID Identifier: N/A

Affiliation Email addresses: [email protected]

Correspondence Address: Dr. Dipl.-Psych. Meinald Thielsch, Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster, Germany.

Permissions & Fee and Test Year

The initial item development for the MFE supplementary modules began in 2003, based on work by Grabbe (2003). A significant item revision, including the introduction of two new moderation-related items (Items 2 and 5), occurred in Summer Semester 2010. The psychometric results reported here are based on data collected between Winter Semester 2010/11 and Summer Semester 2012. Information regarding permissions and usage fees should be directed to the Department of Psychology at the University of Münster.

Reference’s

• Bechler, J. & Thielsch, M. T. (2012). Evaluation von Lehrveranstaltungen am Fach Psychologie der Universität Münster. Bericht 2012. Münster: Universität Münster, Fachbereich Psychologie.
• Göritz, A. S., Soucek, S. & Bacher, J. (2005). Online-Befragung. Wiesbaden: VS Verlag für Sozialwissenschaften.
• Grabbe, A. (2003). Entwicklung eines Fragebogens zur Evaluation von Seminaren. Unveröffentlichte Diplomarbeit. Münster: Universität Münster, Fachbereich Psychologie.
• Haaser, J., Thielsch, M. T. & Moeck, P. (2007). Der Münsteraner Fragebogen zur Evaluation von Lehrveranstaltungen (MFE). Zeitschrift für Inklusion und Wissenschaft, 4(1), 3-10.
• Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76(5), 707–751.
• Moeck, P. & Thielsch, M. T. (2004). Münsteraner Fragebögen zur Evaluation von Lehrveranstaltungen. In B. Strauß & J. H. Baur (Hrsg.), Psychologische Diagnostik in der Sportpsychologie (S. 321–330). Göttingen: Hogrefe.
• Rindermann, H. (1996). Evaluation von Lehre und Lehrevaluation. Zeitschrift für Pädagogische Psychologie, 10(4), 189–205.
• Schmidt, J. & Loßnitzer, T. (2010). Lehrevaluation an Hochschulen. Ein Überblick über Instrumente zur Erfassung der Lehrqualität. Zeitschrift für Evaluation, 9(1), 107–126.

Items of the additional module moderation – Münster questionnaire for evaluation (MFE-ZMo)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.