Additional module presentations – Münster evaluation questionnaire (MFE-ZRe)

Abstract

The additional module presentations—Münster Questionnaire for Evaluation (MFE-ZRe)—is an item battery designed specifically for the evaluation of student papers and presentations (Referate) within university courses. This instrument serves as a specialized, validated component of the comprehensive Münster Questionnaire for Evaluation (MFE) system developed at the Westfälische Wilhelms-Universität Münster.

The MFE-ZRe consists of 13 items in total, with eight core items forming two distinct subscales: support provided by the instructor and the student’s perceived competence acquisition during the presentation process. Developed to address the need for short, economical, and flexible evaluation tools suitable for web-based surveys, the MFE-ZRe is typically administered online following the core MFE modules for seminars (MFE-Sr) or lectures (MFE-Vr).

Keywords

Teaching Evaluation, Lehrevaluation, University, Student Presentations, Competence Acquisition, Instructor Support, MFE-ZRe, Psychometrics

Authors

Ina Grötemeier, Dr. Dipl.-Psych. Meinald Thielsch

[quads id=5]

Purpose

The primary purpose of the MFE-ZRe module is to allow for the specific and detailed evaluation of student presentations (Referate) within higher education settings. As part of a larger system of modular teaching evaluation, it provides focused feedback on aspects that are not covered by the general MFE base questionnaires.

This module helps instructors assess two key areas: the quality and helpfulness of the support they provide to students during presentation preparation and execution, and the extent to which students perceive improvement in their academic and rhetorical competencies as a result of the assignment. The modular approach ensures high economy by only presenting students with questions relevant to the specific didactic format of the course.

Construct

The MFE-ZRe measures aspects of Teaching Evaluation (Lehrevaluation) specific to student presentations. It is structured around two subscales, reflecting the multidimensional nature of academic presentations:

• Instructor Support (Dozent): This subscale captures the student’s perception of the instructor’s engagement, guidance, and expertise provided before and during the presentation (e.g., providing preparation time, helpful suggestions, and supplementary subject-specific knowledge).
• Competence Acquisition (Kompetenzerwerb): This subscale assesses the student’s self-reported development of key academic skills, such as engaging with scientific texts and improving rhetorical abilities, resulting from the preparation and delivery of the presentation.

Validity

The validity of the MFE-ZRe is supported primarily by its conceptual grounding in content validity, as its items were derived from an extensive analysis of characteristics defining good teaching (Grabbe, 2003) and tailored to specific course formats.

Empirical evidence for criterion validity was established by correlating the subscale scores with external indicators from the overall course evaluation (MFE-Sr). The results demonstrated a plausible pattern of correlations:

• The Competence Acquisition scale showed a strong relationship with the overall course assessment (.55) and self-assessed learning success (.54; p < .01).
• The Instructor Support scale showed a weaker, yet significant, relationship with the overall course assessment (.32) and self-assessed learning success (.35; p < .01).

These findings suggest that the Competence Acquisition scale is more closely tied to the perceived overall success and learning outcome of the seminar, while Instructor Support measures a distinct, yet related, component of the teaching process.

Reliability

The internal consistency of the two subscales was assessed using Cronbach’s Alpha, yielding adequate results for evaluation purposes at the course level.

• For the three items of the Instructor Support subscale, Cronbach’s Alpha was .68.
• For the five items of the Competence Acquisition subscale, Cronbach’s Alpha was .76.

Factor Analysis

An exploratory Factor Analysis (Principal Axis Factoring with oblique Promax rotation) was conducted on the core items (N = 331). The analysis suggested a two-factor solution based on the eigenvalue > 1 criterion.

• Factor 1 (Competence Acquisition): Eigenvalue of 3.27, explaining 40.6% of the variance. This factor primarily loaded on Items 6, 7, 9, 10, and 11.
• Factor 2 (Instructor Support): Eigenvalue of 1.15, explaining 14.3% of the variance. This factor primarily loaded on Items 4, 5, and 8.

The two factors exhibited a moderate correlation of .58 (p < .01). While the two-factor solution is supported by the item content, researchers noted that the relatively low factor loadings suggest that future re-analysis with larger datasets may lead to a modified factor structure.

Instrument

Test Type: Academic Evaluation Module/Questionnaire

Format: Online, web-based survey module, administered without separate instruction following the core evaluation module.

Language Available: German (deutsch)

Population Group: University Students (Higher Education)

Age Group: 18 to 40 years (M = 22.4; SD = 3.0)

Population Details: The sample analyzed (N=348, collected between Winter Semester 2008/09 and Winter Semester 2009/10) consisted primarily of Psychology students (54.3% main subject), ranging from the first to the 22nd semester (M = 3.0; SD = 2.2).

Test Methodology: The scale uses a 7-point Likert response format for the core scaled items (Items 4-11), ranging from 1 = “stimme gar nicht zu” (strongly disagree) to 7 = “stimme vollkommen zu” (strongly agree). Additional items utilize open response fields or categorical options. Data collection is implemented via an online platform using PHP and MySQL databases, facilitating efficient data collection and instant feedback comparisons for instructors.

Keywords

Psychological Assessment, Higher Education Quality, Student Feedback, Seminar Evaluation, Questionnaire Development, Psychometrics

[quads id=5]

Authors

Author ORCID Identifier: Not provided in source content.

Affiliation Email addresses: [email protected]

Correspondence Address: Ina Grötemeier, Westfälische Wilhelms-Universität Münster, Institut für Psychologie, Fliednerstr. 21, 48149 Münster; Dr. Dipl.-Psych. Meinald Thielsch, Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster.

Permissions & Fee and Test Year

The scale is part of the internal evaluation system of the Department of Psychology at the Westfälische Wilhelms-Universität Münster. The item batteries for the MFE system were developed starting in 2003, with the module for presentations (MFE-ZRe) being included in the same year. A fundamental item analysis and revision of all supplementary modules were conducted in the Summer Semester of 2010. Specific permission and fee information for external usage is not detailed in the source, but the website for the project is http://www.uni-muenster.de/PsyEva.

Reference’s

The development of the MFE modules relies on foundational work in teaching evaluation and web-based assessment.

• Grabbe, S. (2003). Evaluation von Seminaren: Entwicklung eines Fragebogens zur Erfassung der Lehrqualität. Unveröffentlichte Diplomarbeit, Fachbereich Psychologie, Universität Münster.
• Haaser, S., Thielsch, M. T., & Moeck, T. (2007). Webbasierte Lehrevaluation im Fach Psychologie an der Universität Münster. In S. Krolak-Schwerdt, P. H. M. P. L. T. J. H. J. D. (Hrsg.), Empirische Pädagogik 2007: Tagungsband der 15. Tagung der Kommission “Evaluationsforschung und Qualitätssicherung” der DGfE (S. 27–38). Landau: VEP.
• Moeck, T., & Thielsch, M. T. (2004). Der Münsteraner Fragebogen zur Evaluation von Seminaren (MFE-S). ZIS-Dokumentationssystem, 13(2).
• Rindermann, H. (1996). Evaluation von Lehre und Studium: Theorie, Methoden, Instrumente, Ergebnisse. Landau: Verlag Empirische Pädagogik.
• Schmidt, J., & Loßnitzer, M. (2010). Instrumente zur Lehrevaluation an Hochschulen. Zeitschrift für Evaluation, 9(1), 101–126.

[quads id=5]

Items of the additional module presentations – Münster questionnaire for evaluation (MFE-ZRe)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

Items zu Unterstützung durch den Dozenten (Items on Instructor Support)

1. Der/Die Lehrende nahm sich genügend Zeit, um mich bei der Vorbereitung meines Referats zu unterstützen.
2. Der/Die Lehrende gab hilfreiche Anregungen zur inhaltlichen Vorbereitung meines Referats.
3. Der/Die Lehrende ergänzte die Referate sinnvoll mit seinem/ihrem Fachwissen.

Items zu Kompetenzerwerb (Items on Competence Acquisition)

6. Mir war das Gelingen meines Referats wichtig.
7. Der/Die Lehrende gab mir nützliches Feedback zu meinem Referat.
8. Ich habe meine Kompetenz, mich mit wissenschaftlichen Texten auseinanderzusetzen, durch die Referatsvorbereitung verbessert.
9. Ich habe meine rhetorischen Fähigkeiten durch das Halten des Referats verbessert.
10. Der/Die Lehrende gab hilfreiche Anregungen zu Präsentationstechniken (PowerPoint etc.).

Ergänzend vorgegebene, hier nicht weiter dokumentierte Items (Supplementary Items, not further documented here)

1. Wie viele Referate wurden in den Sitzungen gehalten? (Antwortvorgaben: Offenes Antwortfeld.)
2. Wie lang waren die Referate durchschnittlich? (Antwortvorgaben: Offenes Antwortfeld.)
3. Wie beurteilst du die Länge der Referate? (Antwortvorgaben: “viel zu lang”, “zu lang”, “angemessen”, “zu kurz” und “viel zu kurz”.)
4. Bei der Planung meines Referats gab der/die Lehrende Raum für eigene Ideen/Vorschläge. (Antwortvorgaben: Wie Items 4 bis 11.)
5. Im Punktesystem der gymnasialen Oberstufe (0 [ungenügend] bis 15 [sehr gut +]) bewerte ich die Veranstaltung insgesamt mit folgender Punktzahl: ___ (Antwortvorgaben: Offenes Antwortfeld.)