Additional module discussion – Münster questionnaire for evaluation (MFE-ZDi)

Abstract

The additional module discussion (MFE-ZDi) is an 8-item supplementary instrument developed as part of the Münster questionnaire for evaluation (MFE) system at the University of Münster. Designed specifically for online implementation, the MFE-ZDi assesses the quality and effectiveness of discussion phases within university courses, such as seminars and lectures. Its primary goal is to provide specific, high-quality feedback to instructors regarding student engagement, learning gains, and the moderation of discussions. The scale was revised in 2010 to achieve a robust psychometric structure, demonstrating a strong single-factor solution and acceptable internal consistency (Cronbach’s Alpha = 0.79). The development prioritized high economy, ensuring minimal time burden on participating students.

Keywords

Teaching, Evaluation, Discussion, Course Quality, Higher Education, Psychometrics, MFE, Seminar Evaluation

Authors

Dr. Dipl.-Psych. Meinald Thielsch, B. Sc. Ina Stegemöller

Purpose

The primary purpose of the MFE-ZDi is to provide a focused and efficient measure of the quality of discussions within academic teaching settings. As an optional add-on module to the standardized MFE base instruments (MFE-Sr for seminars and MFE-Vr for lectures), it allows instructors to customize the teaching evaluation process to match their specific didactic design.

This modular approach addresses the need for short, economical scales suitable for web-based surveys, particularly crucial when evaluations occur near the end of the semester when students are highly burdened by examination preparation. By assessing discussion quality, the instrument supports the broader goal of quality assurance in higher education, helping to identify strengths and weaknesses in specific course components and fostering better communication between students and faculty.

Construct

The MFE-ZDi measures a specific dimension of the overarching construct of Teaching Evaluation (Lehrevaluation). Specifically, it targets the perceived quality and utility of discussions held during a course. The items collectively assess several facets of effective discussion, including:

• Adequacy and frequency of discussions.
• Student contribution and participation.
• The depth of understanding achieved through discussion.
• The instructor’s ability to moderate effectively and resolve ambiguities.
• The overall productivity and stimulation of critical thinking afforded by the discussions.

Validity

Evidence for the scale’s validity was established through examining convergent and divergent validity by correlating the MFE-ZDi scale mean with other established scales from the MFE base module (MFE-Sr).

• Convergent Validity: Strong positive correlations were observed between the MFE-ZDi and scales logically expected to relate to discussion quality, including the scales for “Dozent & Didaktik” (Instructor & Didactics, r = .65), “Teilnehmer” (Participants, r = .60), and “Materialien” (Materials, r = .58). Furthermore, strong correlations were found with self-assessed learning success (r = .58) and overall course evaluation (r = .65).
• Divergent Validity: A moderate negative correlation was found with the MFE-Sr scale for “Überforderung” (Overload/Overwhelmed, r = -.32). This suggests that well-structured and effective discussions correspond with a reduced sense of being overwhelmed by the material.
• Discriminative Validity: To confirm the module’s ability to differentiate between various courses, an ANOVA was conducted using the rated courses as the independent variable. The results showed significant differences between the courses (F = 2.08; df = 14; p < .01; Eta-Squared = .20), confirming that the MFE-ZDi is sensitive enough to discriminate between the discussion quality levels of different teaching events.

Reliability

The internal consistency of the additional module discussion (MFE-ZDi) was assessed using Cronbach’s Alpha. The resulting coefficient was 0.79. This level of reliability is considered sufficient and appropriate for evaluations conducted at the course level, which is the typical application context for teaching evaluation instruments.

Factor Analysis

The scale underwent a significant revision in the Summer Semester of 2010. Initially, the module consisted of five items, but item analysis revealed a poor factor structure, preventing a clear interpretation. To address this, the module was revised and expanded to its current form of eight items (Items 1, 7, and 8 were added).

Following the revision, an Exploratory Factor Analysis (EFA) using principal axis factoring with an oblique Promax rotation was performed on the collected data (N = 172). The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was 0.84, indicating the suitability of the dataset for factor analysis. Inspection of the Scree plot strongly supported a clear one-factor solution. This single factor had an Eigenvalue of 4.03 and accounted for 50.5% of the total variance. All eight items demonstrated significant loadings on this common factor, ranging from .47 to .90, confirming the scale’s unidimensionality.

Instrument

Test Type: Specific Teaching Evaluation Module (Add-on Scale)

Format: Web-based questionnaire using a 7-point response format.

Language Available: German (deutsch)

Population Group: University Students

Age Group: Adults (Sample range 19 to 39 years)

Population Details: The validation sample (N=172) consisted primarily of Psychology Bachelor students (84.3%) enrolled at the University of Münster between Winter Semester 2010/11 and Summer Semester 2012.

Test Methodology: The instrument is administered online via a platform developed using PHP and MySQL databases. It is presented as a supplementary module immediately following the core MFE base modules (MFE-Sr or MFE-Vr). Scoring involves summing or averaging the responses due to the scale’s confirmed unidimensionality.

Keywords

Psychometrics, Unidimensionality, Discussion Quality, Student Feedback, University of Münster, Online Survey, Didactics, Seminar

Authors

Author ORCID Identifier: N/A (Not provided in source content)

Affiliation Email addresses: [email protected]

Correspondence Address: Dr. Dipl.-Psych. Meinald Thielsch, Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster, Germany.

Permissions & Fee and Test Year

The scale is part of the Münsteraner Fragebogen zur Evaluation (MFE) system developed at the Department of Psychology at the University of Münster. The scale was developed starting in 2003, with the specific “Discussion” module being substantially revised and validated in 2010. Information regarding specific permissions and fees for external use is typically handled through the corresponding authors at the university, whose contact details are provided. The system is maintained and utilized by the University of Münster (Website: http://www.uni-muenster.de/PsyEval).

Reference’s

The development and validation of the MFE modules rely on several academic works, including those listed below, which underscore the theoretical basis of teaching evaluation and the practical advantages of online survey administration:

• Bechler & Thielsch (2012). [Reference not fully detailed in source, related to time pressure in evaluation].
• Göritz, Soucek & Bacher (2005). [Reference not fully detailed in source, related to online survey advantages].
• Grabbe (2003). [Reference related to analysis of characteristics of good teaching].
• Haaser, Thielsch & Moeck (2007). [Reference related to the web-based evaluation platform].
• Marsh (1984). [Reference related to difficulties in validating teaching evaluation instruments].
• Moeck & Thielsch (2004). [Reference related to the initial development of add-on modules].
• Rindermann (1996). [Reference related to the purpose and potential improvements initiated by teaching evaluation].
• Schmidt und Loßnitzer (2010). [Reference related to overview of teaching evaluation instruments in German-speaking countries].
• Thielsch & Weltzin (2012). [Reference not fully detailed in source, related to online survey advantages].

Items of the additional module discussion – Münster questionnaire for evaluation (MFE-ZDi)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

The 7-point response format provides the following options: 1 = “stimme gar nicht zu” (strongly disagree), 2 = “stimme nicht zu” (disagree), 3 = “stimme eher nicht zu” (tend to disagree), 4 = “neutral”, 5 = “stimme eher zu” (tend to agree), 6 = “stimme zu” (agree), and 7 = “stimme vollkommen zu” (strongly agree). An additional option, “nicht sinnvoll beantwortbar” (not meaningfully answerable), is also available.