Additional Module Computer-Aided Teaching – Münster Questionnaire for Evaluation (MFE-ZcL)

Abstract

The additional module for computer-aided teaching, abbreviated as the MFE-ZcL (Münster Questionnaire for Evaluation – Additional Module Computer-Aided Teaching), is a validated scale designed for the efficient evaluation of technology-supported instruction within academic settings. This item battery functions as a supplement to the main MFE modules (MFE-Sr for seminars and MFE-Vr for lectures) utilized at the University of Münster. Developed specifically for rapid, web-based administration, the MFE-ZcL focuses on assessing students’ perception of the effectiveness, clarity, and personal skill development related to computer-assisted learning methods. The scale consists of six items in total, with five core items forming a unidimensional construct used for calculating overall scores. Psychometric analysis indicates good internal consistency, with a Cronbach’s Alpha of 0.84.

Keywords

Teaching Evaluation, Computer-Aided Teaching, MFE-ZcL, Instructional Quality, Computer-Assisted Learning, University of Münster, Factor Analysis, Psychometrics

Authors

Dr. Dipl.-Psych. Meinald Thielsch, B. Sc. Ina Stegemöller

Purpose

The primary purpose of the MFE-ZcL module is to enable specialized and context-specific teaching evaluation, particularly focusing on courses that incorporate computer-aided instruction. As part of a larger system for quality assurance in higher education, this module allows instructors to tailor the evaluation process to their specific pedagogical methods without overburdening students with irrelevant questions. The scale aims to provide actionable feedback concerning the clarity of instruction, opportunities for practical application, and students’ perceived competence gained through the use of technology in the seminar setting.

The development of the MFE supplementary modules, including MFE-ZcL, was driven by the need for short, economical scales suitable for online administration, which is crucial when evaluating diverse courses simultaneously near the end of the academic semester when students’ time is limited due to exam preparation.

Construct

The MFE-ZcL measures the construct of Computer-Aided Teaching Quality and its effectiveness, specifically within the context of university seminars. The scale assesses student experience across several dimensions related to technology integration, including the adequacy of prior computer knowledge, the clarity of the instructor’s explanation of computer tasks, the provision of practice opportunities, and the students’ self-assessed ability to independently apply the learned skills. The scale is designed to be unidimensional, allowing for the calculation of a single overall score reflecting the quality and effectiveness of the computer-assisted component of the course.

Validity

Validating instruments for teaching evaluation is inherently challenging due to the multitude of factors influencing learning success (Marsh, 1984). To address this, the MFE-ZcL module demonstrated strong evidence for both convergent and divergent validity by examining its correlations with other scales derived from the MFE-Sr basic module (Seminar Evaluation).

The MFE-ZcL showed high positive correlations with other quality indicators, including the scales for “Instructor & Didactics” (.79), “Participants” (.69), and “Materials” (.74). Further high correlations were observed with self-assessed learning success (.81) and overall course assessment (.76). Crucially, a high negative correlation with the MFE-Sr scale “Overburdening” (-.57) supported the notion that effective computer-aided teaching challenges students appropriately without causing excessive strain. Furthermore, discriminative validity was confirmed using an ANOVA, which demonstrated significant differences between the mean module scores across evaluated courses (F = 12.75, df = 6, p < .01, Eta-Squared = .52), indicating the module’s sensitivity to variations in teaching quality.

Reliability

The internal consistency of the MFE-ZcL module, based on the five core items (Items 2-6), was determined to be good. The calculated Cronbach’s Alpha coefficient is .84. This level of reliability indicates that the module is well-suited for reliable evaluation and analysis at the level of individual course events, which is the typical application context for teaching evaluations.

Factor Analysis

An item analysis and revision were conducted in the summer semester of 2010. Following this revision, an exploratory factor analysis (Principal Axis Factoring with oblique Promax rotation) was performed on the collected data (N=77). The Kaiser-Meyer-Olkin (KMO) test yielded an MSA value of .80, confirming the suitability of the data set for factor analysis. Both the Kaiser criterion (Eigenvalues > 1) and the inspection of the scree plot clearly supported a one-factor solution.

This single factor accounts for 52.7% of the total variance, with an eigenvalue of 2.64. All five core items (2-6) were strongly associated with this common factor, with factor loadings (FL) ranging from .57 to .80. This analysis confirms the intended unidimensionality of the scale, allowing for the aggregation and averaging of the response values for items 2 through 6.

Instrument

Test Type: Supplemental Item Battery for Teaching Evaluation

Format: Web-based questionnaire using a 7-point Likert scale format for the core items, and a categorical response format for the supplementary item.

Language Available: German (Documentation and Items)

Population Group: University Students

Age Group: Adults (Sample range: 19 to 44 years)

Population Details: The validation sample (N=77) was primarily composed of students (96.1%) enrolled in the Bachelor of Science program in Psychology at the University of Münster. Data was collected between Winter Semester 2010/11 and Winter Semester 2011/12.

Test Methodology: The scale is administered online via a web-based platform developed using PHP and MySQL databases. It is typically presented as an optional add-on module immediately following the administration of the basic evaluation modules (MFE-Sr/MFE-Vr). The core items (2-6) use a 7-step response format ranging from 1 = “stimme gar nicht zu” (strongly disagree) to 7 = “stimme vollkommen zu” (strongly agree), plus the option “nicht sinnvoll beantwortbar” (not meaningfully answerable).

Keywords

Lehrevaluation, Computer-assisted teaching, MFE-ZcL, Quality Assurance, Internal Consistency, Psychometrics, Higher Education

Authors

Author ORCID Identifier: Not provided in source content.

Affiliation Email addresses: [email protected]

Correspondence Address: Dr. Dipl.-Psych. Meinald Thielsch, Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster, Germany. (Website: www.uni-muenster.de/PsyEval)

Permissions & Fee and Test Year

Information regarding specific permissions and fees is not explicitly detailed in the source content, though the instrument is associated with the evaluation system of the Westfälische Wilhelms-Universität Münster. The scale construction was based on work dating back to 2003, with the version documented here being revised in Summer 2010 and validated using data from the 2010/11 and 2011/12 winter semesters.

Reference’s

• Bechler, S. & Thielsch, M. T. (2012). Evaluation der Lehrevaluation. In M. T. Thielsch (Hrsg.), Lehrevaluation in der Psychologie (S. 119-140). Lengerich: Pabst.
• Göritz, A. S., Soucek, R. & Bacher, J. (2005). Online-Befragung in der Lehrevaluation. Zeitschrift für Evaluation, 4(1), 81-98.
• Grabbe, S. (2003). Entwicklung eines Fragebogens zur Evaluation von Seminaren am Psychologischen Institut der Westfälischen Wilhelms-Universität Münster. Unveröffentlichte Diplomarbeit, Westfälische Wilhelms-Universität Münster.
• Haaser, T., Thielsch, M. T. & Moeck, P. (2007). Der Münsteraner Fragebogen zur Evaluation (MFE). In M. T. Thielsch (Hrsg.), Lehrevaluation in der Psychologie (S. 141-160). Lengerich: Pabst.
• Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. Journal of Educational Psychology, 76(5), 707-754.
• Moeck, P. & Thielsch, M. T. (2004). Der Münsteraner Fragebogen zur Evaluation (MFE). In K. D. Kubinger & A. W. Schick (Hrsg.), Psychologische Diagnostik in Österreich. Beiträge zum 5. Kongress der Österreichischen Gesellschaft für Psychologie (S. 273-280). Wien: Facultas.
• Rindermann, H. (1996). Evaluation von Lehre und Studium. Göttingen: Hogrefe.
• Schmidt, U. & Loßnitzer, K. (2010). Lehrevaluation an Hochschulen. In H. Wottawa & E. T. C. Staudt (Hrsg.), Handbuch Evaluation. Grundlagen, Ziele und Verfahren (S. 327-336). Göttingen: Hogrefe.
• Thielsch, M. T. & Weltzin, J. (2012). Ergebnisse der Lehrevaluation. In M. T. Thielsch (Hrsg.), Lehrevaluation in der Psychologie (S. 161-182). Lengerich: Pabst.

Items of the additional module computer-aided teaching – Münster questionnaire for evaluation (MFE-ZcL)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

The following is a supplementary item with categorical response options: