Münster questionnaire for the evaluation of tutorials (MFE-T)

Abstract

The Münster Questionnaire for the Evaluation of Tutorials (MFE-T) is a specialized item battery designed as an additional module for the overarching Münster Questionnaire for Evaluation (MFE). Its primary function is to assess the quality, utility, and perceived necessity of tutorial services provided in university settings, which are often supplementary to core curricular lectures and seminars. The MFE-T provides a short, efficient instrument for general tutorial evaluation, focusing on student perceptions rather than specific behavioral criteria, making it adaptable across diverse tutorial formats.

Developed and continuously validated since 2008 at the University of Münster, the scale consists of 9 items, including an initial filter question and two core subscales measuring perceived Support and Need. The instrument is designed to aid in institutional Teaching Evaluation and inform the allocation of university resources, particularly those funded by tuition fees.

Keywords

Teaching evaluation, tutorials, higher education, quality assurance, student perception, psychometrics, peer learning, resource management, Münster Questionnaire for Evaluation.

Authors

Dr. Dipl.-Psych. Meinald Thielsch, Tina Dusend, Ina Grötemeier

Purpose

The main purpose of the MFE-T is to provide a standardized, yet flexible, tool for the comprehensive evaluation of student-led tutorials within university courses. Historically, standard teaching evaluation instruments primarily focus on lectures and seminars, neglecting the supplementary role of tutorials.

The instrument serves several critical functions: firstly, ensuring the quality assurance of tutorial offerings; and secondly, assisting administrators in the strategic distribution of university resources, especially funds derived from tuition fees designated for teaching support. Due to the inherent diversity of tutorial formats (e.g., integrated vs. parallel support), the MFE-T is deliberately brief and focuses on general impressions of usefulness and necessity, rather than highly specific behavioral assessments.

Construct

The MFE-T measures the construct of Tutorial Evaluation, conceptualized across two distinct, yet correlated, subscales: Support and Need.

• Support: This subscale assesses the perceived helpfulness and availability of the tutors and the tutorial itself, focusing on whether the support adequately facilitated student learning and problem-solving, and whether coordination between the lecturer and the tutor was effective.
• Need (Bedarf): This subscale evaluates the students’ perception of the necessity of the tutorial for mastering the course material and their willingness to support its continued funding (e.g., through tuition fees).

Validity

The validation of the MFE-T focused on demonstrating both convergent and divergent validity, acknowledging the challenge of validating teaching evaluation instruments due to the lack of universally accepted criteria for “good teaching” (Marsh, 1984).

• Convergent Validity: Evidence for convergent validity is provided by the high correlation between the sum scores of the two subscales—Support (r = .66) and Need (r = .62)—with the students’ overall judgment of the tutorials.
• Divergent Validity: The scale demonstrated sensitivity to different forms of tutorial delivery. The overall evaluation of the associated main lecture or seminar was significantly correlated with the MFE-T subscale scores only when the tutorial was integrated into the primary course (Support: r = .51; Need: r = .38). This correlation was not significant when the tutorial was offered parallel to the main course. This sensitivity to the structural context of the tutorial serves as evidence for the instrument’s divergent validity.

Reliability

The internal consistency of the MFE-T subscales was assessed using Cronbach’s Alpha. Given that each subscale contains only three items, the obtained values are considered sufficient to good.

• The subscale Support achieved a Cronbach’s Alpha of .70.
• The subscale Need achieved a Cronbach’s Alpha of .83.

These reliability findings are comparable to those reported for more extensive teaching evaluation instruments, such as FEVOR (Staufenbiel, 2000) or HILVE (Rindermann, 2009).

Factor Analysis

An Exploratory Factor Analysis (EFA) utilizing Principal Axis Factoring with oblique Promax rotation was conducted on the core items (Items 2–7, N = 374). Both the Scree Plot and Parallel Analysis clearly supported a two-factor solution, confirming the intended subscale structure.

• Factor 1 (Support): Associated primarily with Items 2 through 4. This factor had an eigenvalue of 2.9 and accounted for 48.2% of the total variance.
• Factor 2 (Need): Associated primarily with Items 5 through 7. This factor had an eigenvalue of 1.2 and explained 20.3% of the total variance.

The two factors correlated at r = .44, indicating they are related but distinct dimensions. Importantly, neither factor exhibited cross-loadings greater than .30 on the items of the opposing factor, confirming the clean separation of the Support and Need constructs.

Instrument

Test Type: Questionnaire (Self-report item battery)

Format: Online administration using a 7-point Likert scale, supplemented by a numeric rating item (0–15 points) and an open-ended text field.

Language Available: German (Items and Documentation)

Population Group: University Students

Age Group: 18 to 41 years old (M = 22.82, SD = 3.40)

Population Details: The validation sample (N = 376) consisted primarily of psychology students (65.2% Bachelor, 25.8% Diploma, rest as minors) at the University of Münster during the Winter Semester 2009/10. The majority of respondents were female (83.2%).

Test Methodology: The MFE-T is typically implemented online immediately following the core evaluation modules for the accompanying lecture or seminar. It begins with a filter question to ensure that only students who utilized the tutorial service proceed to the core evaluation items (Items 2–7). Responses utilize a 7-point scale ranging from 1 (“stimme gar nicht zu” / strongly disagree) to 7 (“stimme vollkommen zu” / strongly agree), along with a “not reasonably answerable” option. Scoring involves summing or averaging the scores on the two subscales (Support and Need).

Keywords

Tutorial quality, student support, higher education evaluation, psychometric scale, MFE-T, Westfälische Wilhelms-Universität Münster.

Authors

Author ORCID Identifier: Not provided in source content.

Affiliation Email addresses: [email protected] (Dr. Meinald Thielsch)

Correspondence Address: Dr. Dipl.-Psych. Meinald Thielsch, Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster, Germany.

Permissions & Fee and Test Year

Test Year: Development began in 2008; validation data reported from Winter Semester 2009/10.

Permissions and Fees: Information regarding formal permissions or licensing fees is not explicitly documented in the source material. Users interested in implementation should contact the corresponding authors at the University of Münster.

Reference’s

• Gediga, G., Schöttler, T., & Deller, J. (2000). KIEL: Kieler Fragebogen zur Lehrevaluation. Hogrefe.
• Gollwitzer, M., & Schlotz, W. (2003). TRIL: Trierer Fragebogen zur Evaluation von Lehrveranstaltungen. Psychologisches Institut, Universität Trier.
• Haaser, J., Thielsch, M. T., & Hirschfeld, G. (2007). Online-Evaluation von Lehrveranstaltungen am Fachbereich Psychologie der Westfälischen Wilhelms-Universität Münster. Zeitschrift für Hochschulentwicklung, 2(4), 1-13.
• Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76(5), 707-754.
• Rindermann, H. (2009). HILVE: Hallescher Inventar zur Lehrevaluation. Göttingen: Hogrefe.
• Roscoe, R. D., & Chi, M. T. H. (2007). Thinking as a tutor: Applying a model of instructional interaction to analyze student tutoring. In D. McNamara & G. T. O’Reilly (Eds.), Proceedings of the 29th Annual Conference of the Cognitive Science Society (pp. 1475–1480). Cognitive Science Society.
• Roscoe, R. D., & Chi, M. T. H. (2008). Tutor competence and student learning outcomes in peer tutoring: A comparative study. Instructional Science, 36(4), 323-350.
• Schmidt, M., & Loßnitzer, T. (2010). Lehrevaluation an Hochschulen: Ein Überblick. Psychologie in Erziehung und Unterricht, 57(2), 81-94.
• Souvignier, E., & Gold, A. (2002). Lehrveranstaltungsevaluation: Ein integratives Konzept. Hogrefe.
• Staufenbiel, T. (2000). FEVOR: Fragebogen zur Evaluation von Vorlesungen. Hogrefe.
• Thielsch, M. T., & Hirschfeld, G. (under review). Systematische Evaluation von Tutorien in der Hochschullehre.
• Thielsch, M. T., Dusend, T., & Grötemeier, I. (2010). Münsteraner Fragebogen zur Evaluation von Tutorien (MFE-T). In Leibniz-Zentrum für Psychologie Information und Dokumentation (ZPID) (Ed.), Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS).

Items of the Münster questionnaire for the evaluation of tutorials (MFE-T)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

Instruktion

Die hier dokumentierte Itembatterie zur Bewertung von Tutorien ist ein Zusatzmodul des “Münsteraner Fragebogen zur Evaluation” (MFE) und wird ohne eigene Instruktion präsentiert. Allgemeine Informationen zur Befragung erfolgen beim erstmaligen Zugriff auf das System (Verwendung der Daten, technische Voraussetzungen, u. ä.). Dabei werden auch demografische Merkmale der Teilnehmer erfragt. Üblicherweise wird die Itembatterie als Zusatzmodul direkt nach der Darbietung der Basismodule zur Evaluation von Veranstaltungen mit Tutorien vorgegeben.

Items

Anm. Nur wenn die erste oder zweite Option gewählt wird, werden die weiteren Items des MFE-T dargeboten: Bitte bewerte im Folgenden das Tutorium bzw. den Tutor oder die Tutoren dieser Veranstaltung. Wenn es mehr als einen Tutor gab, versuch bitte deren Leistung als Ganzes einzuschätzen. Einzelnes Feedback an bestimmte Tutoren kann am Ende über das Textfeld gegeben werden.

Antwortvorgaben

7-stufiges Antwortformat mit den Optionen 1 = “stimme gar nicht zu”, 2 = “stimme nicht zu”, 3 = “stimme eher nicht zu”, 4 = “neutral”, 5 = “stimme eher zu”, 6 = “stimme zu” und 7 = “stimme vollkommen zu”. Zusätzlich steht die Antwortoption “nicht sinnvoll beantwortbar” zur Verfügung.

Auswertungshinweise

Angesichts der offensichtlichen Eindimensionalität der Items der zwei Subskalen können ihre Antwortwerte aufsummiert oder gemittelt werden.

Mit dem vorliegenden Instrument können allerdings verschiedene in oder für eine Veranstaltung eingesetzte Tutoren nicht getrennt bewertet werden, da die Instruktion darum bittet, bei mehreren Tutoren deren Leistung als Ganzes einzuschätzen. Es spricht jedoch grundsätzlich nichts dagegen, mit dem MFE-T Daten je Tutor zu erheben und personengebunden an die Tutoren zurück zu melden, indem z.B. die einleitende Filterfrage und die Items der Subskala Unterstützung je Tutor/in abgefragt wird.