Classroom Assessment Fairness Inventory (CAFI)

Abstract

The Classroom Assessment Fairness Inventory (CAFI), developed by Rasooli et al. (2023), is a psychometric instrument designed to measure and interpret students’ perceptions concerning classroom assessment fairness. The inventory is structured around five critical domains of assessment practice: groupwork, exams, cheating, grading, and feedback. Rooted in principles derived from social psychology theory, the CAFI presents five distinct scenarios, each supported by a set of actions (totaling 40 actions) that reflect core fairness principles such as equality, equity, and transparency.

Initial validation studies were conducted on a sample of first-year undergraduate students in Canada, providing robust evidence regarding the scale’s factor structure, validity, and reliability. The CAFI aims to capture the nuanced ways students experience assessment practices and identify areas where perceived unfairness may impede learning and motivation, serving as a diagnostic tool for educators and researchers.

Keywords

Classroom Assessment Fairness, Student Perceptions, Educational Measures, Fairness in Grading, Fairness in Feedback, Unfairness in Groupwork, Cheating, Student Attitudes, Teacher Student Interaction, Psychometric Instrument.

Authors

Amirhossein Rasooli, Christopher DeLuca, Liying Cheng, Amin Mousavi.

Purpose

The primary purpose of the CAFI is to systematically assess and quantify students’ perceptions of fairness across various assessment practices implemented within a classroom setting, providing educators and researchers with a diagnostic tool to understand the student experience of institutional justice.

Construct

The CAFI measures the overarching construct of Perceived Classroom Assessment Fairness, which is operationalized through five distinct domain-specific scenarios: Groupwork, Exams, Cheating, Grading, and Feedback. These domains reflect specific facets of assessment practice where fairness principles (e.g., Voice, Consistency, Respect, Justification, Bias Suppression) become salient to students.

The inventory is not a unidimensional measure; rather, it identifies factors related both to the presence of fairness (e.g., Fairness in Grading) and the perception of unfairness (e.g., Unfairness in Groupwork, Unfairness in Feedback), suggesting a complex, multi-faceted construct derived from organizational and social justice theories applied to the educational context.

Validity

The validity of the CAFI was empirically investigated using multivariate regression analysis, examining the relationship between CAFI subscales and the students’ personal belief in a just world. Key results provided partial empirical evidence supporting the scale’s validity.

Specifically, a personal belief in a just world was found to positively predict perceived fairness in grading (β = 0.262, p < 0.05) and fairness in feedback (β = 0.294, p < 0.05). Conversely, this belief negatively predicted perceived unfairness in feedback (β = −0.19, p < 0.05). These correlations suggest that students who generally believe the world is fair are more likely to perceive specific assessment actions, particularly those involving grading and feedback processes, as fair.

Reliability

The CAFI demonstrated acceptable levels of both interrater reliability and internal consistency across its constructs, reinforcing its psychometric soundness.

• Interrater Reliability: The overall inter-rater agreement showed a moderate average value of 0.56 (p < 0.001).

• Internal Consistency: Cronbach’s alpha values for the various constructs ranged from 0.79 to 0.87, which is indicative of good internal consistency among the items within each factor.

Factor Analysis

Both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were employed during the scale development and validation process to determine the optimal structure of the CAFI. The analyses ultimately supported a robust five-factor model.

The five-factor model demonstrated a satisfactory fit with the collected data, consistent with the EFA results. Key goodness-of-fit indices included RMSEA = 0.06 (90% CI [0.05–0.07]), CFI = 0.91, and TLI = 0.89. Although a six-factor model also showed acceptable fit (RMSEA = 0.07, 90% CI [00.06–0.08], CFI = 0.90, TLI = 0.87), it was disregarded because the additional factor retained only two items (S2–6 and S2–7) from the Exam scenario, which were deemed conceptually insufficient to represent a distinct underlying dimension, favoring the more parsimonious five-factor structure.

Instrument

Test Type: Original psychometric inventory

Format: Students are instructed to read each scenario and then evaluate the extent to which they perceive each associated action as fair, using a 6-point Likert scale (1 = Highly unfair; 6 = Highly fair). An additional option, “7 = ‘I am not sure’,” is also provided for flexibility in response.

Language Available: English

Population Group: Human (Male, Female, Does Not Identify with Gender Binary)

Age Group: Spans multiple age ranges relevant to secondary and post-secondary education: Childhood (birth-12 yrs), School Age (6-12 yrs), Adolescence (13-17 yrs), Adulthood (18 yrs & older), Young Adulthood (18-29 yrs).

Population Details: The initial validation sample consisted of university students aged 15-21 years and older, located in Canada, primarily targeting first-year undergraduates.

Test Methodology: Test Validity; Test Reliability; Internal Consistency; Interrater Reliability; Factor Analysis; Confirmatory Factor Analysis; Exploratory Factor Analysis.

Keywords

Classroom Assessment Fairness, Student Perceptions, Unfairness in Groupwork, Fairness in Cheating, Fairness in Grading, Educational Psychology, Institutional Justice, Psychometrics.

Authors

Amirhossein Rasooli

• Author ORCID Identifier: No data is Available

• Affiliation: University of Alberta Killam Postdoctoral Fellow

• Email Address: [email protected]

• Correspondence Address: University of Alberta, Faculty of Education, 11210 87 Ave NW, Edmonton, Alberta, Canada, T6G 2G5, [email protected]

Christopher DeLuca

• Author ORCID Identifier: No data is Available

• Affiliation: Queen’s University Faculty of Education

• Email Address: No data is Available

Liying Cheng

• Author ORCID Identifier: http://orcid.org/0000-0002-4458-5085

• Affiliation: Queen’s University Faculty of Education

• Email Address: No data is Available

Amin Mousavi

• Author ORCID Identifier: http://orcid.org/0000-0002-6920-2319

• Affiliation: University of Saskatchewan Faculty of Education

• Email Address: No data is Available

Permissions & Fee and Test Year

Permissions: Contact Publisher

Commercial: No

Fee: No

Test Year: 2023

Reference’s

• Rasooli, A., DeLuca, C., Cheng, L., & Mousavi, A. (2023). Classroom assessment fairness inventory: A new instrument to support perceived fairness in classroom assessment. Assessment in Education: Principles, Policy & Practice, 30(5-6), 372–395. https://doi.org/10.1080/0969594X.2023.2255936

Items of the Classroom Assessment Fairness Inventory (CAFI)

The CAFI is a 21-item measure. The specific items are designed within five scenarios, each addressing a different classroom assessment domain. The factors and subscales identified from the measure are:

• Unfairness in Groupwork

• Fairness in Cheating

• Fairness in Grading

• Unfairness in Feedback

• Fairness in Feedback

Specific item details are available in the source reference: Rasooli, A., DeLuca, C., Cheng, L., & Mousavi, A. (2023). Classroom assessment fairness inventory: A new instrument to support perceived fairness in classroom assessment. Assessment in Education: Principles, Policy & Practice, 30(5-6), 372–395. Specific items can be found in Table 3, Page 379; Table 4, Page 379; Table 5, Page 380; Table 6, Page 380; Table 7, Page 381.

Classroom Assessment Fairness Inventory (CAFI)

Scenario 1: Groupwork

Mr. Chu implemented group work based on initial assessments, forming groups with students of varying ability levels: struggling, average, and high-performing. He expected high-performing students to contribute to the learning of other group members. Each group worked on projects and prepared a final presentation. Mr. Chu then held a class discussion on workload distribution and allowed students to discuss group dynamics. In response to students’ questions about assessment, he explained that grades would be based on project participation and a rubric, not solely on individual performance. He encouraged group members to collaborate, stating that all members would receive the same grades as a reflection of group performance and cooperation. Several students were not satisfied with their grades and appealed, but Mr. Chu did not accept their complaints.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 2: Exam

Mr. Ahmed announced that the class would have an exam the day before winter break (in 5 days). Students preferred moving the exam date because they had many assignments for other subjects that were also due on the same date. Even so, Mr. Ahmed was firm on his decision as moving the exam date back would create more intensive workload later in the year. Mr. Ahmed did not explicitly state what would be on the exam. However, he did include a mix of easy and difficult questions to give all students an opportunity to show their learning. He also provided accommodations (e.g. more time) to students with disabilities and English language learners. In general, Mr. Ahmed is a lenient teacher in grading compared with other teachers in the school who teach the same subject. On the exam, all students complained about two questions that were not covered during the course. Mr. Ahmed harshly responded that students should be able to answer the two questions from what had been taught. For a few students who missed the exam date, Mr. Ahmed decided to give another exam opportunity after winter break.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 3: Cheating

Ms. Johnston is very strict when she catches a student cheating. However, she did not tell students her policy on cheating at the beginning of the year. One student was caught cheating on an exam and Ms. Johnston decided to give the student a grade of zero. Ms. Johnston did not give the student an opportunity to explain the reasons for cheating before making her decisions. She explained to the class that cheating is unfair to other students and asked the student to leave the classroom. The exam constituted 20% of students’ final grade. After the exam, she met with the student and explained that cheating is ethically wrong, is unfair in relation to the classmates, and she would punish anyone who cheats. The student provided a reason for their behavior and apologized.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 4: Grading

Ms. Mendes had students from diverse backgrounds in her classroom. She treated all her students respectfully during classroom assessment, teaching, and interactions. Ms. Mendes informed students that she would give grades based on student achievement. 70% of students’ grades were from multiple tests during the course plus 30% for individual essays. Ms. Mendes communicated test results in one week after handing the test in. Due to busy schedule, she would sometimes allow students to appeal their grades if there was enough time in class. She would fully explain her grading for students who spoke up looking for their grade adjustments. At the end of the course, Ms. Mendes adjusted the grades of failing students with at-risk backgrounds to support their success. She also increased marks for a few students to ensure admission into their desired universities. However, she lowered the grades of a few disruptive students who interrupted the classroom learning.

To what extent do you consider each of these actions to be fair based on the above scenario?

Scenario 5: Feedback

Mr. Dembe asked students to write an essay about their science lab project. He has shared with students a clear rubric that he will use to assess students’ essays. Four days after the essay submission deadline, Mr. Dembe got back to students with his feedback and expressed that students can contact him for further discussion of his feedback. Students noticed that Mr. Dembe gave more feedback to students with good quality essays as well as his favorite students than students who had handed in essays that were of low quality. Mr. Dembe harshly explained that he had given variable feedback on the essays based on the amount of effort he deemed each student had put in completing the essays.

To what extent do you consider each of these actions to be fair based on the above scenario?

Note: Students are asked to read each scenario and identify the extent to which they consider each action to be fair (1 = Highly unfair; 6 = Highly fair). An additional option (7 = ‘I am not sure’) is also provided.