Digital Competence for Learning Assessment Test (Digitest)

Abstract

The Digital Competence for Learning Assessment Test (Digitest; Pedaste, Kallas, & Baucal, 2023) is a specialized 41-item instrument designed to evaluate the digital competence of primary and lower secondary school students, specifically within the context of learning. The scale content was meticulously developed by an expert group from the Centre for Educational Technology at the University of Tartu, ensuring coverage of key dimensions outlined in relevant theoretical frameworks (Pedaste et al., 2021; Adov, 2022).

The initial validation study involved an Estonian sample of students spanning the third through ninth grades. Comprehensive psychometric evaluation included Factor Analysis, Item Response Theory (IRT) analysis, and detailed reliability assessments, all of which supported the robust structure and psychometric quality of the resulting nine subscales.

Keywords

Digital Competence for Learning, Students, Perceived Control, Behavioral Intention, Behavior-Related Attitudes, Digital Operations, Communication in the Digital World, Digital Content Programming, Creation of Digital Materials, Protection of Oneself and Others in the Digital World, Legal Behavior in the Digital World.

Authors

Pedaste, Margus, Kallas, Külli, Baucal, Aleksandar.

[quads id=5]

Purpose

The primary purpose of the Digitest is to provide a standardized, psychometrically sound method for assessing students’ level of digital competence for learning. This assessment is specifically tailored for use with students in primary and lower secondary education settings, covering a broad age range from childhood through early adolescence.

By measuring multiple dimensions of digital skills and attitudes, the test aims to help educators and researchers identify strengths and weaknesses in students’ capacity to utilize digital tools effectively for educational purposes, thereby informing curriculum development and pedagogical strategies.

Construct

The Digitest measures digital competence as a complex, multi-faceted construct rooted in Weinert’s (2001) theory. This theoretical framework posits that competence is composed of both cognitive-behavioral and motivational elements, necessitating a higher-order factor structure.

The final instrument utilizes a 2-factor higher-order model, encompassing nine distinct subscales that cover both motivational components (e.g., Perceived Control, Behavioral Intention) and specific skill domains (e.g., Digital Content Programming, Legal Behavior in the Digital World).

Validity

Validity was investigated through the application of Item Response Theory (IRT). The IRT analysis showed that while item infit and outfit scores occasionally fell outside the expected range, these deviations were not considered significant issues, provided other quality indicators remained acceptable.

Key indicators supported the scale’s validity: the expected correlation exceeded the suggested value of 0.3 for almost all items (with two exceptions above the acceptable 0.2 level), and estimated discrimination was satisfactory, with only two items (COMM1 and COMM2) falling outside the suggested range. Ultimately, the IRT analysis confirmed that there were no strong reasons to revise the test by excluding any items, supporting the measurement quality of the instrument.

Reliability

The reliability of the Digitest was established through measures of Internal Consistency. Composite reliabilities were calculated for the nine factors that constituted the final 2-factor second-order factor model.

The composite reliability scores for the different latent variables demonstrated a strong range, varying from 0.65 to 0.91. This indicates that the subscales possess adequate to high internal consistency, confirming that the scales reliably measure their intended components of digital competence.

Factor Analysis

Scale refinement utilized both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA).

EFA was initially conducted to understand the factor structure of the attitude items. Results from both three- and four-factor models indicated that items related to social aspects of digital use did not load cleanly onto a single factor. Based on the initial 10-factor model factor loadings, EFA results, and composite reliability calculations, the attitude dimension concerning social aspects was subsequently removed from the model.

CFA revealed that the 9-factor model, along with the 2-factor and 3-factor higher-order factor models, all provided a similarly good fit to the empirical data. However, consistent with the theoretical definition of competence based on Weinert’s framework (which separates cognitive-behavioral and motivational elements), the 2-factor higher-order model was selected as the final structure, providing the best conceptual fit with the theoretical underpinnings of the scale.

Instrument

Test Type: Original Test

Format: The measure consists of 41 items utilizing six different response formats: (1) multiple-choice questions with only one correct option, (2) multiple-choice questions with more than one correct option, (3) tasks requiring the formation of a sequence of phases, (4) matching items, (5) tasks involving marking something in a picture, and (6) open-ended questions. All items are scored dichotomously (correct or incorrect).

Language Available: Estonian

Population Group: Human, Male, Female

Age Group: Childhood (birth–12 yrs), School Age (6–12 yrs), Adolescence (13–17 yrs)

Population Details: Respondents were third to ninth grade students in Estonia.

Test Methodology: Test Reliability, Internal Consistency, Factor Analysis, Confirmatory Factor Analysis, Exploratory Factor Analysis, Item Response Theory.

Keywords

Digital Competence, Educational Technology, Psychometric Testing, Student Assessment, Estonia, Primary Education, Secondary Education.

[quads id=5]

Authors

Author ORCID Identifier:

• Pedaste, Margus: http://orcid.org/0000-0002-5087-9637

• Baucal, Aleksandar: http://orcid.org/0000-0002-7965-7659

Affiliation Email addresses:

• Pedaste, Margus (University of Tartu, Institute of Education): [email protected]

• Kallas, Külli (University of Tartu, Institute of Education): [email protected]

• Baucal, Aleksandar (University of Belgrade, Department of Psychology): [email protected]

Correspondence Address:

Kallas, Külli: University of Tartu, Institute of Education, Jakobi 5, Tartu, Estonia, 51005, [email protected]

Permissions & Fee and Test Year

Permissions: Contact Corresponding Author

Commercial: No

Fee: No

Test Year: 2023

References

Pedaste, M., Kallas, K., & Baucal, A. (2023). Digital competence test for learning in schools: Development of items and scales. Computers & Education, 203, 1–19. https://doi.org/10.1016/j.compedu.2023.104830

[quads id=5]

Items of the Digital Competence for Learning Assessment Test (Digitest)

Subscales: Perceived control; Behavior-related attitudes; Behavioral intention; Creation of digital materials; Digital content programming; Communication in the digital world; Digital operations; Legal behavior in the digital world; Protection of oneself and others in the digital world.

Number of items: This measure consists of 41 items.

Test Items Available: No data is Available