Artificial Intelligence Literacy Questionnaire (AILQ)

Abstract

The Artificial Intelligence Literacy Questionnaire (AILQ), developed by Ng et al. (2024), is a comprehensive, self-report psychometric instrument designed specifically for assessing AI literacy among secondary school students. The questionnaire operationalizes AI literacy based on the Affective, Behavioral, Cognitive, and Ethical (ABCE) framework.

The development process was rigorous, starting with an initial pool of 60 items that underwent expert validation, qualitative interviews, and pilot testing. Following detailed exploratory and confirmatory factor analyses, a refined, reliable, and valid version consisting of 32 items was established. The AILQ provides a multidimensional view of students’ readiness to engage with AI technologies across affective, behavioral, cognitive, and ethical domains.

Keywords

Artificial Intelligence Literacy, Affective Learning, Behavioral Learning, Cognitive Learning, Ethical Learning, Secondary Students, Psychometrics, Educational Measures

Authors

Ng, Davy Tsz Kit; Wu, Wenjie; Leung, Jac Ka Lok; Chiu, Thomas Kin Fung; Chu, Samuel Kai Wah

Purpose

The primary purpose of the AILQ is to provide an accurate and multidimensional assessment of Artificial Intelligence Literacy (AIL) among adolescents in secondary education settings. Recognizing the growing necessity for students to engage responsibly and effectively with AI technologies, the scale aims to quantify specific competencies that extend beyond mere technical knowledge.

This measure serves as a diagnostic tool for educators and researchers to identify strengths and weaknesses across the four core ABCE dimensions. By quantifying aspects such as intrinsic motivation, collaboration skills, knowledge application, and ethical awareness, the AILQ facilitates the design and evaluation of targeted AI education curricula and interventions.

Construct

The AILQ is structured around the integrated Affective, Behavioral, Cognitive, and Ethical (ABCE) model of AI literacy. This model posits that effective engagement with AI requires a holistic set of competencies, which are operationalized into four learning domains:

• Affective Learning: This domain measures emotional and motivational factors, including self-efficacy, confidence in using AI, and intrinsic motivation related to AI concepts and career interest.
• Behavioral Learning: This domain focuses on observable actions and practical engagement, specifically measuring behavioral commitment toward learning AI and skills related to collaboration in AI-related tasks.
• Cognitive Learning: This domain assesses traditional knowledge and intellectual skills, encompassing the ability to apply, evaluate, and create knowledge related to AI systems and principles.
• Ethical Learning: This domain addresses the critical understanding and application of ethical principles concerning AI, including awareness of bias, fairness, and societal impact.

Validity

The validity of the AILQ was established through a rigorous process involving both qualitative (expert review and interviews) and quantitative methods. Initial content validity was confirmed during the refinement process, resulting in the final 32-item structure.

Assessment of Convergent Validity, using the Average Variance Extracted (AVE), showed acceptable results (AVE exceeded 0.50) for most latent factors. While intrinsic motivation (0.44) and AI ethics (0.46) were slightly below the conventional threshold, the overall model fit remained strong. Discriminant Validity was assessed using the Heterotrait–Monotrait ratio of correlations (HTMT), with all values below the acceptable threshold of 0.85, confirming that the four latent factors are empirically distinct constructs.

Reliability

The internal consistency of the AILQ demonstrated high reliability. The overall measure reported a strong Cronbach’s alpha coefficient of 0.93. This high value indicates excellent internal consistency across the 32 items, suggesting that the items reliably measure the underlying construct of Artificial Intelligence Literacy across the target population.

Factor Analysis

Both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were utilized to establish and verify the underlying factor structure of the AILQ. The EFA confirmed the four-factor structure corresponding to the ABCE dimensions, which collectively explained 51.41% of the total variance.

Subsequent Confirmatory Factor Analysis (CFA) was performed to test the structural model. Model comparison favored the second-order model (where the four factors load onto a single higher-order AI literacy construct) over the third-order model. Key fit indices for the favored second-order model were highly acceptable: χ²(452) = 1001.54, p < 0.01; RMSEA = 0.06; CFI = 0.92; TLI = 0.91; and SRMR = 0.06. These findings validate the theoretical structure of the AILQ.

Instrument

Test Type: Self-report inventory/Questionnaire

Format: 32 items measured on a 5-point Likert scale, ranging from ‘strongly disagree’ (1) to ‘strongly agree’ (5).

Language Available: English (Primary publication language)

Population Group: Secondary school students (Adolescents)

Age Group: 12–17 years

Population Details: Human participants; Male and Female respondents primarily sampled in Hong Kong.

Test Methodology: The scale uses a self-report methodology to capture students’ perceptions, attitudes, and reported competencies across the four key dimensions: Affective learning, Behavioral learning, Cognitive learning, and Ethical learning.

Keywords

AI Ethics, Digital Competence, Adolescent Development, Educational Technology, Affective Domain, Psychometric Validation

Authors

Author ORCID Identifier: Ng, Davy Tsz Kit: 0000-0002-2380-7814; Leung, Jac Ka Lok: 0000-0001-6490-7005; Chiu, Thomas Kin Fung: 0000-0003-2887-5477; Chu, Samuel Kai Wah: 0000-0003-1557-2776

Affiliation Email addresses: Affiliations include the University of Hong Kong, Chinese University of Hong Kong, Hong Kong University of Science and Technology, and Hong Kong Metropolitan University.

Correspondence Address: Ng, Davy Tsz Kit – [email protected]

Permissions & Fee and Test Year

The AILQ was published and validated in 2024. Information regarding specific usage fees or formal licensing permissions for the scale itself is not detailed in the original validation study. Researchers interested in using the scale should contact the corresponding author for usage permissions.

Reference’s

Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024). Design and validation of the AI Literacy Questionnaire: The affective, behavioural, cognitive and ethical approach. British Journal of Educational Technology, 55(3), 1082–1104. https://doi.org/10.1111/bjet.13411

Items of the Artificial Intelligence Literacy Questionnaire (AILQ)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

The specific 32 items comprising the finalized AILQ were not provided in the source content. The finalized scale measures the following four key dimensions:

• Affective Learning (Intrinsic motivation, self-efficacy, confidence, career interest)
• Behavioral Learning (Behavioral commitment, collaboration)
• Cognitive Learning (Knowledge application, evaluation, and creation)
• Ethical Learning (AI ethics)