AI Risks and Benefits Scale

Abstract

The AI Risks and Benefits Scale (AIRBS) is a specialized psychometric instrument designed to quantify individuals’ perceptions regarding the potential risks and advantages associated with the integration of Artificial Intelligence (AI) in healthcare settings. Developed using 19 items derived from extensive previous research and empirical testing, the scale was primarily utilized in research examining patient preference dynamics, specifically contrasting choices for human physicians versus treatments suggested by AI systems.

The scale’s development included rigorous testing of its underlying structural integrity and internal consistency to ensure reliable measurement of these complex risk-benefit perceptions. It serves as a crucial tool for researchers investigating patient acceptance, ethical concerns, and the role of trust in the adoption of emerging medical technologies.

Keywords

AI Risks and Benefits Scale, AI in healthcare, patient perception, medical ethics, technology acceptance, risk perception, benefit perception, Likert scale, trust.

Authors

Kerstan, Sophie, Bienefeld, Nadine, Grote, Gudela.

Purpose

The primary purpose of the AIRBS is to systematically measure how individuals perceive the likelihood of various potential risks and benefits arising from the application of AI in clinical and healthcare contexts. This measurement is critical for understanding the psychological barriers and facilitators influencing the public acceptance of AI-driven medical tools and diagnoses.

Specifically, the scale was instrumental in a study investigating the phenomenon of favoring human doctors over AI-suggested treatment plans. By quantifying perceived risks (e.g., dehumanization, data breaches) and perceived benefits (e.g., error reduction, personalized care), the scale provides quantifiable data on the underlying cognitive associations that drive comparative trust and decision-making in medical scenarios.

Construct

The AIRBS measures the psychological construct of Perceived Risk and Benefit of AI in Healthcare. This construct is multidimensional, encompassing two primary domains: Perceived Risks (10 items) and Perceived Benefits (9 items). The scale captures the subjective probability assigned by respondents to the occurrence of specific positive and negative outcomes resulting from the implementation of AI technology in medical settings.

The items cover critical areas reflecting public concern and hope regarding AI implementation, ranging from operational issues (e.g., equipment failures, data inaccuracy) to societal and ethical challenges (e.g., job loss, dehumanization, ethical problems).

Validity

The source documentation indicates that the scale’s creation was grounded in previous research and underwent initial testing for its structure. This suggests that steps were taken to establish content validity, ensuring the items comprehensively cover the intended domains of AI risks and benefits relevant to healthcare.

While specific coefficients for construct validity (e.g., convergent or discriminant validity) are not detailed in the abstract, the foundational testing of the scale structure implies that the researchers verified that the items group together logically, likely separating into distinct Risk and Benefit factors, thus supporting the theoretical construct.

Reliability

The scale was explicitly tested for its reliability, indicating that assessments were conducted to confirm the instrument’s internal consistency. Reliability testing ensures that the 19 items consistently measure the same underlying dimensions of risk and benefit perceptions across different administrations or subsets of items.

Typical reliability measures, such as Cronbach’s alpha, would have been calculated for the overall scale and its sub-dimensions (Risks and Benefits) to demonstrate acceptable levels of internal consistency, although these statistical results are not present in the provided source summary.

Factor Analysis

The development process included testing the scale’s structure, which strongly suggests that Factor Analysis (likely Exploratory Factor Analysis or Confirmatory Factor Analysis) was performed. Given the clear separation into 10 risk items and 9 benefit items, the structure likely supports a two-factor model corresponding precisely to Perceived Risks and Perceived Benefits of AI in healthcare.

This analysis would have confirmed that the items load significantly onto their intended factors, ensuring that the scale accurately captures two distinct, albeit potentially correlated, dimensions of perception regarding AI adoption.

Instrument

Test Type: Psychological scale; Self-report questionnaire.

Format: 19 items rated on a 7-point Likert scale.

Language Available: English (implied by item structure) and potentially German (implied by author affiliation/context of publication).

Population Group: General adult population (potential users/recipients of AI in healthcare).

Age Group: Adults (specific range not provided).

Population Details: Utilized in studies focusing on individuals’ preferences regarding human vs. AI medical providers.

Test Methodology: Respondents rate the likelihood of occurrence for each risk or benefit statement, using a scale where 1 represents very unlikely and 7 represents very likely.

Keywords

AI adoption, patient trust, medical technology, technology risk, healthcare preference, comparative judgment, risk analysis, psychometrics.

Authors

Author ORCID Identifier: Not provided in source material.

Affiliation Email addresses: Not provided in source material.

Correspondence Address: Not provided in source material.

Permissions & Fee and Test Year

Test Year: 2024 (Year of publication of the foundational study).

Permissions & Fee: Specific permissions and fee structures are not detailed in the adapted source. Researchers should refer directly to the publication in Risk Analysis for details regarding scale usage and licensing, as intellectual property rights likely reside with the journal publisher (Wiley) and the authors.

Reference’s

The scale was adapted from the following seminal work:

• Kerstan, Sophie, Bienefeld, Nadine, & Grote, Gudela. (2024). Choosing human over AI doctors—How comparative trust associations and knowledge relate to risk and benefit perceptions of AI in healthcare. Risk Analysis, Vol 44(4), 939-957. doi: https://dx.doi.org/10.1111/risa.14216.

Items of the AI Risks and Benefits Scale

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

In your opinion, how likely are the following risks or benefits to occur? Al in healthcare might …

Rating Scale (7-point Likert scale):

• 1 = very unlikely
• 7 = very likely

Risks (R1-R10):

1. … fail to recognize the uniqueness of each patient’s condition
2. … result in a loss of jobs for healthcare professionals
3. … increase patient data security breaches
4. … lead to ethical problems
5. … introduce new bugs and equipment failures
6. … result in overreliance on technology
7. … dehumanize care
8. … increase medical errors
9. … provide unreliable information
10. … measure patient parameters inaccurately

Benefits (B1-B9):

1. … reduce medical errors
2. … enable a more personalized care
3. … facilitate the prediction of negative health events
4. … improve information sharing between patients and healthcare providers
5. … improve the accessibility of care
6. … provide more accurate health data measurements
7. … help in monitoring treatment efficiency
8. … help in diagnosing health problems
9. … help in choosing adequate treatments