Assessment of Situational Judgement (ASJ)

Abstract

The Assessment of Situational Judgement (ASJ) is a comprehensive, 17-item psychological instrument designed to quantify an individual’s scam susceptibility across diverse environmental settings, including clinical, community, and financial domains. The core function of the ASJ is to evaluate the respondent’s capacity for accurate judgment, specifically their ability to differentiate between fraudulent (scam) and benign (legitimate) situations. Items are derived from real-world examples, including documented scams reported by the Florida Division of Consumer Services. Development involved a rigorous pilot study and validation testing on a sample of adults recruited through Amazon’s Mechanical Turk platform. Respondents utilize a Likert scale to rate the perceived likelihood of engaging with or falling for the described scenarios.

Keywords

Scam susceptibility, Situational Judgement, Financial fraud, Likert scale, Older adults, Decision-making, Consumer services.

Authors

Getz, Sarah J., Levin, Bonnie E., Galvin, James E.

Purpose

The primary purpose of the Assessment of Situational Judgement (ASJ) is to precisely measure the cognitive recognition abilities required to distinguish fraudulent requests from genuine, legitimate requests within various decision-making contexts. This assessment is highly relevant in an era of increasing digital and telephonic fraud.

The instrument was specifically developed to assess how effectively older adults, a population often targeted by financial exploitation, can discern complex scamming scenarios across multiple modalities, including online interactions, in-person solicitations, and telephone calls.

Construct

The central psychological construct measured by the ASJ is Scam Susceptibility. This construct encompasses the vulnerability of an individual to fraudulent schemes, reflecting not only their exposure to scams but, more critically, their judgment capacity and ability to accurately assess the risk inherent in specific situational vignettes.

Validity

The validity of the ASJ was established through multiple statistical approaches, confirming its ability to accurately measure the intended construct. Group Validity was demonstrated via comparisons based on key sociodemographic variables, levels of computer and social media usage, and documented prior exposure to scams. Results confirmed that the ASJ successfully differentiates between individuals categorized as having higher versus lower risk profiles for financial exploitation.

Furthermore, diagnostic accuracy was assessed using Receiver Operating Characteristic (ROC) curve analyses. These analyses yielded robust Areas Under the Curve (AUC) values: 0.790 for the Scam factor, 0.730 for the Legitimate factor, and 0.767 for the combined Scam/Legitimate ratio. Optimal cut-off scores were determined based on these analyses, maximizing the instrument’s sensitivity and specificity for clinical use.

Reliability

The reliability of the ASJ was primarily assessed through measures of internal consistency. The instrument demonstrated very high reliability for the critical factor assessing vulnerability to fraud. Specifically, the Scam factor achieved a high Cronbach’s alpha coefficient of 0.98, indicating excellent consistency among the items designed to capture fraudulent scenarios.

The Legitimate factor also demonstrated acceptable internal consistency, yielding a Cronbach’s alpha of 0.73. These results suggest that the two primary factors of the ASJ are reliably measuring distinct yet related aspects of situational judgment regarding financial safety.

Factor Analysis

Exploratory factor analysis confirmed the two-factor structure underlying the ASJ. The analysis revealed that the Scam factor accounted for a substantial portion of the total variance observed, explaining 50.6%. This highlights the dominance of the vulnerability component in the overall assessment.

The Legitimate factor, representing appropriate situational responses, explained 10.6% of the variance. The clear separation and significant variance explained by the Scam factor validate the instrument’s focus on detecting high-risk situational judgment deficits related to scam susceptibility.

Instrument

Test Type: Vignette/Scenario-based Assessment

Format: Items are rated using a 7-point Likert scale, ranging from 1 (extremely unlikely) to 7 (extremely likely). Scores for the Scam and Legitimate factors are calculated by averaging the responses across their respective items.

Language Available: English

Population Group: Adults (specifically designed for older adults)

Age Group: Adult population (No specific age range provided, but emphasis on older adults)

Population Details: Validation sample consisted of adults recruited electronically via Amazon’s Mechanical Turk platform.

Test Methodology: Test Validity, Test Reliability, Internal Consistency, Factor Analysis, Test Sensitivity, Test Specificity, Receiver Operating Characteristic (ROC) curve analysis.

Keywords

Financial exploitation, Fraud detection, Situational awareness, ROC curve, Cronbach’s alpha, Geriatric assessment, Vulnerability.

Authors

Author ORCID Identifier: Not provided in source material.

Affiliation Email addresses: Not provided in source material.

Correspondence Address: Not provided in source material.

Permissions & Fee and Test Year

The instrument was published in 2024. Information regarding specific permissions required for non-academic use and applicable testing fees is not detailed in the primary source reference.

Test Year: 2024

Reference’s

Getz, S. J., Levin, B. E., & Galvin, J. E. (2024). The Assessment of Situational Judgment questionnaire: A novel instrument to detect susceptibility to financial scamming. Journal of Alzheimer’s Disease, 97(3), 1365–1379. The digital object identifier (DOI) for this publication is https://doi.org/10.3233/JAD-231194.

Items of the Assessment of Situational Judgement (ASJ)

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

• Scam items
• Legitimate items