Diagnostic Competency During Simulation-Based Learning Tool

Abstract

The Diagnostic Competency During Simulation-Based (DCDS) Learning Tool, developed by Burt and Olson in 2023, functions as a specialized educational instrument designed to comprehensively assess individual diagnostic reasoning competencies among Nurse Practitioner (NP) students. This assessment is specifically intended for use within controlled simulation-based learning environments.

The tool is structured around six distinct competency domains, which were meticulously derived from established instruments and standard curriculum learning objectives, referencing foundational works such as those by Ahmed et al. (2017) and the American Board of Internal Medicine (2021). Comprising 28 items, the DCDS Learning Tool underwent thorough evaluation, including ratings provided by a sample of academic experts and faculty. Psychometric testing established both the reliability and validity of these measures. The primary goal of the DCDS is to equip nurse practitioner educators with detailed, actionable, and competency-specific data to facilitate targeted improvement in student diagnostic skills.

Keywords

Individual Diagnostic Reasoning Competencies, Nurse Practitioner Students, Nursing Education, Simulation-Based Learning Experiences, Educational Assessment, Clinical Competency.

Authors

Burt, Leah, Olson, Andrew.

[quads id=5]

Purpose

The fundamental purpose of the DCDS Learning Tool is to significantly broaden and refine the scope of diagnostic reasoning assessment in advanced practice nursing education. It moves beyond general evaluations to provide nurse practitioner educators with precise, actionable, and highly competency-specific measurements.

These detailed measurements are crucial for identifying specific areas of strength and weakness in a student’s diagnostic process. By facilitating this level of specificity, the DCDS tool serves as a mechanism to structure targeted feedback and interventions, ultimately driving measurable improvement in students’ clinical diagnostic skills before they transition to independent practice.

Construct

The DCDS Learning Tool measures Diagnostic Competency, operationalized within the context of high-stakes clinical simulations. This construct is defined as the integrated set of cognitive and behavioral skills required by a clinician to accurately and efficiently arrive at a diagnosis.

The scale’s 28 items are organized into six core competency domains, reflecting a multidimensional view of diagnostic excellence. These domains cover the entire diagnostic process, from initial data gathering and hypothesis generation to final diagnostic decision-making and communication, ensuring a comprehensive assessment of the student’s clinical judgment during the simulated encounter.

Validity

The validity of the DCDS Learning Tool was rigorously assessed, primarily focusing on Content Validity.

Content validity was established through expert review to ensure the items accurately represent the full scope of diagnostic reasoning required of NP students. The final individual competency domain Content Validity Index (CVI) scores demonstrated high agreement among experts, ranging from 0.9175 to a perfect 1.0. The total scale CVI score further confirmed the tool’s content relevance and representativeness, reaching an excellent value of 0.98.

Reliability

The evaluation of reliability focused on Interrater Reliability, which is critical for observational instruments used by multiple educators.

The statistical measure employed was the Intra-class Correlation Coefficient (ICC). The calculated ICC for the DCDS Learning Tool was 0.548 (p < 0.0001), with a 95% confidence interval (CI) ranging from 0.482 to 0.612. While statistically significant, this moderate ICC suggests that while the tool captures meaningful differences in performance, training and standardization for raters remain important to maximize consistency in scoring across faculty members.

Factor Analysis

Data regarding the use of Factor Analysis (e.g., Exploratory or Confirmatory Factor Analysis) to confirm the underlying dimensional structure of the six proposed competency domains is currently unavailable in the reported psychometric documentation.

Instrument

Test Type: Original instrument developed for educational assessment.

Format: The DCDS Learning Tool is designed for use across a variety of simulation scenarios. It guides educators to consider scenario characteristics before evaluating learner performance. Raters assign ordinal-level judgments (minimal, partial, or complete) to observed behaviors. If a student demonstrates only a portion of the behaviors described within an item, raters are advised to assign a lower rating. Educators utilizing the DCDS tool should analyze learner behavior ratings to identify trends and inconsistencies between competencies. During feedback sessions, educators should emphasize behaviors that will facilitate the student’s progression to the next competency rating level.

Language Available: English

Population Group: Human; Male; Female

Age Group: Adulthood (18 yrs & older)

Population Details:

• Location: United States

• Respondents: Experts; Nursing Education Faculty

Test Methodology: Test Validity; Content Validity; Test Reliability; Interrater Reliability.

Keywords

Diagnostic Competence, Nursing Education Faculty, Intra-class Correlation Coefficient (ICC), Content Validity, Nurse Practitioner Training, Clinical Simulation.

[quads id=5]

Authors

Author ORCID Identifier: No data is Available

Affiliation Email addresses:

• Leah Burt: [email protected]

• Andrew Olson: No data is Available

Correspondence Address:

Leah Burt: University of Illinois Chicago College of Nursing, Department of Biobehavioral Nursing Science, 845 South Damen Avenue (MC 802), Chicago, Illinois, United States, 60612, [email protected]

Permissions & Fee and Test Year

Permissions: Contact Corresponding Author

Fee: No

Test Year: 2023

Reference’s

Burt, L., & Olson, A. (2023). Development and psychometric testing of the Diagnostic Competency During Simulation-based (DCDS) Learning Tool. Journal of Professional Nursing, 45, 51–59. https://doi.org/10.1016/j.profnurs.2023.01.008

[quads id=5]

Items of the Diagnostic Competency During Simulation-Based Learning Tool

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

This measure consists of 28 items. No data is Available.