Taxonomy of non-technical skills for Officers of the Deck

Abstract

The Nontechnical Skills Taxonomy for Officers of the Deck (NTSOD) is a behavioral marker system developed by Long and O’Connor to evaluate the essential nontechnical skills of U.S. Navy Officers of the Deck (OODs). The OOD is responsible for the safe and proper operation of the ship, making nontechnical competencies critical for performance. The taxonomy was derived from an initial set of 17 skill categories identified via literature review and refined using focus groups of qualified OODs. This process led to the classification of 149 behavioral statements collected through the Critical Incident Technique. The final structure comprises four core categories—Leadership, Communications, Situational Awareness, and Decision Making—each broken down into specific behavioral elements. This prototype system is designed to enhance safety and performance evaluation in both military and civilian maritime contexts.

Keywords

Nontechnical skills, NTSOD, Officer of the Deck, OOD, Maritime safety, Behavioral assessment, Leadership, Situational Awareness, U.S. Navy, Crew resource management.

Authors

William M. Long, Patrick O’Connor, Michael McCauley.

Purpose

The primary purpose of the NTSOD taxonomy is to serve as a standardized behavioral marker system for the systematic observation and evaluation of the cognitive and social skills demonstrated by Officers of the Deck during watch periods. This tool provides trainers and evaluators with a structured framework for assessing performance dimensions that extend beyond technical seamanship, focusing instead on the soft skills critical for effective command and control.

The NTSOD is intended specifically for use in training and assessment environments within the U.S. Navy. It aims to quantify performance deficiencies and strengths related to high-stakes operational duties where the OOD is directly accountable to the Commanding Officer for the safety and proper functioning of the vessel.

Construct

The NTSOD taxonomy measures four overarching constructs of nontechnical skills that are indispensable for effective command and control in a maritime environment. These constructs align closely with established Crew Resource Management (CRM) skills utilized in high-reliability organizations like aviation.

The four primary categories identified are Leadership, Communications, Situational Awareness, and Decision Making. These categories were derived iteratively, starting from 17 initial skill areas identified in the literature and distilled into a manageable taxonomy based on input from qualified OODs. The final behavioral elements within each category represent observable actions that reflect the proficiency of the officer in these crucial domains, ensuring the safe navigation and operation of the ship.

Validity

While formal statistical validation studies beyond initial development are not fully detailed in the source material, the NTSOD exhibits strong content validity. The development process systematically ensured that the taxonomy covered all critical nontechnical skills required for the Officer of the Deck role, which is essential for any performance assessment tool.

The initial framework was built upon an extensive literature review and subsequently validated through multiple iterations of adaptation involving expert focus groups of qualified OODs. Furthermore, the use of the Critical Incident Technique ensured that the behavioral markers directly corresponded to real-world, high-stakes incidents observed during ship operations, lending high face validity and practical utility to the instrument for training purposes.

Reliability

The reliability of the NTSOD taxonomy was specifically established through the assessment of inter-rater agreement during the classification phase. The methodology involved the collection of 149 behavioral statements related to OOD performance, which were then independently classified by two trained raters.

Following three necessary iterations of refinement and adaptation to the taxonomy structure, the developers successfully achieved acceptable levels of inter-rater reliability. This finding confirms that the behavioral categories and elements are sufficiently clear, distinct, and operationally defined, allowing different evaluators to consistently apply the rating system when observing or analyzing OOD performance.

Factor Analysis

Formal statistical factor analysis is not explicitly documented as part of the NTSOD taxonomy development. The structure of the scale relies instead on a rigorous, expert-driven qualitative grouping process common in the development of behavioral marker systems.

The developers utilized qualitative methods, including literature review and critical incident analysis, to thematically group related behaviors into the four core categories (Leadership, Communications, Situational Awareness, and Decision Making). This process established the structural integrity based on operational relevance and theoretical alignment with established nontechnical skills frameworks, rather than purely statistical reduction of variance.

Instrument

Test Type: Observational Behavioral Marker System / Performance Rating Tool

Format: Structured observation checklist and rating form using a 4-point rating scale (1=Unsatisfactory to 4=Outstanding), supplemented by a “Not Observed” (N/O) option.

Language Available: English

Population Group: Military personnel (U.S. Navy officers)

Age Group: Adult (Officers in training or qualified watchstanders)

Population Details: Specifically designed for evaluating the competence of Officers of the Deck (OODs) aboard U.S. Navy ships, focusing on their command and bridge management skills.

Test Methodology: Direct observation of the OOD during live watch periods, simulated exercises, or analysis of critical incident reports, followed by rating based on predefined behavioral elements within the four core skill categories.

Keywords

Maritime psychology, Nontechnical skills assessment, Naval training, Performance measurement, Ship handling, Crew performance, Safety Science, Reliability assessment.

Authors

Author ORCID Identifier: Not provided in source.

Affiliation Email addresses: Not provided in source.

Correspondence Address: Research was conducted at the Naval Postgraduate School, Monterey, CA, where the initial user guide was published.

Permissions & Fee and Test Year

The initial development and publication of the user guide occurred in 2010. Information regarding current usage permissions or associated fees is not provided in the source material, though the original technical report was published by the Naval Postgraduate School (NPS).

Reference’s

• Long, W.M., O’Connor, P., & McCauley M. (2010). Nontechnical skills for Officers of the Deck (NTSOD) rating form: A user’s guide. Monterey, CA: Naval Postgraduate School. The original PDF can be downloaded here: http://edocs.nps.edu/npspubs/scholarly/TR/2010/NPS-OR-10-009.pdf
• O’Connor, P. & Long, W.M. (2011). The development of a prototype behavioural marker system for U.S. Navy Officers of the Deck. Safety Science, 49, 1381-1387.

Items of the Nontechnical skills taxonomy for Officers of the Deck

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

NON-TECHNICAL SKILLS FOR OFFICERS OF THE DECK (NTSOD) RAT

Ship _______________________________________________

Trainee _____________________________________________

Watch ________

Date _______________________________________________

Trainer _____________________________________________

Evolution_______

Category Category Rating* Element Element Rating* Notes Leadership   Managing Watch Team     Coping with Stress     Communications   Providing Information     Issuing Orders       Situational Awareness   Gathering Awareness     Understanding Awareness     Anticipating Future Events       Decision Making   Analytical Decision Making     Following Orders & Procedures     Intuitive Decision Making

* 1 – Unsatisfactory; 2 – Marginal; 3 – Satisfactory; 4 – Outstanding; N/O – Not Observed

1 – Unsatisfactory: Watchstander could endanger ship and crew without considerable improvement. 2 – Marginal: Watchstander requires improvement.

3 – Satisfactory: Watchstander performed at an acceptable level, but room for improvement exists. 4 – Outstanding: Watchstander performed at a consistently high level.