Communication Training Impact Questionnaire

Abstract

The Communication Training Impact Questionnaire (CTIQ) is a 20-item self-report instrument developed by S. DeWine in 1987 to evaluate the efficacy of organizational communication training programs. Specifically, the CTIQ is designed to measure behavioral changes and the subsequent application of learned skills in actual work environments. This focus aligns directly with Level Three (Improvements in Job Behavior) of Kirkpatrick’s research model for training evaluation. The scale demonstrates strong internal consistency across diverse professional and student populations and utilizes a two-factor structure identified through oblique factor analysis, assessing both the relevance of the training to the job and the resulting skilled performance.

Keywords

Communication Training Impact Questionnaire, CTIQ, organizational communication, training evaluation, Kirkpatrick model, behavioral change, job performance, skill transfer, communication competency.

Authors

S. DeWine

Purpose

The primary purpose of the CTIQ is to provide a standardized, post-training measure of competency transfer. It moves beyond simple satisfaction or knowledge retention to assess whether participants successfully integrate newly acquired communication skills into their daily occupational routines. This evaluation is critical for organizations seeking to justify investment in training initiatives and measure the return on investment in human capital development.

The instrument is fundamentally anchored in the framework established by Donald L. Kirkpatrick, serving as a robust measure for the third level of training evaluation. While Kirkpatrick’s research model includes general reaction (Level 1), knowledge/attitudes/skills (Level 2), and organizational results (Level 4), the CTIQ specifically focuses on measuring the tangible improvements in job behavior resulting from the training intervention.

Construct

The CTIQ measures the construct of Organizational Communication Competency Application, defined as the degree to which individuals successfully apply communication skills learned in a formal training setting to their professional duties. The 20-item scale operationalizes this construct through two distinct factor dimensions derived from factor analysis: the perceived relationship of the training content to the actual job requirements, and the individual’s self-reported execution of skilled performance post-training.

A supplemental instrument, the 13-item Supervisor’s Training Impact Questionnaire, also exists, offering a crucial external perspective by measuring the impact of the training program on the worker’s job behavior as observed by their supervisor, thereby enhancing the triangulation of training impact data.

Validity

Evidence supporting the validity of the CTIQ was established through statistical procedures, specifically an oblique factor analysis and a scree test, which confirmed the underlying structural dimensions. These analyses support the scale’s construct validity by demonstrating that the items cluster logically into meaningful factors related to training impact and skill transfer.

Furthermore, the original study presented evidence of predictive validity, suggesting the scale is effective not only in measuring current skill application but also in anticipating future job performance or success following training completion. The identification of two distinct factors—Relationship of Training to Job and Skilled Performance—further solidifies the theoretical relevance of the scale’s structure.

Reliability

The CTIQ demonstrates strong internal consistency across the initial diverse sample population, as evidenced by high alpha coefficient scores. For the entire sample, the Cronbach’s alpha coefficient was reported at 0.88, indicating excellent reliability for an instrument designed to measure behavioral application.

Reliability estimates remained consistently high when examined across specific subgroups: middle managers achieved an alpha of 0.87; college students showed exceptional consistency with an alpha of 0.93; and student doctors combined with workers in a financial organization reported an alpha of 0.90. These scores confirm that the CTIQ is a highly reliable measure of training impact across varied professional and educational contexts.

Factor Analysis

The scale construction was refined using rigorous factor analytic procedures. Initially, 31 items were drafted based on literature review and topics recommended by Cornwell (1980), categorized into five theoretical areas. Subsequent application of factor analysis reduced the final instrument to 20 items, ensuring psychometric efficiency and clarity.

The final structure, derived from an oblique factor analysis and scree test, yielded two primary factors:

• Relationship of Training to Job: This factor measures the perceived relevance and applicability of the training content to the demands of the participant’s actual job. Items loading on this factor include 1–4, 10, 14, and 20–23 (Note: The item numbering in the original source suggests some discrepancy between the final 20-item scale count and the factor item list).
• Skilled Performance: This factor assesses the self-reported competence and successful execution of the learned communication skills in the work setting. Items loading on this factor include 5–9, 11–13, 16, and 19.

Instrument

Test Type: Self-report Questionnaire (Psychometric Scale)

Format: 20 items utilizing a 5-point Likert scale, ranging from Strongly Disagree (1) to Strongly Agree (5). Scoring is reversed on items 3–12 and 14–16 to ensure consistent interpretation of the underlying construct.

Language Available: English (Original)

Population Group: Professionals and students undergoing organizational or communication training.

Age Group: Adult (Implied by inclusion of middle managers, student doctors, and college students).

Population Details: The original sample (N=267) included 110 middle managers, 63 college students enrolled in a leadership training program, 60 student doctors, and 34 workers in a financial organization. Participants completed the measure two to four weeks following the conclusion of their respective training programs.

Test Methodology: Data analysis included the calculation of means and standard deviations, alongside the detailed results of the factor analysis used for scale refinement and validation.

Keywords

Organizational development, human resources, training effectiveness, communication skills, organizational behavior, professional training evaluation, self-efficacy, job transfer.

Authors

Author ORCID Identifier: Information not provided in source.

Affiliation Email addresses: Information not provided in source.

Correspondence Address: Information not provided in source.

Permissions & Fee and Test Year

The scale was formally published in 1987 in the Journal of Applied Communication Research. Information regarding current permissions, usage fees, or licensing for contemporary use of the CTIQ is not provided in the original publication abstract.

Reference’s

DeWine, S. (1987). Evaluation of organizational communication competency: The development of the Communication Training Impact Questionnaire. Journal of Applied Communication Research 15:113–27.

Cornwall, J. B. (1980). Measuring back-on-the-job performance. Training 289–92.

DeWine, S., and Pearson, J. S. (1985). The most frequently used self-report instrument in communications. Paper presented at the International Communication Association.

Kirkpatrick, D. L. (1967). Evaluation of training. In R. L. Craig and L. R. Bittel (Eds.), Training and Development Handbook. New York: McGraw-Hill.

Kirkpatrick, D. L. (1979). Techniques for evaluating training programs. Training and Development Journal 78–92.

Ledebur, G. W. (1982). Effects of interpersonal skill training upon the quality of parent-teacher conferences. Doctoral dissertation, Lehigh University.

Items of the Communication Training Impact Questionnaire

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

1. I use this skill regularly on the job.
2. After this training program I would perform this skill without practicing.
3. I didn’t learn this skill in the training program, so I had to learn it on the job.
4. I perform the skill differently on the job because work conditions don’t permit me to perform it the way I learned in the training program.
5. I perform the skills differently on the job because the skill doesn’t work the way I learned it in training.
6. I perform the skill differently on the job because my supervisor told me to do it differently.
7. I never perform this skill on the job.
8. The skill isn’t part of my job.
9. I get help to perform the skill because I didn’t learn it in the training program.
10. I don’t perform the skill on the job because the skill is too difficult for me.
11. I don’t perform the skill on the job because the skill comes up so rarely that I forgot how to do it.
12. I don’t perform the skill because I was assigned a different job.
13. I learned to perform the task well in the training program because the program was effective.
14. The skill could be learned from a manual or an instruction sheet as easily as in a training program.
15. I had trouble learning the skill because the training program was confusing.
16. The skill would have been easier to learn with more reference materials.
17. Because of learning this skill I feel more comfortable about doing my job.
18. Because of attending this training program, I feel better about the company.
19. After attending this training program, I am interested in attending other training programs.
20. I think my participation in this training program will help me to advance in the company.

Scoring: Strongly Disagree = 1; Disagree = 2; Sometimes Agree and Sometimes Disagree = 3; Agree = 4; and Strongly Agree = 5. Scoring is reversed on the following items: 3–12 and 14–16.