Principal Evaluation Instrument

Abstract

The Principal Evaluation Instrument (PEI) was developed by E. J. Condon, III. in 2009 as part of a doctoral dissertation investigating the relationship between principal evaluation practices and student achievement in public elementary schools across select counties in Illinois. The instrument is a comprehensive survey designed to assess two primary facets of evaluation: the specific methods used to formally evaluate principal performance and the perceived objectives accomplished by the evaluation process. It integrates quantitative rating scales alongside qualitative, open-ended questions to gather nuanced data regarding principals’ beliefs and experiences related to their professional assessments.

The scale is divided into two main matrices, comprising 21 Likert-type items in total, plus four additional open-ended questions. The PEI was initially tested on a sample of 130 K–8 public school principals. Reliability analyses demonstrated acceptable to good internal consistency for both quantitative sections, while content validity was established through rigorous academic review and pilot testing processes.

Keywords

Principal evaluation, student achievement, school leadership, educational administration, performance appraisal, professional development, evaluation methods, school principals.

Authors

E. J. Condon, III.

Purpose

The primary purpose of the Principal Evaluation Instrument (PEI) is to systematically gather data on the current state of principal evaluation practices within educational settings, specifically focusing on how these practices relate to school outcomes, such as student achievement. The instrument aims to quantify the frequency of use for various evaluative methods and the perceived effectiveness of the evaluation process in achieving specific organizational and professional goals.

The instrument was designed specifically for research purposes within Condon’s 2009 study to explore the hypothesis that formal principal evaluation is linked to improved student outcomes. By separating the inquiry into methods used and objectives achieved, the scale allows researchers to differentiate between the procedural aspects of evaluation and the intended or perceived functional outcomes.

Construct

The Principal Evaluation Instrument primarily measures two distinct but related constructs within the field of educational leadership and accountability: Evaluation Methods Utilized and Evaluation Objectives Perceived. The instrument also probes the construct of Principal Beliefs Regarding Evaluation Impact through its open-ended questions.

The first matrix operationalizes the construct of Evaluation Methods by listing ten common techniques (e.g., narrative evaluation, data-based evaluation, stakeholder feedback). The second matrix measures the perceived degree to which evaluation achieves eleven critical objectives, ranging from satisfying district accountability requirements to improving pupil achievement and fostering professional growth. Together, these constructs provide a comprehensive profile of how evaluation is performed and what impact it is believed to have on both the principal and the school environment.

Validity

The validity of the Principal Evaluation Instrument was established primarily through a focused, three-step approach to content validity. This process was crucial for ensuring that the instrument’s items accurately reflected the intended constructs related to principal evaluation practices and objectives.

The first step involved careful review and input from the dissertation committee, which provided expert guidance on the wording of the instrument questions and their alignment with the theoretical constructs being measured. Secondly, the instrument underwent a pretest phase with a small cohort of respondents to identify potential ambiguities or issues in clarity. Finally, the instrument was subjected to a rigorous pilot study involving an independent group of respondents who possessed similar professional backgrounds to the main study participants but were excluded from the final data collection. This multi-stage validation process strengthened confidence that the PEI adequately sampled the domain of principal evaluation.

Reliability

The reliability of the Principal Evaluation Instrument was assessed using different statistical measures appropriate for its mixed-method design. For the two quantitative matrices, internal consistency reliability was calculated using Cronbach’s alpha coefficient.

• Matrix 1 (Evaluation Methods): The alpha coefficient was reported as 0.70. This value indicates acceptable internal consistency, suggesting that the ten items measuring the use of evaluative methods are reasonably interrelated.
• Matrix 2 (Evaluation Objectives): The alpha coefficient was reported as 0.88. This high value indicates good to excellent internal consistency, confirming that the eleven items assessing the perceived objectives of evaluation are strongly unified.

For the four qualitative, open-ended questions, reliability was established through the assessment of inter-rater reliability. The Kappa Measure of Agreement was utilized to ensure consistency in the coding and interpretation of the qualitative responses provided by the principals, thereby maintaining the trustworthiness of the textual data collected.

Factor Analysis

The source documentation primarily reports on descriptive statistics and internal consistency reliability (Cronbach’s alpha). While Pearson product-moment correlation coefficients were calculated, the original summary does not explicitly detail the execution or findings of a formal Factor Analysis (e.g., Exploratory or Confirmatory Factor Analysis) to confirm the underlying factor structure of the two matrices. The strong internal consistency scores (0.70 and 0.88) suggest that the items within each matrix are unidimensional or highly correlated, but formal factor structure validation remains unconfirmed based on the provided abstract.

Instrument

Test Type: Survey Instrument / Self-Report Questionnaire

Format: Mixed-method instrument combining two seven-point Likert scales (Matrices 1 and 2) and four qualitative, open-ended questions requiring narrative explanation.

Language Available: English

Population Group: Educational Administrators / School Principals

Age Group: Adult (Professionals in K–8 education)

Population Details: The sample consisted of 130 public school principals serving grades K–8 in DuPage, Will, and Lake Counties, Illinois, USA. Detailed demographic information regarding the respondents is available within the original dissertation.

Test Methodology: Data collection involved administering the two quantitative matrices, scored on a seven-point scale (1 = Not At All, 7 = Very Much). Descriptive statistics (means, standard deviations, skewness, and kurtosis) were used to analyze the quantitative responses. Correlation analysis was also performed using Pearson coefficients.

Keywords

Educational leadership, school accountability, performance measurement, professional growth, K-8 schools, Illinois education, quantitative research, qualitative assessment.

Authors

Author ORCID Identifier: Not provided in source.

Affiliation Email addresses: Not provided in source.

Correspondence Address: Based on the source, the author, E. J. Condon, III., completed the PhD dissertation at Loyola University of Chicago (2009).

Permissions & Fee and Test Year

The instrument was developed and used for research purposes in 2009 as part of a doctoral dissertation. Information regarding commercial permissions, licensing fees, or public availability beyond the academic dissertation is not provided in the source summary. Researchers interested in utilizing the instrument should contact the author or Loyola University of Chicago’s institutional repository for access to the full dissertation, which contains the complete instrument details.

Reference’s

Condon, E. J., III. (2009). Principal evaluation and student achievement: A study of public elementary schools in Du- Page, Will, and Lake Counties, Illinois. PhD dissertation, Loyola University of Chicago.

Catano, N., and Stronge, J. (2006). What are principals expected to do?: Congruence between principal evaluation and performance standards. National Association of Secondary Schools Principals Bulletin, 90:221–37.

Kaplan, L. S., Owings, W. A., and Nunnery, J. (2005). Principal quality: A Virginia study connecting interstate school leaders licensure consortium standards with student achievement. National Association of Secondary Schools Principals Bulletin, 89(643):28–44.

Kearney, K. (2005). Guiding improvements in principal performance. Leadership, 35:18–21.

Thomas, D. W., Holdaway, E. A., and Ward, K. L. (2000). Policies and practices involved in the evaluation of school principals. Journal of Personnel Evaluation in Education, 14:215–40.

Items of the Principal Evaluation Instrument

IMPORTANT: The following scale items must be preserved in their original language and must not be changed in any way.

To what extent, if at all, have the following methods been used to formally evaluate your performance as principal?

1. Narrative self-evaluation
2. Portfolio/dossier
3. Checklist/rating system
4. Supervisor observation
5. Narrative evaluation by supervisor
6. Data-based evaluation
7. Survey data from teachers, parents, or students
8. Peer supervision/review
9. Anecdotal evidence.
10. Perception feedback from stakeholders

Scoring: A seven-point scale ranging from Not At All = 1 to Very Much = 7. Scores range from a low of 10 to a high of 70.

To what extent, if at all, do you perceive principal evaluation accomplishing the following objectives?

11. Satisfy district accountability requirements
12. Increase standardized assessment scores
13. Provide principals with professional growth
14. Document substandard principal performance
15. Identify the needs for principal professional development
16. Provide incentive for performance improvement
17. Reward exemplary principal performance
18. Ensure adherence to policies and procedures
19. Support the maintenance of the instructional program
20. Improve pupil achievement
21. Foster positive school climate

Scoring: A seven-point scale ranging from Not At All = 1 to Very Much = 7. Scores range from a low of 11 to a high of 77.

22. What effect, if any, has your experience with evaluation had on your beliefs about principal performance? Please explain.
23. What effect, if any, has your experience with evaluation had on your beliefs about your own professional development? Please explain.
24. Do you believe that the principal evaluation process has an impact on pupil performance? Please explain.
25. Do you believe that the principal evaluation process has an impact on your own professional development? Please explain.