Ecur 809 Module 1 Assignment 1, Program Evaluation Commentary

With hopes that I would locate and be able to use an evaluation pertaining to my work, I was very happy to find an evaluation about a teacher professional development model that is similar to one I currently participate in.  

In March 2004, the Center for Research on Teaching and Learning presented their findings on three Professional Development Laboratory programs implemented in Community School District (CSD) 20 in New York City for the 2002-2003 school year. This document can be found at Evaluation of the Professional Development Lab (PDL) Programs in Community School District 20 .

The Professional Development Laboratory, a professional development unit instituted in 1989, focuses on professional development and training programs for teachers and school administrators.   They developed and implemented the following three PDL programs in CSD 20 which are the focus of this evaluation:

• PDL for New Teachers Program was implemented over the course of three years and brought new teachers with 2 or fewer years of experience to an induction program for the purposes of increasing their professional competency in four focused teaching areas, providing them with high-quality mentoring, and visiting Resident Teacher classrooms for observations and learning opportunities to improve their pedagogical skills;
• PDL for Middle School Social Studies Program, was also three years long and provided training institutes for participating teachers to build their professional competency in five focused teaching areas along with further developing their leadership and communication skills with principles of mentoring; and, 
• Learning Through Practice Leadership Program (LTPLP), a seven day training program was delivered to develop leadership skills of selected district teachers and staff developers.

The intended purpose of this evaluation was to document the implementation of the Professional Development Lab programs for their middle school teachers in Community School District 20.  It was also designed to determine the impact of the PDL programs upon the mentors, new middle school teachers, and the selected middle school teachers involved as well as the students served by the programs.

I believe this evaluation is a mixed model incorporating Scriven’s goal-based summative approach along with participatory empowerment elements.  There is a clear goal and data was used to determine if the goal was achieved by the end of the program.  To collect qualitative and quantitative data, a wide range of tools were implemented: questionnaires, interviews, surveys, rating scales, teacher observations, teacher assessment logs, and a state achievement test for students, along with statistics on teacher retention rates from New York City's Department of Education. Formative evaluations were completed on a weekly basis to guide teachers and their mentors in the ongoing work of supporting teacher’s needs as they worked toward the goal of improving "...teaching and learning by helping to build the professional competency of teachers." (p. 2) which was the common goal among the three programs.

I found the report to be credible with disclosures about the context and limitations and the inability to report on one area of evaluation which was restated twice in the report rather than mentioned and hidden.  There are many strengths to the evaluation including the detailed information provided about each program being implemented, the underlying rationale, and research supporting this professional development model.  As well, complete information about the implementation of the programs and contextual information further helps the reader paint a clear picture of the program, implementation, data collection, and evaluation findings.  Many evaluation tools were implemented and along with quantitative results included to indicate teacher and student improvement and the submission of completed assessment tools, this comprehensive evaluation included comments and recommendations from the participants while their identities were protected, being identified only by their role.  Many of the tools used to collect the data are familiar tools in the teaching profession with processes that are widely accepted in the educational community.  Also, tools did not just collect data at the end of the school year but throughout the year to measure the impacts of the various components provided to participants for their successful improvement.   Finally, I feel the evaluation has included abundant information so that it is not only useful to stakeholders but also to others interested in this professional development model.

One of the aspects of this evaluation that I felt was a weakness is that four of the evaluation tools were being piloted in this process which questions their reliability and validity.  This would have not been such a big issue if there was a second established tool to cover the same areas being evaluated that the pilot tools were addressing but, it led to having inconclusive findings for one of the evaluation queries regarding teacher efficacy. Teachers simply did not return the completed evaluation which makes me wonder if it was a theory failure or an implementation failure.  In regard to the conclusions and recommendations, the majority of comments made reflected the findings but recommendations concerning the middle school social studies program was lacking.  Student achievement scores for middle years social studies did not improve as a result of their teachers being in the program and I would have liked a more detailed comment for the single recommendation to "...pay greater attention to the collection of more complete data." (p. 46). Contributing to the content of this single recommendation was the failure of teachers to collect and submit student work.  As student work submissions were inconsistent they could not be used for the evaluation in a significant way.  I do not think more student work submissions would completely address the lack of teacher improvement and would like to have seen recommended change in the areas focused on for selected middle school teacher improvement.

I have learned much from the process of completing this assignment, especially regarding the intensity of completing an evaluation as well as about the lab classroom professional development model.

Thanks for reading!  

I welcome your comments, 

Corinne