Module 1 Assignment 2, Choosing a Program Evaluation

In reading Klomp, Dyck, and Sheppard’s 2003 "Description and evaluation of a prenatal exercise program for urban Aboriginal women" published in the Canadian Journal of Diabetes, 27: 231-238, I feel that Scriven’s goal-free model would work well to evaluate the Prenatal Exercise Program for Urban Aboriginal Women. The program goals are not clearly stated in the description referenced above allowing for an exploration of what the goals and impacts of the program are.  With the goal-free model the evaluator “….searches for all effects of a program regardless of its developers’ objectives.” (Stufflebeam and Shinkfield, 2007, p. 374).  I feel that using this model will identify both the anticipated and unanticipated outcomes and, by gathering all this information, program developers may more deeply understand the program’s benefits or shortcomings for any future efforts of health professionals and Aboriginal peoples with addressing diabetes and related health issues in the Aboriginal population. 

I then wondered who will most directly benefit from this prenatal exercise program and how it might be brought to the larger group.  In applying the consumer-oriented approach (Stufflebeam and Shinkfield, 2007, p. 374) to this evaluation model, the participants, their children, and the larger Aboriginal population will see their needs addressed. The evaluation focus would not be solely on the program itself but will value the consumer’s experience and need.  “Irrespective of the goals the evaluator must identify actual outcomes and assess their value from the perspective of consumer’s needs.” (Stufflebeam and Shinkfield, 1985, p. 312).  The participant’s voice will be heard when determining the effects of the program and the reasons for the observed outcomes and effects.

I also feel the goal-free evaluation tool brings forth a less prescriptive evaluation approach allowing the evaluator to be responsive to participants.  There is a larger picture that can be focussed on than just “…a means and an end….”  As a specific demographic was the focus of this exercise program, the evaluation needs to respect the attitudes and values of the participants.  The goal-free model, according to Scriven, will guide an evaluation that is “…less prone to social, perceptual, and cognitive bias; … and more equitable in considering a wide range of values.” (Stufflebeam and Shinkfield, 2007, p. 374).  There is significant emphasis on the humanistic side of this program regarding implementation and supports.  I wonder if this attention was present when the first actions in developing the program were taken or, if it was the statistical data that brought about the program designers’ efforts in creating the program initially.  The focus on participants is observed through the many supports provided to them for their regular attendance and participation in the program.  Using the goal-free, consumer-oriented approach gives the evaluator the chance to authentically understand the participants as well as the people delivering and supporting the program to gain better insight into the program’s effects and perspectives on why they occurred.  

Applying the goal-free model allows the evaluator to determine the path of the evaluation and can be responsive to the various factors within the program while gathering qualitative and quantitative data.  I feel that both formative and summative assessment should be applied to this evaluation effort to effectively identify the program goals and its effects.  The two year program was flexible and adjusted mid-course regarding services provided to participants; therefore, using formative feedback and assessments, flexibility and responsiveness to all involved can be supported.  Gathering not only the quantitative data regarding gestational diabetes mellitus regarding the participants and for the long-term study of diabetes developing in their children, but also the qualitative information including the participants' stories, values and attitudes in relation to this study will be highly relevant in determining the program goals and effects.  One further evaluation I would suggest incorporating is a cost-benefit analysis, such as from the Provus model, to further evidence how funding impacts the program.  Many human resources were put in place for the success of the program along with services and access to other resources for the participants to achieve successful participation.  Analysing the costs of these resources to the program benefits could be of use as the effects of the program are explored and the program replicated or expanded.  These costs could also be compared to the long-term anticipated costs of diabetes 2 which can develop in the participants and their children in the future. 

I believe using the goal-free, consumer-oriented approach with a cost-benefit analysis will provide a comprehensive report, which includes and respects participants' cultural perspective, and will see the summative report as a culturally responsive evaluation that brings validity to the program.  

References:

Stufflebeam, D. & Shinkfield, A. (2007). Evaluation Theory, Models and Application. San Francisco, CA: Jossey-Bass. 

Stufflebeam, D. & Shinkfield, A. (1985). Systematic Evaluation. USA: Kluwer-Nijhoff Publishing.