Even though Oregon State is a co-sponsor for this program, being in Oregon in winter (i.e., now) is not the land of sunshine, and since Vitamin D is critical for everyone’s well being, I chose Tucson for our capstone event. Our able support person, Gretchen, chose the La Paloma, a wonderful site on the north side of Tucson. So even if it is not warm, it will be sunny. Why, we might even get to go swimming; if not swimming, certainly hiking. There are a lot of places to hike around Tucson…in Sabino Canyon ; near/around A Mountain (first year U of A students get to whitewash or paint the A) ; Saguaro National Park ; or maybe in one of the five (yes, five) mountain ranges surrounding Tucson. (If you are interested in other hikes, look here.)
We will be meeting Tuesday afternoon, all day Wednesday, and Thursday morning. Participants have spent the past 17 months participating in and learning about evaluation. They have identified a project/program (either big P or little p), and they participated in a series of modules, webinars, and office hours on topics used everyday in evaluating a project or program.We anticipate over 20 attendees from the cohorts. We have participants from five Extension program areas (Nutrition, Agriculture, Natural Resources, Family and Community Science, and 4-H), from ten western states (Oregon, Washington, California, Utah, Colorado, Idaho, New Mexico, Arizona, Wyoming, and Hawaii.), and all levels of familiarity with evaluation (beginner to expert).
I’m the evaluation specialist in charge of the program content (big P) and Jim Lindstrom (formerly of Washington State, currently University of Idaho) has been the professional development and technical specialist, and Gretchen Cuevas (OSU) has been our wonderful support person. I’m using Patton’s Developmental Evaluation Model to evaluate this program. Although some things were set at the beginning of the program (the topics for the modules and webinars, for example), other things were changed depending on feedback (readings, office hours). Although we expect that participants will grow their knowledge of evaluation, we do not know what specific and measurable outcomes will result (hence, developmental). We hope to run the program (available to Extension faculty in the Western Region) again in September 2013. Our goal is to build evaluation capacity in the Western Extension Region. Did we?
Implementation. To implement an evaluation, one needs to have a plan, often called a protocol. Typically, this is a step-by-step list of what you will do to present the program to your target audience. In presenting your program to your target audience, you will also include a step-by-step list of how you will gather evaluation information (data). What is important about the plan is that it be specific enough to be replicated by other interested parties. When a plan is developed, there is typically a specific design behind each type of data to be collected. For example, specific knowledge change is often measured by a pretest-posttest design; behavioral change is often measured with a repeated measures design. Campbell and Stanley, in their classic book, Experimental and quasi-experimental designs for research, present a wealth of information about designs that is useful in evaluation (as well as research).
There are numerous designs which will help develop the plan for the implementation of the program AND the evaluation.
Monitoring. Simply put, monitoring is watching to see if what you said would happen, actually does. Some people think of monitoring as . Although monitoring may seem like being watched, it is being watched with a plan. When I first finished my doctorate and became an evaluator, I conceptualized evaluation simply as process, progress, product. This helped stakeholders understand what evaluation was all about. The monitoring part of evaluation was answered when I asked, “Are we making progress? Are we where we said we would be at the time we said we would be there?” This is really important because sometimes, as Jonny Morell points out in his book, evaluation don’t always go as planned, even with the best monitoring system.
Delivering. Delivering is the nuts and bolts of what you are going to do. It addresses the who, what, where, when, how, and why of the implementation plan. All of these questions interrelate–for example, if you do not identify who will conduct the evaluation, often the evaluation is “squeezed in” at the end of a program because it is required.
In addition to answering these questions when delivering the evaluation, one thinks about the models, or evaluation approaches. Stufflebeam, Madaus, and Kellaghan (in Evaluation models: Viewpoints on educational and human services evaluation) discuss various approaches and state that the approach used by the evaluator will provide a framework for conducting an evaluation as well as presenting and using the evaluation results.