I regularly follow Harold Jarche’s blog .
Much of what he writes would not fall under the general topic of evaluation. Yet his blog for February 18 does. This blog is titled Why is learning and the sharing of information so important?
I see that intimately related to evaluation, especially given Michael Quinn Patton’s focus on use. The way I see it, something can’t be used effectively unless one learns about it. Oh, I know you can use just about anything for anything–and I am reminded of the anecdote of when you have a hammer everything looks like a nail, even if it isn’t.
That is not the kind of use I’m talking about.
I’m talking about rational, logical, systematic use based on thoughtful inquiry, critical thinking, and problem solving. I’m talking about making a difference because you have learned something new and valuable (remember the root of evaluation?). In his blog, Jarche cites the Governor-General of Canada, David Johnston and Johnston’s article recently published in the Globe and Mail, a newspaper published in Toronto. What Johnston says makes sense. Evaluators in this context are diplomats, making learning accessible and sharing knowledge.
Sharing knowledge is what statistics is all about. If you think the field of statistics is boring, I urge you to check out the video called The Joy of Stats presented by Swedish scholar Hans Rosling . I think you will have a whole new appreciation of statistics and the knowledge that can be conveyed. If you find Hans Rosling compelling (or even if you don’t), I urge you to check out his TED Talks presentation. It is an eye-opener.
I think he makes a compelling argument about learning and sharing information. About making a difference. That is what evaluation is all about.
The GAO (Government Accounting Office) has a long and respected history of evaluation. Many luminaries at AEA (American Evaluation Association) have spent/are spending their professional careers at GAO. The GAO has just published (January 2012) their handbook on evaluation. It is called, DESIGNING EVALUATIONS 2012 Revision. Nancy Kingsbury, a longtime AEA luminary, wrote the preface. For those of us who receive Federal money in any form (grants, contracts, Extension) this will be a worthwhile read. Fortunately, it is a relatively short read (text is 61 pages plus another 7 pages of appended material). This manuscript explains the “official” federal view of evaluation. It is always good to know what is expected. I highly recommend this read. The worst it could be is good bedtime reading…zzzzzz-zz-z.
I have a quandary. Perhaps you have a solution.
I am the evaluator on a program where the funding agency wants clear, measurable, and specific outcomes. (OK, you say) The funding agency program people were asked to answer the question, “What do you expect to happen as a result of the program?”
These folks responded with a programmatic equivalent of “world peace.” I virtually rolled my eyes. IMHO there was no way that this program would end in world peace. Not even no hunger (a necessary precursor to world peace). After suggesting that perhaps that goal was unattainable given the resources and activities intended, they came out of the fantasy world in which they were living and said, realistically, “We don’t know, exactly.” I probed further. The sites (several) were all different; the implementation processes (also several) were all different; the resources were all different (depending on site); and the list goes on. Oh, and the program was to be rolled out soon in another site without an evaluation of the previous sites. BUT THEY WANTED CLEAR, MEASURABLE, AND SPECIFIC OUTCOMES.
What would you do in this situation? (I know what I proposed–there was lukewarm response. I have an idea what would work–although the approach was not mainstream evaluation and these were mainstream folks.) So I turn to you, Readers. What would you do? Let me know. PLEASE.
Oh, and Happy Groundhog’s Day. I understand there will be six more weeks of winter (there was serious frost this morning in Corvallis OR).