Tag Archives: critical thinking

Evaluability assessment

Posted on by

Evaluability assessment.

I get my ideas to blog from a lot of places. One place I get ideas are from other blogs.

The content of this blog was Evaluability Assessment.

The blog author says that evaluability assessments tend to cover the following topics:

  • Clarity of the intervention and its objectives: Is there a logical and clear theory of change that articulates how and under what conditions intervention activities influence particular processes of change?
  • Availability of data: which data are available that can be used in assessing the merit and worth of the intervention (e.g. generated by the intervention, external data sets, policy and academic literature)
  • Stakeholder interest and intended use: to what extent is there a clear interest (and capacity) among stakeholders to use the evaluation’s findings and recommendations in strategic decision-making, program improvement, learning about what works, etc.?

What, you ask, is evaluability assessment?

You certainly can go to the blog and read what it says there. OR…You can go to Scriven’s book and read the history on page 138 .

Suffice it to say that evaluability is the extent to which they (projects and programs) can be evaluated. Scriven goes on to say: “It should be thought of as the first commandment of accountability or as the last refinement of Popper’s (Sir Karl Raimond Popper) requirement of falsifiability.

 

Sources.

I learned about evaluability assessment (EA) from Midge Smith (shown here with her husband Carl Wisler) in her book by the same name (published by Springer, search for it by title).  She says that EA is “…a diagnostic and prescriptive tool for improving programs and making evaluations more useful.” Like all tools used in evaluation, it is systematic and describes the structure of a program.

There is a newer volume  of that name. It is by Michael S. Trevisan and Tamara M. Walser (they do an AEA365 blog on that topic). It is not, unfortunately, on my shelf.  The blurb that accompanies the book (by the publisher, Sage) says: “Evaluability assessment (EA) can lead to development of sound program theory, increased stakeholder involvement and empowerment, better understanding of program culture and context, enhanced collaboration and communication, process and findings use, and organizational learning and evaluation capacity building.”

More detail than Midge offers, then her book is copyrighted in 1989.

EA is getting a lot of press lately (you may need to search for evaluability assessment when you go to AEA365).

I find it amazing how previously important things (EA) are now once again in vogue.

my .

 

 

Alternative facts?

Posted on by

Alternative facts.

Never. Never. has evaluation been questioned with the label of “alternative facts.”

Over the years, I have been very aware that evaluation is a political activity.

I have talked about evaluation being political (here, and here, and here, and here ).

But is it? Is it just another way of making the answer be what we want it to be? A form of alternative fact?

I’ve been an evaluator for a long time. I want to make a difference to the people who experience my programs (or the programs for which I’m consulting as an external evaluator). The thought that I might be presenting “alternative facts” is troublesome.

Did I really determine that outcome? Or is the outcome bogus? Liars use statistics, you know. (This is a paraphrase of a quote that Mark Twain attributed to Benjamin Disraeli.)

Big news brings out the fakers. But are evaluation results “big news”? Or…do people not want to hear what is actually happening, what the outcome really is?

Reminds me of 1984 ( George Orwell): War is peace. Freedom is slavery. Ignorance is strength (the English Socialist Party–aka. INGSOC). Kevin Siers added, in his cartoon of Sean Spicer,  “2017 is 1984”.  Two contradictory ideas existing at the same time as correct.

Statistics.

Statistics is a tool that evaluators use on a regular basis. It allows evaluators to tease apart various aspects of a program. The “who” , the “what”, the “when”, maybe even the “why”. Statistics can certainly help determine if I made a difference  But how I see statistics may not be how you see them, interpret them, use them. Two people can look at a set of statistics and say they do not agree. Is that an example of alternative facts?

Bias.

Everyone comes to any program with preconceived bias. You, the evaluator, want to see a difference. Preferably a statistically significant difference, not just a practical significance (although that would be nice as well).

Even if you are dealing with qualitative data, and not with quantitative data yielding statistics, you come to the program with bias. Objectivity is not an option. You wouldn’t be doing the program if you didn’t think that the program will make a difference. Yet, the individuals who have funded the program (or in some other way are the folks who get the final report) can (and do) not accept the report as it is written. That is not what they want to see/hear/read. Does that make the report alternative facts? Or is bias speaking without acknowledging that bias?

Perhaps Kierkegaard is right.

There are only two ways you can be fooled.

 

my .

molly.

Thinking

Posted on by

Thinking. We do it all the time (hopefully). It is crucial to making even the smallest decisions (what to wear, what to eat), and bigger decisions (where to go, what to do). Given this challenging time, even news watchers would be advised to use evaluative and critical thinking.  Especially since evaluation is an everyday activity.

This graphic was provided by WNYC. (There are other graphics; use your search engine to find them.)This graphic makes good sense to me and this applies to almost every news cast (even those without a shooter!). Continue reading →

Learning. Real learning.

Posted on by

I want to talk about learning. Real learning. This week I am borrowing a blog from another writer intact. I have never done this. True, I have taken parts of blogs and quoted them. This blog post from the blog called “adapting to perpetual beta” by Harold Jarche is applied here in its entirety because I think the topic is important. I have added the visuals except for the Rodin, which was in the original post.

Yes, it relates to evaluation. We learn (those who value evaluation) throughout our careers. The various forms of learning are engaged (see: Edgar Dale who designed the learning cone though not with percentages that are usually attributed to the styles).Cone of learning(This particular version was developed by Bruce Hyland based on Dale’s work.) When you read the post below, think about how you learn. Engages? Reflective?

real learning is not abstract

Posted 2016-06-20

Are we entering an era that heralds ‘The End of Reflection’, as this NY Times article suggests?

Continue reading →

Settling

Posted on by

The only way to do great work is to love what you do. If you have not found it yet, keep looking. Do not settle. ~~Steve Jobs.

Last week I wrote about an epiphany I had many years ago, one in which I did not settle. don't settle cropped

I made choices about the work I did. I made choices about the life I lived. I did not settle.

It is an easy life to “go with the flow”; to settle, if you will. Convenience is not always the best way even though it might be the easiest. Did I do great work? I don’t know. Did I hear stories of the work I did? I was told after the fact that I had made a difference because of the work I had done. Perhaps, making a difference is doing great work. Perhaps.

However, this quote from Steve Jobs reminded me that loving what one does is important, even if one does not do “great work”. If one does not love what one does, one needs to do what one loves.love Continue reading →

Making a difference

Posted on by

We close every rehearsal and concert with the song, “Be the change”. Using the words from Gandhi, I try to remember to make a difference;difference 2 to be the change I want to see in the world.

That is not easy. I ride my bike all the time. (Yep. Really.) I compost. I grow my own vegetables in the summer and support my farmers’ market and CSA (both of which, thankfully, run through Thanksgiving). But I ask my self, “Am I making a difference?” make a difference Continue reading →

Evaluation Pioneers

Posted on by

Having written about evaluation history previously, I identified  those who contributed, not those who could be called evaluation pioneers; rather those who had influenced my thinking.  I think it is noteworthy to mention those evaluation pioneers who set the field on the path we see today, those whom I didn’t mention and need to be. As a memorial (it is Memorial Day weekend Memorial-Day-weekend, after all), Michael Patton (whom I’ve mentioned previously) is coordinating an AEA365 to identify and honor those evaluation pioneers who are no longer with us. (Thank you, Michael). The AEA365 link above will give you more details.  I’ve also linked the mentioned evaluation pioneers that have been remembered. Some of these pioneers I’ve mentioned before; all are giants in the field; some are dearly loved as well. All those listed below have died. Patton talks about the recent-dead, the sasha, and the long-dead, the zamani. He cites the Historian James W. Loewen when he makes this distinction. Some of the listed are definitely the sasha (for me); some are zamani (for me). Perhaps photos will help (for whom photos could be found) and dates. There are Continue reading →

Taking a stand.

Posted on by

I just got back from a road trip across Southern Alabama with my younger daughter.southern alabama We started from Birmingham and drove a very circuitous route ending in Mobile and the surrounding areas, then returned to Birmingham for her to start her second year at Birmingham-Southern College.

As we traveled, I read a book by Bill McKibben (one of many) called Oil and Honey: The Education of an Unlikely Activist. It is a memoir, a personal recounting of the early years of this decade, which corresponded with the years my older daughter was in college (2011-2014). I met Bill McKibben, who, in 2008, is credited with starting the non-profit, 350.0rg, and is currently listed as “senior adviser and co-founder”. He is a passionate, soft-spoken man, who beleives that the world is on a short fuse. He really seems to believe that there is a better way to have a future. He, like Gandhi, is taking a stand.  Oil and Honey puts into action Gandhi’s saying about being the change you want to seegandhi and change. As the subtitle indicates, McKibben is an unlikely activist. He is a self-described non-leader who led and advises the global effort to increase awareness of climate change/chaos. When your belief is on the line, you do what has to be done.

Evaluators are the same way. When your belief is on the line, you do what has to be done. And, hopefully, in the process you are the change that you want to see in the world. But know it cannot happen one pipeline at a time. The fossil fuel industry has too much money. So what do you do? You start a campaign. That is what 350.org has done:  “There are currently fossil fuel divestment campaigns at 308 colleges and universities, 105 cities and states, and 6 religious institutions.”(Wikipedia, 350.0rg) (Scroll down to the heading “Fossil Fuel Divestment” to see the complete discussion.) Those are clear numbers, hard data for consumption. (Unfortunately, the  divestment campaign at OSU failed.)

So I see the question as one of impact, though not specifically world peace (my ultimate impact). If there is no planet on which to work for world peace, there in no need for world peace. Evaluators can help. They can look at data critically. They can read the numbers. They can gather the words. This may be the best place for the use of pictures (they are, after all, worth 1000 words).  Perhaps by combining efforts, the outcome will be an impact that benefits all humanity and builds a tomorrow for the babies born today.

my two cents.

molly.