Nov
17
Filed Under (program evaluation) by Molly on 17-11-2017

Good ideas. Maybe.

Did I get good ideas? Maybe.

I recently returned (Saturday,November 11, 2017, late) from the 2017 annual American Evaluation Association conference. This year the meeting was held in Washington, D. C.  (Thank you Lance Wyman, for this photo.) I realize that this is not the iconic view of D.C. that one imagines (like this: .) It was fall and it was mostly clear. I did get to the zoo as part of the conference.

As you know, I determine if a conference is good by seeing three long time friends, meeting three new people I want to see again, and getting three new ideas . This year was bitter sweet. Yes, I did see three long time friends (however, there were only 10). Used to be that I could not go across the lobby without seeing someone I knew well and wanted to see again. This year, many friends (both professional and personal) were not there–they had retired; they were frail and not traveling; they had died and I thought of my own mortality and realized that I had less time to take breaths, even those that take my breath away. I did not meet (although I did interact with young people) three new people I wanted to see again. I think I got only two good ideas–maybe three; hard to say.

Read the rest of this entry »

Sep
14
Filed Under (program evaluation) by Molly on 14-09-2017

Evaluability assessment.

I get my ideas to blog from a lot of places. One place I get ideas are from other blogs.

The content of this blog was Evaluability Assessment.

The blog author says that evaluability assessments tend to cover the following topics:

  • Clarity of the intervention and its objectives: Is there a logical and clear theory of change that articulates how and under what conditions intervention activities influence particular processes of change?
  • Availability of data: which data are available that can be used in assessing the merit and worth of the intervention (e.g. generated by the intervention, external data sets, policy and academic literature)
  • Stakeholder interest and intended use: to what extent is there a clear interest (and capacity) among stakeholders to use the evaluation’s findings and recommendations in strategic decision-making, program improvement, learning about what works, etc.?

What, you ask, is evaluability assessment?

You certainly can go to the blog and read what it says there. OR…You can go to Scriven’s book and read the history on page 138 .

Suffice it to say that evaluability is the extent to which they (projects and programs) can be evaluated. Scriven goes on to say: “It should be thought of as the first commandment of accountability or as the last refinement of Popper’s (Sir Karl Raimond Popper) requirement of falsifiability.

 

Sources.

I learned about evaluability assessment (EA) from Midge Smith (shown here with her husband Carl Wisler) in her book by the same name (published by Springer, search for it by title).  She says that EA is “…a diagnostic and prescriptive tool for improving programs and making evaluations more useful.” Like all tools used in evaluation, it is systematic and describes the structure of a program.

There is a newer volume  of that name. It is by Michael S. Trevisan and Tamara M. Walser (they do an AEA365 blog on that topic). It is not, unfortunately, on my shelf.  The blurb that accompanies the book (by the publisher, Sage) says: “Evaluability assessment (EA) can lead to development of sound program theory, increased stakeholder involvement and empowerment, better understanding of program culture and context, enhanced collaboration and communication, process and findings use, and organizational learning and evaluation capacity building.”

More detail than Midge offers, then her book is copyrighted in 1989.

EA is getting a lot of press lately (you may need to search for evaluability assessment when you go to AEA365).

I find it amazing how previously important things (EA) are now once again in vogue.

my .

 

 

Sep
06
Filed Under (program evaluation) by Molly on 06-09-2017

LIBRARIES 

With them, we are amazing!

Without them, we humons are limited.

I can only speak for myself–I do not want to be limited. I am a library champion. (A library champion is someone who fosters public awareness on the extensive range of resources and services available at public, school, academic and special libraries nationwide.)

If you were to see my professional library, perhaps you would understand. (It looks like the photo above, only the shelves are white; I have four bookcases with extensions. The shelves are full.)

If you were to see my personal library, you would understand. Most of it is still in boxes.

Because I believe that literacy is important, When they were young and beginning to read, I gave four of my bookshelves to my daughters (who are now grown, although their bookshelves are still full, mostly, of books). One each is in their room; two are now in the quest room. Hence, my library is mostly still in boxes.

I’ve stopped buying books for personal use (I still get professional ones). I use the library to get hard copy. I have 13 books at home plus two book club books. I have 11 books on hold.

I have an iPad on which I have at least four books and an equal number on hold. (I read a lot.)

But libraries do so much more than provide us with books (still their most important function). They move information in new directions! And they have magic fingers on the keyboard. I would be lost without libraries.

The information to which they have access is astounding.

Libraries and evaluation.

I want to discuss my professional library. I have one whole bookcase (of seven plus shelves) which is filled with books relating to evaluation. Some books are in many editions. And that doesn’t include statistics books or measurement books or the hard-copy journals that have come over the years.

“Why?” you ask, do I have multiple versions of the same (well, almost) book, different editions? Ah. Perhaps one edition will provide the answer (to the puzzle) and the others do not. Does that mean that the answer is not relevant? No. Does that mean that the information is passe? Maybe. Maybe not. The book may be the seminal reference and needs to be sited. It may give a history that isn’t found any place else. It is important to see how the volume changes with each edition. Having multiple volumes adds value. (And the root of evaluation is value.)

Do I need all this? Probably not, especially in the age of the internet and access to all that it provides. Yet, there is something about hard copy; you know a book  (whether a paperback or not), with its binding, its smell, its feel, that cannot be duplicated on-line. Something that cannot be diminished. Something that definitely adds value, merit and worth.

 

“To live in the world without becoming aware of the meaning of the world is like wandering in a great library without touching the books.” ~~Manly P. Hall

My feeling exactly.

 

 

 

Feb
13
Filed Under (criteria, program evaluation) by Molly on 13-02-2017

Love.

Tomorrow is Valentine’s Day, the day traditionally set aside for lovers–you know the lovey dovey kind. And if you forgot…watch out.

It is the day when Saint Valentine    (officially Saint Valentine of Terni), a widely recognized third-century Roman saint, has his feast day. Since the  High Middle Ages it is associated with a tradition of courtly love. It is said that Valentine’s day was established to counteract the pagan celebration of Lupercalia. There is much we do not know about St. Valentine.

Not courtly love.

I want to talk about a different kind of love (and I do not mean the various definitions of  that word). I want to talk about  your calling; your passion.

A good friend of mine said:  Know what your calling is, your vocation, and follow it faithfully.

She also said in that same missive: “When you are most disgruntled, take a moment of conscious breath or five moments of conscious play!”

This is the love I’m talking about. The love for your calling; your vocation (passion).

And what to do when you feel disgruntled (breathe/play).

Passion.

Susan Kistler,  AEA Executive Director Emeritus, shares perhaps an important message about love:

“Success is made manifest in health and happiness, confidence that you are loved and the capacity to love with others.”

That is passion.

How does that relate to evaluation?

We are all evaluators and  live and work by criteria, whether they are implicit or explicit. Our passions are found in the criteria. We continue that passion for long in our lives–some of us because of family responsibilities; some of us because it is fun. When we get tired, we stop. We still have the passion and that passion comes out when we least expect it. Because once an evaluator (whether formally or not), always an evaluator.

So celebrate your passion tomorrow. And remember to breath…or play!

Jan
16
Filed Under (program evaluation) by Molly on 16-01-2017

Resolutions. Renewal.

Renewal is appropriate for the new year. So are resolutions.new-years-resolutions

It has been over a month since I blogged here. And the longer I wait for inspiration, the harder it is to write.

But I’m waiting for inspiration. Really difficult, to be sure.

We all know that resolutions have a great tendency to fail.

So how can one find renewal in these difficult times?

Perhaps it is time to re-evaluate your priorities.

Priorities can change. Depending on circumstances.

Is this a time for you to be more articulate?

Or a time to be more proactive?

A time to be more (fill in the blank)?

Writer’s block

Sheila Robinson the sometime Saturday contributor for AEA 365 Read the rest of this entry »

Nov
10
Filed Under (criteria, Methodology, program evaluation) by Molly on 10-11-2016

Trustworthiness. An interesting topic.

Today is November 9, 2016. An auspicious day, to be sure. (No, I’m not going to rant about November 8, 2016; just post this and move on with my living.) Keep in mind trustworthiness, I remind myself.

I had the interesting opportunity to review a paper recently that talked about trustworthiness. This caused me much thought as I was troubled by what was written. I decided to go to my source on “Naturalistic Inquiry”lincoln book . Given that the paper used a qualitative design, employed a case study method, and talked about trustworthiness, I wanted to find out more. This book was written by two of my long time evaluation guides, Yvonna Lincoln yvonna lincolnand Egon Gubaegon guba bw. (Lincoln’s name may be familiar to you from the Sage Handbook of Qualitative Research which she co-edited with Norman Denzin.)

Trustworthiness

On page 218, they talk about trustworthiness. About the conventional criteria for trustworthiness (internal validity, external validity, reliability, and objectivity). They talk about the questions underlying those criteria (see page 218).

They talk about how the criteria formulated by conventional inquirers are not appropriate for naturalistic inquiry. Guba (1981a) offers four new terms as they have “…a better fit with naturalistic epistemology.” These four terms and the terms they propose to replace are: Read the rest of this entry »

Sep
14
Filed Under (criteria, program evaluation) by Molly on 14-09-2016

Decisions

How do we make decisions when we think none of the choices are good?   decision

(Thank you for this thought, Plexus Institute.)

No, I’m not talking about the current political situation in the US. I’m talking about evaluation.

The lead for this email post was “Fixing the frame alters more than the view“. fixing the frame

Art Markman makes this comment (the “how do we make decisions…” comment) here. He says “If you dislike every choice you’ve got, you’ll look for one to reject rather than one to prefer—subtle difference, big consequences.” He based this opinion on research, saying that the rejection mind-set allows us to focus on negative information about options and fixate on the one with the smallest downside. Read the rest of this entry »

Jul
22
Filed Under (program evaluation) by Molly on 22-07-2016

Thinking. We do it all the time (hopefully). It is crucial to making even the smallest decisions (what to wear, what to eat), and bigger decisions (where to go, what to do). Given this challenging time, even news watchers would be advised to use evaluative and critical thinking.  Especially since evaluation is an everyday activity.

This graphic was provided by WNYC. (There are other graphics; use your search engine to find them.)This graphic makes good sense to me and this applies to almost every news cast (even those without a shooter!). Read the rest of this entry »

Jul
13
Filed Under (criteria, program evaluation) by Molly on 13-07-2016

I want to talk about learning. Real learning. This week I am borrowing a blog from another writer intact. I have never done this. True, I have taken parts of blogs and quoted them. This blog post from the blog called “adapting to perpetual beta” by Harold Jarche is applied here in its entirety because I think the topic is important. I have added the visuals except for the Rodin, which was in the original post.

Yes, it relates to evaluation. We learn (those who value evaluation) throughout our careers. The various forms of learning are engaged (see: Edgar Dale who designed the learning cone though not with percentages that are usually attributed to the styles).Cone of learning(This particular version was developed by Bruce Hyland based on Dale’s work.) When you read the post below, think about how you learn. Engages? Reflective?

real learning is not abstract

Posted 2016-06-20

Are we entering an era that heralds ‘The End of Reflection’, as this NY Times article suggests?

Read the rest of this entry »

Jul
08
Filed Under (program evaluation) by Molly on 08-07-2016

The only way to do great work is to love what you do. If you have not found it yet, keep looking. Do not settle. ~~Steve Jobs.

Last week I wrote about an epiphany I had many years ago, one in which I did not settle. don't settle cropped

I made choices about the work I did. I made choices about the life I lived. I did not settle.

It is an easy life to “go with the flow”; to settle, if you will. Convenience is not always the best way even though it might be the easiest. Did I do great work? I don’t know. Did I hear stories of the work I did? I was told after the fact that I had made a difference because of the work I had done. Perhaps, making a difference is doing great work. Perhaps.

However, this quote from Steve Jobs reminded me that loving what one does is important, even if one does not do “great work”. If one does not love what one does, one needs to do what one loves.love Read the rest of this entry »