change « Evaluation is an Everyday Activity

Good ideas. Maybe.

Did I get good ideas? Maybe.

I recently returned (Saturday,November 11, 2017, late) from the 2017 annual American Evaluation Association conference. This year the meeting was held in Washington, D. C. (Thank you Lance Wyman, for this photo.) I realize that this is not the iconic view of D.C. that one imagines (like this: .) It was fall and it was mostly clear. I did get to the zoo as part of the conference.

As you know, I determine if a conference is good by seeing three long time friends, meeting three new people I want to see again, and getting three new ideas . This year was bitter sweet. Yes, I did see three long time friends (however, there were only 10). Used to be that I could not go across the lobby without seeing someone I knew well and wanted to see again. This year, many friends (both professional and personal) were not there–they had retired; they were frail and not traveling; they had died and I thought of my own mortality and realized that I had less time to take breaths, even those that take my breath away. I did not meet (although I did interact with young people) three new people I wanted to see again. I think I got only two good ideas–maybe three; hard to say.

Continue reading →

Love.

Ah…love. Tomorrow is Valentine’s Day, the day traditionally set aside for lovers–you know the lovey dovey kind. And if you forgot…watch out.

It is the day when Saint Valentine (officially Saint Valentine of Terni), a widely recognized third-century Roman saint, has his feast day. Since the High Middle Ages it is associated with a tradition of courtly love. It is said that Valentine’s Day was established to counteract the pagan celebration of Lupercalia. There is much we do not know about St. Valentine.

Not courtly love.

I want to talk about a different kind of love (and I do not mean the various definitions of that word). I want to talk about your calling; your passion.

A good friend of mine said: Know what your calling is, your vocation, and follow it faithfully.

She also said in that same missive: “When you are most disgruntled, take a moment of conscious breath or five moments of conscious play!”

This is the love I’m talking about. The love for your calling; your vocation (passion).

And what to do when you feel disgruntled (breathe/play).

Passion.

Susan Kistler, AEA Executive Director Emeritus, shares perhaps an important message about love:

“Success is made manifest in health and happiness, confidence that you are loved and the capacity to love with others.”

That is passion.

How does that relate to evaluation?

We are all evaluators and live and work by criteria, whether they are implicit or explicit. Our passions are found in the criteria. We continue that passion for long in our lives–some of us because of family responsibilities; some of us because it is fun. When we get tired, we stop. We still have the passion and that passion comes out when we least expect it. Because once an evaluator (whether formally or not), always an evaluator.

So celebrate your passion tomorrow. And remember to breath…or play!

Resolutions. Renewal.

Renewal is appropriate for the new year. So are resolutions.

It has been over a month since I blogged here. And the longer I wait for inspiration, the harder it is to write.

But I’m waiting for inspiration. Really difficult, to be sure.

We all know that resolutions have a great tendency to fail.

So how can one find renewal in these difficult times?

Perhaps it is time to re-evaluate your priorities.

Priorities can change. Depending on circumstances.

Is this a time for you to be more articulate?

Or a time to be more proactive?

A time to be more (fill in the blank)?

Writer’s block

Sheila Robinson the sometime Saturday contributor for AEA 365 Continue reading →

Evaluation is political. I am reminded of that fact when I least expect it.

In yesterday’s AEA 365 post, I am reminded that social justice and political activity may be (probably are) linked; are probably sharing many common traits.

In that post the author lists some of the principles she used recently:

Evaluation is a political activity.
Knowledge is culturally, socially, and temporally contingent.
Knowledge should be a resource of and for the people who create, hold, and share it.
There are multiple ways of knowing (and some ways are privileged over others).

Evaluation is a trans-discipline, drawing from many many other ways of thinking. We know that politics (or anything political) is socially constructed. We know that ‘doing to’ is inadequate because ‘doing with’ and ‘doing as’ are ways of sharing knowledge. (I would strive for ‘doing as’.) We also know that there are multiple ways of knowing.

(See Belenky , Clinchy [with Belenky] , Goldberger , and Tarule, Basic Books, 1986 as one.)

(See: Gilligan , Harvard University Press, 1982; among others.)

How does evaluation, social justice, and politics relate?

What if you do not bring representation of the participant groups to the table?

If they are not asked to be at the table or for their opinion?

What if you do not ask the questions that need to be asked of that group?

To whom ARE your are your questions being addressed?

Is that equitable?

Being equitable is one aspect of social justice. There are others.

Evaluation needs to be equitable.

I will be in Atlanta next week at the American Evaluation Association conference. atlanta-georgia-metropolitan ‘

Maybe I’ll see you there!

my .

molly.

Process is the “how”.

Recently, reminded of the fact that process is the “how”, I had the opportunity to help develop a Vision

and a Mission statement.

The person who was facilitating the session provided the group with clear guidelines.

The Vision statement, defined as “the desired future condition”, will happen in 2-5 years (i.e., What will change?). We defined the change occurring (i.e., in the environment, the economy, the people). The group also identified what future conditions would be possible. We would write the vision statement so that it would happen within 2-5 years, be practical, be measurable, and be realistic. OK…

And be short…because that is what vision statements are.

The Mission statement (once the Vision statement was written and accepted) defined “HOW” we would get to the vision statement. This reminded me of process–something that is important in evaluation. So I went to my thesaurus to find out what that source said about process. Scriven to the rescue, again.

Process Evaluation

Scriven, in his Evaluation Thesaurus, defines process as the activity that occurs “…between the input and the output, between the start and finish”. Sounds like “how” to me. Process relates to process evaluation. I suggest you read the section on process evaluation on page 277 in the above mentioned source.

Process evaluation rarely functions as the sole evaluation tool because of weak connections between “output quantity and quality”. Process evaluations will probably not generalize to other situations.

However, PROCESS evaluation “…must be looked at as part of any comprehensive evaluation, not as a substitute for inspection of outcomes…” The factors include “the legality of the process, the morality, the enjoyability, the truth of any claims involved, the implementation…, and whatever clues…” that can be provided.

Describing “how ” something is to be done is not easy. It is not output nor outcome. Process is the HOW something will be accomplished if you have specific inputs . It happens between the inputs and the outputs.

To me, the group needs to read about process evaluation in crafting the mission statement in order to get to the HOW.

my two .

molly.

AEA365 is honoring living evaluators for Labor Day (Monday, September 5, 2016).

Some of the living evaluators I know (Jim Altschuld, Tom Chapel, Michael Patton, Karen Kirkhart, Mel Mark, Lois-Ellin Datta, Bob Stake); Some of them I don’t know (Norma Martinez-Rubin, Nora F. Murphy, Ruth P. Saunders, Art Hernandez, Debra Joy Perez). One I’m not sure of at all (Mariana Enriquez). Over the next two weeks, AEA365 is hosting a recognition of living evaluator luminaries.

The wonderful thing is that this give me an opportunity to check out those I don’t know; to read about how others see them, what makes them special. I know that the relationships that develop over the years are dear, very dear.

I also know that the contributions that these folks have made to evaluation cannot be captured in 450 words (although we try). They are living giants, legends if you will.

These living evaluators have helped move the field to where it is today. Documenting their contributions to evaluation enriches the field. We remember them fondly.

If you don’t know them, look for them at AEA ’16 in Atlanta . Check out their professional development sessions or their other contributions (paper, poster, round-table, books, etc). Many of them have been significant contributors to AEA; some have only been with AEA since the early part of this century. All have made a meaningful contribution to AEA.

Many evaluators could be mentioned and are not. Sheila B. Robinson suggests that “…we recognize that many, many evaluators could and should be honored as well as the 13 we feature this time, and we hope to offer another invitation next year for those who would like to contribute a post, so look for that around this time next year, and sign up!

Evaluators honored

Thomas J. Chapel

James W. Altschuld Thomas J. Chapel

Norma Martinez-Rubin Michael Quinn Patton

Nora F. Murphy Ruth P. Saunders

Art Hernandez Karen Kirkhart

Melvin Mark

Mel Mark Lois-Ellin Datta

Debra Joy Perez Bob Stake

Mariana Enriquez (Photo not known/found)

my .

molly.

Probable? Maybe. Making a difference is always possible.

Oxford English Dictionary defines possible as capable of being (may/can exist, be done, or happen). It defines probable as worthy of acceptance, believable.

Ray Bradbury : “I define science fiction as the art of the possible. Fantasy is the art of the impossible.”

Somebody asked me what was the difference between science fiction and fantasy. Certainly the simple approach is that science fiction deals with the possible (if you can think it, it can happen). Fantasy deals with monsters, fairies, goblins, and other mythical creatures, i.e., majic and majical creatures.

(Disclaimer: I personally believe in majic; much of fantasy deals with magic.) I love the Arthurian legend (it could be fantasy; it has endured for so long it is believable). It is full of majic. I especially like the Marion Zimmer Bradley book, The Mists of Avalon . (I find the feminist perspective refreshing.)

Is fantasy always impossible as Bradbury suggests, or is it just improbable? (Do the rules of physics apply?) This takes me back to Bradbury’s quote and evaluation after the minor digression. Bradbury also says that “Science fiction, again, is the history of ideas, and they’re always ideas that work themselves out and become real and happen in the world.” Not unlike evaluation. Evaluation works itself out and becomes real and happens. Usually.

Evaluation and the possible.

Often, I am invited to be the evaluator of record after the program has started. I sigh. Then I have a lot of work to do. I must teach folks that evaluation is not an “add on” activity. I must also teach the folks how to identify the difference the program made. Then there is the issue of outputs (activities, participants) vs. outcomes (learning, behavior, conditions). Many principal investigators want to count differences pre-post.

Does the “how many” provide a picture of what difference the program made? If you start with no or few participants and you end with many participants, have you made a difference? Yes, it is possible to count. Counts often meet reporting requirements. They are possible. So is documenting the change in knowledge, behavior, and conditions. It takes more work and more money. It is possible. Will you get to world peace? Probably not. Even if you can think it. World peace may be probable; it may not be possible (at least in my lifetime).

my .

molly.

Recently, I read a Washington Post article on innovation. The WP interviewed Calestous Juma (see below), author of the July, 2016 book, “Innovation and Its Enemies:Why People Resist New Technologies.” The book was published by Oxford University Press (prestigious, to be sure). Priced at $29.95 plus an estimated s/h of $5.50, it sounds like a good purchase. There is quite a bit of information about the book and the author on the Oxford University Press site. This prompted me to think about what has changed in evaluation (not just technology) over the last 30+ years. First, though, I want to talk about the article.

Article by Juma.

Calestous Juma (Courtesy of Harvard)

Juma says that “people don’t fear innovation simply because the technology is new, but because innovation often means losing a piece of their identity or lifestyle.” He goes on to say that “Innovation can also separate people from nature or their sense of purpose.” He argues that these two things are fundamental to the humon experience. I have talked about sense of purpose previously. I wonder if nature is part of purpose or if a sense of purpose comes from a person’s nature? Continue reading →

The only way to do great work is to love what you do. If you have not found it yet, keep looking. Do not settle. ~~Steve Jobs.

Last week I wrote about an epiphany I had many years ago, one in which I did not settle.

I made choices about the work I did. I made choices about the life I lived. I did not settle.

It is an easy life to “go with the flow”; to settle, if you will. Convenience is not always the best way even though it might be the easiest. Did I do great work? I don’t know. Did I hear stories of the work I did? I was told after the fact that I had made a difference because of the work I had done. Perhaps, making a difference is doing great work. Perhaps.

However, this quote from Steve Jobs reminded me that loving what one does is important, even if one does not do “great work”. If one does not love what one does, one needs to do what one loves. Continue reading →

The Highest Appreciation

As we express our gratitude, we must never forget that the highest appreciation is not to utter words, but to live by them.

– John F. Kennedy

Gratitude must be a habit. Each day needs to be began and ended with gratefulness. Then if you can live by that gratefulness, you will utter the words and be grateful. That is what evaluation is all about–holding to the higher ground. Not just doing something to get it done; doing something (in this case the evaluation) because it is right as you know it today, in this moment, under these circumstances.

Doing evaluation just for the sake of evaluating, because it would be nice to know, is not the answer. Yes, it may be nice to know; does it make a difference? Does the program (policy, performance, product, project, etc.) make a difference in the lives of the participants. As a social scientist, it is important for me to look at the “social” side of what I do; that means dealing with people, the participants, you know the social part. I want to determine what the participants are thinking, feeling, doing. That means, I must walk my talk. And be grateful.

There are lots of resources available that help the nascent evaluator do just that. My recommendation is to start with Jody Fitzpatrick’s volume . I would also check out the American Evaluation Association site. There is a lot of information available to non-members (becoming a member is worth the cost). Then depending on what you specifically want to know, let me know. I’ll suggest references to you.

molly.

Evaluation is an Everyday Activity

Program Evaluation Discussions

Tag Archives: change

Good ideas.

Good ideas. Maybe.

Love. Passion. Success.

Love.

Not courtly love.

Passion.

Renewal

Resolutions. Renewal.

Writer’s block

Evaluation is political.

Process is the “how”

Process is the “how”.

Process Evaluation

Living evaluators…

Evaluators honored

Possible? You bet!

Probable? Maybe. Making a difference is always possible.

Evaluation and the possible.

Innovation

Article by Juma.

Settling

Gratitude

Contact Info