Audience.

We all have an audience (unless we are a reclusive hermit). The issue is what kind an audience.

An absent audience?

An all inclusive audience?

A captive audience?

An exclusive audience?

Reminds me of the saying, “You can please some of the people, some of the time; all of the people some of the time and not all of the people all of the time.”

So when I teach evaluation (or try to teach evaluation–evaluation is a scary concept), I begin by saying that we are all evaluators, that we evaluate daily. It is truly an everyday activity.

Some people engage at that point. Others are skeptical. Others are deniers.

Like the old saying–some of the people…

Unconsciously (maybe consciously) you identify criteria that will help you make a decision.

That decision will provide you with a decision tree, to be able to make decisions throughout the day.

Criteria.

The other day, I had the opportunity to explore a “teacher-made” test; to determine if it had face validity. The audience was similar to and different from the audience who will ultimately take the test (survey, actually). I needed to capture the concerns about the instrument; to determine if it did what it was supposed to do; and wording issues.

Then a decision needed to be made about its use. We were using the TOP model (Bennett and Rockwell, 1995*).  Specifically the KASA portion of the model.  This model can be used for program planning/development AND program evaluation/performance. Just one  example of a logic model.

There were concerns that the “knowledge” wasn’t specific enough; that the words used were not clear; that “skills” were not specific enough; that not all response options were included in the response set. Some KASA questions   could be easily fixed (the response set); some were not (the specificity issue). I sent the compiled responses to the people in charge of the project. I did not make a decision on the specificity. Perhaps if I knew more about the topic, I could have. Obviously, the survey needed changing.

Decisions.

Is all evaluation about making decisions? It all depends.

Evaluation is determining the merit, value, worth of a program or project.

You want to find out if you have made a difference in the lives of the target audience.

If you can answer that, you have captured your audience.

Citation.

*Bennett, C. & Rockwell, K. (1995, December). Targeting outcomes of programs (TOP): An integrated approach to planning and evaluation. Unpublished manuscript. Lincoln, NE: University of Nebraska.

https://www.uaf.edu/files/ces/reporting/logicmodel/TOP.pdf

 

 

To handle yourself, use your head; to handle others, use your heart. ~~Eleanor Roosevelt

This quote is often attributed to Eleanor Roosevelt (1884-1962); there are some sites that attribute this quote to Donald Anderson Laird (1897-1969), a psychologist and author (no photo found). Probably, more accurate. I’m not sure that the origin of the saying is really important. It may be enough to keep in mind the saying itself. (I know–how does this relate to evaluation? Trust me, it does.)

Before I was an evaluator, I was a child therapist (I also treated young women). I learned many skills as a therapist that have served me well as an evaluator. Skills like listening, standing up for your self, looking at alternatives. Which leads me to this saying. I had to “handle” others all the time at the same time I had to “handle” my self. I could not “blow up” when reprimanded. I could not become discouraged when someone (the client, the funder) criticized me. I had to learn to laugh when the joke was on me. I had to keep my spirits up when things went wrong. I had to keep cool in emergencies. I had to learn to tune out gossip and negative comments from others. This was a hard time for me. I tend to be passionate when I have an opinion; I have/had opinions (often).

As an evaluator, I am still passionate. Once my evaluation “on” button is pushed, it is hard to turn it off. Yet I still have to handle people. This morning, for example, I met with a fellow faculty member. I had to listen. I had to look for (and at) alternatives. I “handled” with my head; remember, I am passionate about evaluation. I provided her with alternatives and followed through with those alternatives. I handled with my heart.

When others are involved (and in evaluation there are always others), they must be handled with care, with the heart. It goes back to the standards (propriety)  and the guiding principles   (integrity/honesty, direct respect for people, and responsibilities for general and public welfare).  In the current times, it is especially important to have direct respect for people. All people. (Regardless of race, ethnicity, religion, gender identity, sex, national origin, veteran status, and disability.) To be honest and have integrity. One way to make sure you have integrity is to handle with your heart.

 

The person without a purpose is like a ship without a rudder. (Thomas Carlyle)

There is much written about finding your purpose if life.  Songs are written about purpose; self-help books are written about purpose; businesses are devoted to the concept; jewelry, leadership, among other things, all focus on purpose.

So how do you find purpose? How do you know what your are “supposed” to do in this life? How does that relate to evaluation? Finding your purpose can be really confusing.  Let me share a story with you.

I lived in Birmingham, AL in the 80s and 90s. Birmingham is the only place I have lived (and I’ve lived many places) where if you woke up on the first day of spring, EVERYTHING would be in bloom. Everything! In Oregon, spring creeps up on you (a wonderful experience, to be sure). In Minnesota, it feels like it is spring one day and summer the next (or if you are not lucky, winter, again). In Tucson, spring happens in February and if you blink you miss it (well, almost). So I was marveling one day around the first day of spring how wonderful life was and I had an epiphany. I conceptualized what were the three things I wanted to do in this life. I wanted to do good work. I wanted to be a good friend. I wanted to grow spiritually. (I knew that being a boss was not for me, even though it came with perks.)

I had just finished a doctoral program in program evaluation. I realized that I would be “in the trenches” a long time and would spend most of my career doing evaluation work (as opposed to teaching evaluation, researching evaluation, writing about evaluation). I saw that as my purpose. To do good work–good evaluation work.

So what does it mean to do “good evaluation work”? Continue reading →