The only way to do great work is to love what you do. If you have not found it yet, keep looking. Do not settle. ~~Steve Jobs.

Last week I wrote about an epiphany I had many years ago, one in which I did not settle. don't settle cropped

I made choices about the work I did. I made choices about the life I lived. I did not settle.

It is an easy life to “go with the flow”; to settle, if you will. Convenience is not always the best way even though it might be the easiest. Did I do great work? I don’t know. Did I hear stories of the work I did? I was told after the fact that I had made a difference because of the work I had done. Perhaps, making a difference is doing great work. Perhaps.

However, this quote from Steve Jobs reminded me that loving what one does is important, even if one does not do “great work”. If one does not love what one does, one needs to do what one loves.love Continue reading

We close every rehearsal and concert with the song, “Be the change”. Using the words from Gandhi, I try to remember to make a difference;difference 2 to be the change I want to see in the world.

That is not easy. I ride my bike all the time. (Yep. Really.) I compost. I grow my own vegetables in the summer and support my farmers’ market and CSA (both of which, thankfully, run through Thanksgiving). But I ask my self, “Am I making a difference?” make a difference Continue reading

Having written about evaluation history previously, I identified  those who contributed, not those who could be called evaluation pioneers; rather those who had influenced my thinking.  I think it is noteworthy to mention those evaluation pioneers who set the field on the path we see today, those whom I didn’t mention and need to be. As a memorial (it is Memorial Day weekend Memorial-Day-weekend, after all), Michael Patton (whom I’ve mentioned previously) is coordinating an AEA365 to identify and honor those evaluation pioneers who are no longer with us. (Thank you, Michael). The AEA365 link above will give you more details.  I’ve also linked the mentioned evaluation pioneers that have been remembered. Some of these pioneers I’ve mentioned before; all are giants in the field; some are dearly loved as well. All those listed below have died. Patton talks about the recent-dead, the sasha, and the long-dead, the zamani. He cites the Historian James W. Loewen when he makes this distinction. Some of the listed are definitely the sasha (for me); some are zamani (for me). Perhaps photos will help (for whom photos could be found) and dates. There are Continue reading

I am a social scientist. I look for the social in the science of what I do.

I am an evaluator as a social scientist. I want to determine the merit, worth, value of what I do. I want to know that the program I’m evaluating (or offering) made a difference. (After all, the root of evaluation is value.)

Keeping that in mind has resulted (over the years) in the comment, “no wonder she is the evaluator” when I ask an evaluative question. So I was surprised when I read a comment by a reader that implied that it didn’t matter. The reader said, “The ugly truth is, it does not matter if it makes a difference. Somewhere down the road someone will see your post and may be it will be useful for him.” (Now you must know that I’ve edited the comment, although the entire comment doesn’t support my argument:  Evaluators need to know if the program made a difference.)

So the thought occurred to me, what if it didn’t make a difference? What if the program has no value? No worth? No merit? What if by evaluating the program you find that it won’t be useful for the participant? What does that say about you as an evaluator? You as a program designer? You as an end user? Is it okay for the post to be useful “somewhere down the road”? Is blogging truly “a one way channel to transfer any information you have over the web.” How long can a social-scientist-always-looking-at-the-social continue to work when the information goes out and rarely comes back? I do not know. I do know that blogging is hard work. After six and one-half years of writing this blog almost weekly,  writer’s block is my constant companion.writers-block 2 (although being on a computer, I do not have a pile of paper, just blank screens). So I’m turning to you, readers:

Does it make a difference whether I write this blog or not?

Am I abdicating my role as an evaluator when I write the blog?

I don’t know. Over the years I have gotten some interesting comments (other than the “nice job” “keep up the work” types of comments). I will pause (not in my writing; I’ll continue to do that) and think about this. After all, I am an evaluator wanting to know what difference this program makes.

my two cents.

molly.

The Highest Appreciation

As we express our gratitude, we must never forget that the highest appreciation is not to utter words, but to live by them.

– John F. Kennedy

Gratitude must be a habit. Each day needs to be began and ended with gratefulness. Then if you can live by that gratefulness, you will utter the words and be grateful. That is what evaluation is all about–holding to the higher ground. Not just doing something to get it done; doing something (in this case the evaluation) because it is right as you know it today, in this moment, under these circumstances.

Doing evaluation just for the sake of evaluating, because it would be nice to know, is not the answer. Yes, it may be nice to know; does it make a difference? Does the program (policy, performance, product, project, etc.) make a difference in the lives of the participants. As a social scientist, it is important for me to look at the “social” side of what I do; that means dealing with people, the participants, you know the social part. I want to determine what the participants are thinking, feeling, doing. That means, I must  walk my talk. And be grateful.

 

There are lots of resources available that help the nascent evaluator do just that. My recommendation is to start with Jody Fitzpatrick’s volume fitzpatrick book 2. I would also check out the American Evaluation Association site. There is a lot of information available to non-members (becoming a member is worth the cost). Then depending on what you specifically want to know, let me know. I’ll suggest references to you.

my two cents

molly.

 

Alan Rickman quote

Alan Rickman Alan-Rickman died this month. He was an actor of my generation; one that provided me with much entertainment. I am sad. Then I saw this quote on the power of stories. How stories explain. How stories can educate. How stories can help reduce bias.  And I am reminded how stories are evaluative.

Dick Krueger dick-1997 did a professional development session (then called a “pre-session”) many years ago. It seems relevant now. Of course, I couldn’t find my notes (which were significant) so I did an online search, using “Dick Krueger and stories” as my search terms. I was successful! (See link.) When I went to the link, he had a whole section on story and story telling. What I remember most about that session is what he has listed under “How to Analyze the Story”. Specifically the four points he lists under problems with credibility:

  • Authenticity – Truth
  • Accuracy – Memory Problems
  • Representativeness and Sampling
  • Generalizability / Transferability

The next time you tell a story think of it in evaluative terms. And check out what Dick Krueger has to say. Continue reading

The US just celebrated Thanksgiving, the annual day of thankfulness. Thanksgiving Canada celebrated in mid October (October 12). Although other countries celebrate versions of the holiday, originally the US and Canada celebrated in honor of the previous harvest.

Certainly, the Guiding Principles Guiding principles and the Program Evaluation Standards program evaluation standards provide evaluators with a framework to conduct evaluationEvaluation3 work. The work for which I am thankful.

Continue reading

I got back to the office Monday after spending last week in Chicago at the AEA annual conference, Evaluation 2015.Evaluation 2015 theme Next year AEA will be in Atlanta, October 24-29, 2016.  atlanta-georgia-skyline Mark your calendars!

I am tired. I take a breath (many breaths), try to catch up (I don’t), and continue to read my email (hundreds of email). I’m sure there are some I will miss–I always do.  In the meantime, I process what I experienced. And pass the conference through my criteria for a successful conference: Did I

  1. See three (and visit with) long time friends: yes.
  2. Get three new ideas: maybe.
  3. Meet three new people I’d like to add to my “friendlies” category: maybe.

Why three. Seemed like a good number; more than one (not representative) and less than five (too hard to remember). Continue reading

My friend and colleague, Patricia Rogers, says of cognitive bias , “It would be good to think through these in terms of systematic evaluation approaches and the extent to which they address these.” This was in response to the article herecognitive bias The article says that the human brain is capable of 10 to the 16th power (a big number) processes per second. Despite being faster than a speeding bullet, etc., the human brain has ” annoying glitches (that) cause us to make questionable decisions and reach erroneous conclusions.”

Bias is something that evaluators deal with all the time. There is desired response bias, non-response bias, recency and immediacy bias, measurement bias, and…need I say more? Isn’t evaluation and aren’t evaluators supposed to be “objective”? That we as evaluators behave in an ethical manner? That we have dealt with potential bias and conflicts of interest. That is where cognitive bias appear. And you might not know it at all. Continue reading