Tag Archives: difference

Decisions. Decisions.

Posted on by

Decisions

How do we make decisions when we think none of the choices are good?   decision

(Thank you for this thought, Plexus Institute.)

No, I’m not talking about the current political situation in the US. I’m talking about evaluation.

The lead for this email post was “Fixing the frame alters more than the view“. fixing the frame

Art Markman makes this comment (the “how do we make decisions…” comment) here. He says “If you dislike every choice you’ve got, you’ll look for one to reject rather than one to prefer—subtle difference, big consequences.” He based this opinion on research, saying that the rejection mind-set allows us to focus on negative information about options and fixate on the one with the smallest downside. Continue reading →

Making a difference

Posted on by

We close every rehearsal and concert with the song, “Be the change”. Using the words from Gandhi, I try to remember to make a difference;difference 2 to be the change I want to see in the world.

That is not easy. I ride my bike all the time. (Yep. Really.) I compost. I grow my own vegetables in the summer and support my farmers’ market and CSA (both of which, thankfully, run through Thanksgiving). But I ask my self, “Am I making a difference?” make a difference Continue reading →

Gratitude

Posted on by

The Highest Appreciation

As we express our gratitude, we must never forget that the highest appreciation is not to utter words, but to live by them.

– John F. Kennedy

Gratitude must be a habit. Each day needs to be began and ended with gratefulness. Then if you can live by that gratefulness, you will utter the words and be grateful. That is what evaluation is all about–holding to the higher ground. Not just doing something to get it done; doing something (in this case the evaluation) because it is right as you know it today, in this moment, under these circumstances.

Doing evaluation just for the sake of evaluating, because it would be nice to know, is not the answer. Yes, it may be nice to know; does it make a difference? Does the program (policy, performance, product, project, etc.) make a difference in the lives of the participants. As a social scientist, it is important for me to look at the “social” side of what I do; that means dealing with people, the participants, you know the social part. I want to determine what the participants are thinking, feeling, doing. That means, I must  walk my talk. And be grateful.

 

There are lots of resources available that help the nascent evaluator do just that. My recommendation is to start with Jody Fitzpatrick’s volume fitzpatrick book 2. I would also check out the American Evaluation Association site. There is a lot of information available to non-members (becoming a member is worth the cost). Then depending on what you specifically want to know, let me know. I’ll suggest references to you.

my two cents

molly.

 

Conference evaluation

Posted on by

The OSU Extension Service conference started today (#OSUExtCon). There are concurrent sessions, plenary sessions, workshops, twitter feeds, (Jeff Hino is Tweeting), tours, receptions, and meal gatherings. There are lots of activities and they cover four days. But I want to talk about conference evaluation.

The thought occurs to me: “What difference is this making?” Ever the evaluator, I realize that the selection will be different next year (it was different last year) so I wonder how valuable is it to evaluate the concurrent sessions? Given that time doesn’t stand still (fortunately {or not, depending}), the plenary sessions will also be different. Basically, the conference this year will be different from the conference the next time. Yes, it will be valuable for the presenters to have feedback on what they have done and it will be useful for conference planners to have feed back on various aspects of the conference. I still have to ask, “Did it make a difference?”

A long time colleague of mine (formerly at Pennsylvania State University penn-state-logo), Nancy Ellen Kiernannancy ellen kiernan proposed a method of evaluating conferences that I think is important to keep and use. She suggested the use of “Listening Post” as an evaluation method. She says, “The “Listening Posts” consisted of a group of volunteer conference participants who agreed beforehand to “post” themselves in the meeting rooms, corridors, and break rooms and record what conferees told them about the conference as it unfolded [Not unlike Twitter, but with value; parenthetical added]. Employing listening posts is an informal yet structured way to get feedback at a conference or workshop without making participants use pencil and paper.” She put it in “Tipsheet #5” and published the method in Journal of Extension (JoE), the peer reviewed monthly on-line publication.

Quoting from the abstract of the JoE article, “Extension agents often ask, “Isn’t there an informal but somewhat structured way to get feedback at a conference or workshop without using a survey?” This article describes the use of ‘Listening Posts’ and the author gives a number of practical tips for putting this qualitative strategy to use. Benefits include: quality feedback, high participation and enthusiastic support from conferees and the chance to build program ownership among conference workers. Deficits: could exclude very shy persons or result in information most salient to participants.”

I’ve used this method. It works. It does solicit information about what difference the conference made, not whether the participants liked or didn’t like the conference. (This is often what is asked in the evaluation.) Nancy Ellen suggests that the listening post collectors ask the following questions:

  1. “What did you think of the idea of …this conference? and
  2. What is one idea or suggestion that you found useful for your professional work? (The value/difference question)
  3. Then, she suggests, that the participant tell anything else about the conference that is important for us to know.

Make sure the data collectors are distinctive. Make sure they do not ask any additional questions. The results will be interesting.

Cognitive bias and evaluation

Posted on by

My friend and colleague, Patricia Rogers, says of cognitive bias , “It would be good to think through these in terms of systematic evaluation approaches and the extent to which they address these.” This was in response to the article here. cognitive bias The article says that the human brain is capable of 10 to the 16th power (a big number) processes per second. Despite being faster than a speeding bullet, etc., the human brain has ” annoying glitches (that) cause us to make questionable decisions and reach erroneous conclusions.”

Bias is something that evaluators deal with all the time. There is desired response bias, non-response bias, recency and immediacy bias, measurement bias, and…need I say more? Isn’t evaluation and aren’t evaluators supposed to be “objective”? That we as evaluators behave in an ethical manner? That we have dealt with potential bias and conflicts of interest. That is where cognitive bias appear. And you might not know it at all. Continue reading →

Social Justice

Posted on by

social-justice.312132658_stdEarlier this week I attended a meeting of the College of Education (my academic home) Social and Environmental Justice (SJE) Work Group.  This is a loosely organized group of interested faculty and staff, led by an individual who is the ESOL Program Coordinator & Instructor. We had representatives from each of the four program areas (Adult and Higher Education [AHE], Teacher and Counseling Education [TCE], Science, Technology, Engineering, and Math [STEM], and Cultural and Linguistic Diversity [CLD]) in person (AHE, TCE,. CLD) or on paper [STEM]. The intent was to document for the work group, what each program area is doing in the area of social justice. Social Justice is a mandate for the College and OSU. The AHE and the TCE representatives provided us with information. We never did get to the STEM response. Then we got on to a discussion of what exactly is meant by social justice (since AHE has not defined the term specifically). My response was the evaluation response: it depends.

Most of the folks in the group focused on the interface of race and gender. OK. Others focused on the multiple and different voices. OK. Others focused on the advantages and disadvantages experienced. How is that not based in economics? Others focused on power and privileged. (As an accident of birth?) What is social justice exactly? Can you have social justice without environmental justice? How does that fit with the issue of diversity? How does any of this relate to evaluation?

The American Evaluation Association has had in place for a long time (since 1994) a set of five guiding principles (see Background section at the link for a bit of history). The fourth and fifth principles are, respectively, Respect for People and Responsibilities for General and Public Welfare. Respect for people says this:  Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders. Responsibilities for the General and Public Welfare says this: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation. Although both talk about parts of social justice that we talked about earlier this week, is this a complete view? Certainly, security, dignity, and self worth and diversity of interests and values approach the discussion we had. Is there still something missing? I think so. Where is fairness addressed?

To me, fairness is the crux of the issue.  For example, it certainly isn’t fair that in the US, 2% of the population has the cumulative wealth of the remaining 98%. (Then we are into economics.) Although Gandhi said “be the change”social justice 4 is that enough? What if that change isn’t fair?  And the question must be addressed, fair to whom? What if that change is only one person? Is that fair?  I always talk about the long term outcome as world peace (not in my lifetime, though). If you work for justice (for me that is fairness) will peace result? I don’t know. Maybe.social justice 3

 

 

Tomorrow is the Lunar New Year. It is the year of the goat/sheep/ram. I wish you the best. Eat jiaozi and tangerines (for encouraging wealth), and noodles without breaking/biting them (you do want a long life, right?). Happy New Year.

 

 

Chinese-Year-of-the-Goat-2015

 

 

Making a difference

Posted on by

making a difference 5I am an evaluator, charter member of the American Evaluation Association, and former member of the forerunner organization, Evaluation Network. When you push my “on ” button, I can talk evaluation until (a lot of metaphors could be used here); and often do. (I can also talk about other things with equal passion, though not professionally.) When my evaluation button is pushed or, for that matter, most of the time, I wonder what differencemake a difference am I making. In this case, I wonder what difference I am making with this blog.

One of my readers (I have more than I ever imagined) suggested that I develop an “online” survey that I can include regularly in my posts. I thought that was a good idea. I thought I’d go one better and have it be a part of the blog. Then I would tabulate the findings (if there are any 🙂 ). Just so you know, I DO read all the comments; I get at least six daily. I often do not comment on those, however.

So, reader, here is the making a difference survey . This link will (should) take you to Surveymonkey and the survey. Below, I’ve listed the questions that are in the survey.

Check all that apply.

Reading this blog makes a difference to me by:

  1. _____ Giving me a voice to follow
  2. _____ Providing interesting content
  3. _____ Providing content I can use in my work
  4. _____ Providing dependable post
  5. _____ Providing me with information to share
  6. _____ Building my skills in evaluation
  7. _____ Showing me that there are others in the world concerned with similar things
  8. _____ Offering me good reading about an interesting weekly topic
  9. _____ Offering me content of value to me
  10. _____ Other. Please specify in comment

Please complete the survey.

my two cents.

molly.

 

 

Update–making a difference

Posted on by

It has been about  five/chinese_symbols_number fiveyears since I started this blog (more or less–my anniversary is actually in early December) .

Because I am an evaluator, I have asked several time is this blog making a difference. And those posts, the ones in which I ask “is this blog making a difference”, are the ones which get the most comments.  Now, truly, most comments are often either about marketing some product, inviting me to view another blog, mirroring comments made previously, or comments in a language which I cannot read (even with an online translator). Yet, there must be something about “making a difference” that engages viewers and then engages them to make a comment.

Today, I read a comment Continue reading →

Nature of work and evaluation

Posted on by

Periodically I read Harold Jarche’s blog, Shedding Light on Workplace Transformation.  The post on June 13, 2014 talks about  the topic of  “work is changing”. He offered (as he is wont to do) some insights he has gleaned from social media over the the past fortnight including a statement from Hugh MacLeod @gapingvoid. MacLeod also wrote about what is corporate culture. A relevant topic for evaluators.

It gave me pause, especially in light of evaluation and the work I do. MacLeod says, “The nature of work is changing. People’s relationship with work is changing. The changes to society will be vast.” Because the nature of work is changing, the nature of evaluation is also changing. The change in evaluation is evidenced by looking at the evolution of evaluationevolution from the early 1960s to the present. That is over 50 years of evolution–I know that in the grand scheme of things, 50 years is nothing. For this profession, it is a lot. It kinda mirrors the leaps and bounds in technology. Of that 50+ years, I’ve been in the field over 30; I’ve seen so many changes.  Recognizing the time to step out and let the next generation of evaluators work is critical. Reinventing yourself so that you can still play in the sandbox sandbox(so to speak) is also important.

I recently had a conversation with a long time friend who has been successful at reinventing self. At the age where most people want only to play with their grandchildren grandparent_child_op, my friend is jetting around the world looking NOT at evaluation rather another field entirely, albeit still professionally and scholarly invested; being invited as a consultant, a speaker, a presenter at various meetings. Another long-time friend of mine who has been “retired” for 10 years has just finished a new book and has not one, TWO monographs in press. This is retirement?

Not only are evaluators re-inventing themselves to match the times, evaluation is also evolving, changing, and thinking differently than 50 years ago. Is this the evolution of a profession? Is this the evolution of the workforce? Is this the evolution of work? Yes. Yes. Yes.

What are you doing to further the profession? How will your contribution to the field help make the profession better? Let me know.

my two cents.

molly.