Evaluation is an Everyday Activity « Program Evaluation Discussions

Having just returned from the annual AEA conference (Evaluation 2014) in Denver, I am taking this moment to reflect, process, and apply.

For years my criteria for a “good” conference was the following

See three long time friends and spend some time catching up;
Meet three people I didn’t know before and would like to continue to know;
Get three new ideas that I can use.

I think this year’s conference was a success (despite the difficulty in identifying who was doing what when because the management corporation minimized the program in an attempt to be ecological, if excluding). If I were to ask my daughters to rate the conference on a scale of one (1) to 10 (ten), one being not “good”, 10 being “good”, I think they would have said an 8 – 8.5. (They have their own following of friends and their own interests.)

I saw and talked to three long time friends, although I missed those who have chosen not to attend AEA any more (I must be getting old) and those with whom I didn’t spend time.

I met more than three people I didn’t know before and I must say, if they are any indication (and I think they are) of the evolution of the association, the association is in good hands (even though I miss the intimacy I “grew up with”). Continue reading →

Recently, I drafted a paper about a capacity building; I’ll be presenting it at the 2014 AEA conference. The example on which I was reporting was regional and voluntary; it took a dedication, a commitment from participants. During the drafting of that paper, I had think about the parts of the program; what would be necessary for individuals who were interested in evaluation and had did not have a degree. I went back to the competencies listed in the AJE article (March 2005) that I cited in a previous post. I found it interesting to see that the choices I made (after consulting with evaluation colleagues) were listed in the competencies identified by Stevahn et al., yet they list so much more. So the question occurs to me is: To be competent, to build institutional evaluation capacity are all those needed? Continue reading →

Over the last several months, the Local Arrangements Working Group has been blogging at AEA365. One of ways evaluators can get ready for the upcoming annual conference is to read what the LAWG has to say about the conference. This year, the conference is once again in Denver. AEA was in Denver in 2008. Be forewarned–Denver is the mile high city. The air is rarefied and very dry. It may snow!

The LAWG has a lot to say about the conference and there are A LOT of links in these posts that are worth checking. For those who have not been to AEA before or for those who have recently embraced evaluation, reading their posts are a wealth of information, as is the AEA website.

I will be presenting at two sessions this year–one on blogging (duh…) and one on capacity building. I see them related. I will also (like last year) be assisting with a professional development session (number 25) with my long time friend and colleague, Jim Altschuld. The professional development session occurs on Wednesday, October 15, 2014 from 8:00am MT – 3:00pm MT. It is titled Practical Ways to Link Needs Assessment (NA) with Asset/Capacity Building. (Just a little advertisement 🙂 ) It will draw from his new book, Bridging the Gap Between Asset/Capacity Building and Needs Assessment. Continue reading →

I’ve just spent all of August and most of September editing chapters for a volume of New Directions in Evaluation (NDE) on Accreditation, Certification, and Credentialing. These topics all relate to competencies which all relate to building capacity. Now I can site a lot of references for competencies. (For example, Stevahn, King, Ghere, Minnema, 2005, AJE 26(1), pp. 43-59., among others see the work by King and cadre–that one cited just happens to be on my desk right now.) This group has been working on competencies for the last 15 or so years. This is important work–as well as problematic (hence the issue of NDE). I won’t go into details here because the NDE volume pretty much addresses these issues from a variety of perspectives. We (my co-editor, Jim Altschuld and I) have assembled (what I think is) a stellar collection of writers who have good ideas. Editing an issue of NDE (again) was a valuable experience for me: I learned again why I don’t write the definitive text on anything; I learned again how important Accreditation, Certification, and Credentialing are; I am reminded how complicated it is to assemble a list of competencies that adequately capture what is an evaluator; and I am once again humbled, recognizing that cynicism does not come with the territory–it is acquired.

Now, a bit on competencies and why they are important. Continue reading →

This summer I spent a lot of time dealing with needs assessments and talking about needs and assets. It occurred to me that the difference between need and wants has a lot to do with evaluation (among other things). So what are needs? What are wants? How does all this relate to evaluation?

Needs.

Maslow spoke eloquently about needs in his hierarchy, and although the hierarchy is often presented as a pyramid, Maslow didn’t present the needs this way. He did present this hierarchy as a set of building blocks with basic needs (physiological) as the foundation, followed by safety, loving/belonging, esteem, and self-actualization. He talks about this theory of motivation in his book, Motivation and Personality (a 3rd edition is available as well). This view of the individual ushered in the humanistic view of psychology (often called the third theory after behaviorism and psychoanalysis). He believed that human could not live without these needs and advocated that they are necessary for survival.

Wants

A “want” is often considered a desire based purely in economic, social, or psychological reality of human existence. It is something that an individual would like to have. (Chocolate, any one?) A want is not essential to human existence; it is only something an individual would like to have. Unfortunately, there are limited resources (as well as a large body of literature) talking about having enough. If you have enough, then wants are few and resources are available for everyone.

Continue reading →

I just finished a chapter on needs assessment in the public sector–you know that part of the work environment that provides a service to the community and receives part of its funding from the county/state/federal governments. Most of you know I’ve been an academic for at least 31 years, maybe more (depending on when you start the clock). In that time I’ve worked as an internal evaluator, a program planner, and a classroom teacher. Most of what I’ve done has an evaluative component to it. (I actually earned my doctorate in program evaluation when most people in evaluation came from other disciplines.) During that time I’ve worked on many programs/projects in a variety of situations (individual classroom, community, state, and country). I find it really puzzling that evaluators will take on evaluation without having a firm foundation on which to base those evaluations. (I know I have done this; I can offer all manner of excuses, only not here).

If I had been invited to participate in the evaluation at the beginning of the program, at the conceptualization stage, I would have asked if a needs assessment had been done and what was the outcome of that assessment. Was there really a lack (i.e., a need); or was this “need” contrived to do something else (bring in grant money, further a career, make a stakeholder happy, etc.)? Continue reading →

From Social Networks: What Maslow Misses | Psychology Today – via @mslogophiliac

“Humans are social animals for good reason. Without collaboration, there is no survival. It was not possible to defeat a Woolley Mammoth, build a secure structure, or care for children while hunting without a team effort. It’s more true now than then. Our reliance on each other grows as societies became more complex, interconnected, and specialized. Connection is a prerequisite for survival, physically and emotionally.”

This statement, which I found on Harold Jarche ‘s blog, applies to evaluation as much as it applies to the example provided by Psychology Today.

Evaluation is a collaborative effort; a team effort, a social effort. Without the collaboration, evaluation lacks much. I’m not sure that survival is dependent on evaluation and collaborative effort; perhaps. The evaluator may know all about evaluation and yet not be able to solve the problem presented by the client because the evaluator doesn’t know about the topic needing to be evaluated. The evaluator may know about something similar to and different from what the client needs and yet, not know about the specific problem. Let me give you an example.

I’ve done a lot of evaluation in the natural resources area and as a result, I’ve learned much about various natural resource topics, including horticulture, plant science, crop science, marine science. I do not know much about potatoes. A while back, a colleague called me and asked if I could/would serve as the evaluator on a ZEBRA CHIP project. Before I said, Sure, I asked about ZEBRA CHIP. Apparently, it is a potato disease transmitted by bacteria carrying psyllid that is causing much economic devastation among growers. It shows up best when the potatoes are made into chips (hence the name). It looks like this: . To me, it isn’t particularly stripped like the animal which offers its name, yet it doesn’t look like potato chips I’m used to seeing. I”m told that there is an unpleasant flavor to the chips as well. I knew a lot about growing things, not about potatoes, even though I’ve worked with potato growers before, just not about this disease.

So, I said sure, only to discover that I have 11 lines in which to write a cogent evaluation section for the work that Extension will be doing (if the grant is funded). If the grant is funded, it will be a five year effort. A continuation actually, which brings me full circle–a collaboration of multiple universities, multiple disciplines, multiple investigators. So what could I say cogently in 11 lines? I suggested that perhaps looking at intention and confidence would be appropriate because we (remember, I said, “Sure”) would not be able to measure actual behavior change. And to overcome the psyllids and eradicate this problem (not unlike the spotted wing drosophila which is affecting the soft fruits of the NW, specifically blueberries), we would need to get as close to behavior change as possible once the teaching has occurred. How can social media be used here? Good question–something to explore. At what level of Maslow’s hierarchy is this collaboration? Survival, sure. Somehow I don’t think Maslow was focused on economic survival.

my .

molly.

Within the last 24 hours I’ve had two experiences that remind me of how tenuous our connection is to others.

Yesterday, I was at the library to return several books and pick up a hold. As I went to check out using the digitally connected self-check out station, I got an “out of service” message. Not thinking much of it, as I had received that message before, I moved to another machine. And got the same message! So I went to the main desk. There was a person in front of me; she was taking a lot of time. Turns out it wasn’t her; it was the internet (or intranet, don’t know which). There was no connection! After several minutes, a paper system was implemented and I was assured that the book would be listed by this evening. That the library had a back up system impressed me; I’ve often wondered what would happen if the electricity went out for a long periods of time since the card catalogs are no longer available.
Also, yesterday, I received a phone call on my office land line (!), which is a rare occurrence these days. On the other end was a long time friend and colleague. We are working feverishly on finishing a NDE volume. We have an August 22 deadline and I will be out of town taking my youngest daughter to college. Family trumps everything. He was calling because the gardeners at his condo had cut the cable to his internet, television, and most importantly, his wi-fi. He couldn’t Skype me (our usual form of communication)! He didn’t expect resumption of service until the next day (August 20 at 9:47am PT he went back on line–he lives in the Eastern Time Zone). Continue reading →

Wikipedia says the phrase of “dog days” refers to the sultry (read: HOT) days of summer, typically July and August in the northern hemisphere. In Oregon, it is definitely August. It was 100.4 yesterday; 96 the day before. I didn’t sleep well, even with fans and cool evenings (low 60s). This is the hottest summer I’ve experienced in the 16 years I’ve been in Oregon–there were 10 days of hot weather in May, June, and July; and now so far in August. The Dog days are also when the star Sirius can be seen (assuming there is no cloud cover, a common phenomenon in Oregon–if not the winter rainy season, then summer thunderstorms.) Dog days were thought to be when “the Sea boiled, the Wine turned sour, Dogs grew mad, and all other creatures became languid; causing to man (sic), among other diseases, burning fevers, hysterics, and phrensies.” according to Brady’s Clavis Calendaria, 1813 (A Compendious Analysis of the Calendar; Illustrated with Ecclesiastica, Historical, and Classical Anecdotes 2 volumes). Continue reading →

Periodically I read Harold Jarche’s blog, Shedding Light on Workplace Transformation. The post on June 13, 2014 talks about the topic of “work is changing”. He offered (as he is wont to do) some insights he has gleaned from social media over the the past fortnight including a statement from Hugh MacLeod @gapingvoid. MacLeod also wrote about what is corporate culture. A relevant topic for evaluators.

It gave me pause, especially in light of evaluation and the work I do. MacLeod says, “The nature of work is changing. People’s relationship with work is changing. The changes to society will be vast.” Because the nature of work is changing, the nature of evaluation is also changing. The change in evaluation is evidenced by looking at the evolution of evaluation from the early 1960s to the present. That is over 50 years of evolution–I know that in the grand scheme of things, 50 years is nothing. For this profession, it is a lot. It kinda mirrors the leaps and bounds in technology. Of that 50+ years, I’ve been in the field over 30; I’ve seen so many changes. Recognizing the time to step out and let the next generation of evaluators work is critical. Reinventing yourself so that you can still play in the sandbox (so to speak) is also important.

I recently had a conversation with a long time friend who has been successful at reinventing self. At the age where most people want only to play with their grandchildren , my friend is jetting around the world looking NOT at evaluation rather another field entirely, albeit still professionally and scholarly invested; being invited as a consultant, a speaker, a presenter at various meetings. Another long-time friend of mine who has been “retired” for 10 years has just finished a new book and has not one, TWO monographs in press. This is retirement?

Not only are evaluators re-inventing themselves to match the times, evaluation is also evolving, changing, and thinking differently than 50 years ago. Is this the evolution of a profession? Is this the evolution of the workforce? Is this the evolution of work? Yes. Yes. Yes.

What are you doing to further the profession? How will your contribution to the field help make the profession better? Let me know.

my .

molly.

Evaluation is an Everyday Activity

Program Evaluation Discussions

A good conference

Capacity building and competencies

Evaluation ’14–AEA Annual Conference

Competencies

Needs, wants, and evaluation

Evaluation and Needs Assessment

Evaluation, Maslow’s hierarchy of needs, and social media

Evaluation and the electronic age

Dog days of summer

Nature of work and evaluation

Contact Info