I’ve just spent all of August and most of September editing chapters for a volume of New Directions in Evaluation (NDE) on Accreditation, Certification, and Credentialing. These topics all relate to competencies which all relate to building capacity. Now I can site a lot of references for competencies. (For example, Stevahn, King, Ghere, Minnema, 2005, AJE 26(1), pp. 43-59., among others see the work by King and cadre–that one cited just happens to be on my desk right now.) This group has been working on competencies for the last 15 or so years. This is important work–as well as problematic (hence the issue of NDE). I won’t go into details here because the NDE volume pretty much addresses these issues from a variety of perspectives. We (my co-editor, Jim Altschuld and I) have assembled (what I think is) a stellar collection of writers who have good ideas. Editing an issue of NDE (again) was a valuable experience for me: I learned again why I don’t write the definitive text on anything; I learned again how important Accreditation, Certification, and Credentialing are; I am reminded how complicated it is to assemble a list of competencies that adequately capture what is an evaluator; and I am once again humbled, recognizing that cynicism does not come with the territory–it is acquired.
Now, a bit on competencies and why they are important.
I think everyone will agree that there are certain knowledge (what a person can learn), skills (what a person can do), and dispositions/attitudes (what a person can think and/or feel–they are different BTW) necessary for an individual to function effectively as an evaluator. The question is what exactly are they? And can evaluation be a profession without an established list of competencies? The Worthen 1994 article is important here (Worthen, B. R. [1994]. Is evaluation a mature profession that warrants the preparation of evaluation professionals? In J. W. Altschuld & M. Engle [Eds.], New Directions for Program Evaluation: No. 62. The preparation of professional evaluators: Issues, perspectives and programs. [pp. 3–15]. San Francisco: Jossey-Bass). The Stevahn et al. article lists six different categories of competencies (professional practice, systemmatic inquiry, situational analysis, project management, reflective practice, and interpersonal competence). The CES list includes five categories (reflective practice, technical practice, situational practice, management practice, and interpersonal practice). They are similar, yet different.
The Canadian Evaluation Society has established a credentialing process that involves the a list of competencies that went through an extensive and exhaustive process research, consultation, and validation process. The AEA has yet to develop (or endorse) a similar list, and a similar list exists (see the Stevahn, King, Gere, & Minnema citation above).
How many of you who are practicing evaluators can honestly say you were taught in your preparation programs (even if you did a preparation program in a discipline other than evaluation) to analyze situations? To manage projects? To reflect on practice? About interpersonal communications? I’m guessing most people were exposed (even briefly) to professional practice (after all part of preparation is the socialization to the profession) and technical practice/systematic inquiry. With that disparity across preparation, how can evaluation be a profession?
my 
.
molly.
I always feel fortunate that I was doing my doctoral work at a university with a graduate education program (I’m an educator and was studying educational leadership) at a time when a program evaluation certificate program was introduced. I was among the first to receive this certificate along with my degree. I took courses in eval and research methods, and conducted an independent evaluation (meaning on my own) and did a program evaluation dissertation. However, that being said, I received none of the training you mention – nothing on analyzing projects, reflecting on practice, or interpersonal communications. I did though, get some of these topics through the other part of my coursework in leadership. Perhaps I chose well two topics that dovetail (education and eval), or perhaps I was just fortunate that my educational leadership program included courses on organizational dynamics, human resources, site management, etc. that gave me a taste of these critical topics. Great food for thought, Molly! Looking forward to the NDE issue!
Sheila, AEA has not (now or ever in my tenure with the organization) seen the need to gate keep entry into the profession. So you question is a good one–what does the certificate mean if you didn’t experience the competencies that have been identified and validated? Perhaps AEA will move in this direction–I do not know. Knowing that the membership is not particularly enthused about credentialing similar to the Canadians (see the NDE volume) will make this challenge difficult for AEA–if they go there at all. Lots of food for thought, to be sure.
Despite my program at MSU wanting to prepare professionals who could identify problems and build solutions to implement and disseminate (as well as validly research), there was nothing about management. I’ve had to learn a little about managing projects during my years of practice, but even that has been only with simple evaluation projects with only a few staff.
Years later I got a seminary degree on doing “ministry”. That included interpersonal communication, counseling, and leadership, but, oddly, nothing on management.
Today I am near the end of the Claremont Certificate program. That includes extensive discussion of the various approaches to evaluation practice and reflecting on practice. Through the 3 programs I probably got most of what you listed, except for project/program management. But I think universities have complete degree programs for that. Should an evaluator have to complete a double-major undergraduate degree and two masters to be minimally qualified?
The issue of competencies is a thorny one, with no easy answers. It all depends. Most people take what they have in terms of skills (and therefore competencies) and apply them to evaluation. Learning happens because the learner wants to learn. The number of degrees seems irrelevant. My degree is in program evaluation; I didn’t get exposed to many of the competencies listed in the Stavahn and King list. Could I apply for the Canadian certification, maybe. Do I consider myself a competent evaluator, yes. To be a competent evaluator does an individual need all the competencies listed by Stavahn and King, probably not. How best can an individual be a competent evaluator? It all depends.