Personal and situational bias are forms of cognitive bias and we all have cognitive bias.

When I did my dissertation on personal and situational biases, I was talking about cognitive bias (only I didn’t know it, then).

According to Wikipedia, the term cognitive bias was introduced in 1972 (I defended my dissertation in 1983) by two psychologists Daniel Kahneman  and Amos Tversky .

Then, I hypothesized that previous research experience (naive or sophisticated)  and the effects of exposure to expected project outcomes (positive, mixed, negative) would affect the participant and make a difference in how the participant would code data. (It did.)  The Sadler article which talked about intuitive data processing was the basis for this inquiry. Now many years later, I am encountering cognitive bias again. Sadler says that “…some biases can be traced to a particular background knowledge…”(or possibly–I think–lack of knowledge), “…prior experience, emotional makeup or world view”. (This, I think, falls under the category of, according to Tversky and Kahneman, human judgements and it will differ from rational choice theory (often given that label).

This is important for evaluators to remember…what you bring to the table does affect you; any assumptions you make because of your experience, world view, and/or perceptions affects you AND the evaluation. One way to help mitigate those assumptions is to make them explicitly clear–put them on the table.

Today, I was in a meeting about diversity. Although the term had been defined previously, there were many new players at the table for whom this term had not been clearly defined. Diversity is more than just the intersection of race and gender. Daryl Smith presents a model addressing this (she presented this model at a presentation at OSU in 2012). The discussion until that point had focused only on race; all the other forms of diversity including gender were not being addressed. Yet to talk about this topic all forms of diversity needed to be considered. Smith’s model included climate, access, success, education, scholarship, outreach, and capacity and they were all listed as “…overarching institutional goals for equity, inclusion, and diversity.” We had not clarified assumptions in this discussion. We were being influenced by personal and situational biases. If this had been an evaluation, there would have been a lot of cognitive dissonance; even not being an evaluation, there was a lot of cognitive dissonance.  We will resolve this dissonance, even if it takes a while.

What I ask of evaluators is to remember that what you have experienced and what you know does affect the evaluation–any evaluation. Evaluations are not free of bias; evaluations can never be bias free. All we can do is try to mitigate the biases.

Citation:

Sadler, D. R. (1981). Intuitive data processing as a potential source of bias in naturalistic evaluations. Educational Evaluation and Policy Analysis 3(4), 25-31.