A good friend of mine asked me today if I knew of any attributes (which I interpreted to be criteria) of qualitative data (NOT qualitative research). My friend likened the quest for attributes for qualitative data to the psychometric properties of a measurement instrument–validity and reliability–that could be applied to the data derived from those instruments.
Good question. How does this relate to program evaluation, you may ask. That question takes us to an understanding of paradigm.
Paradigm (according to Scriven in Evaluation Thesaurus) is a 
general concept or model for a discipline that may be influential in shaping the development of that discipline. They do not (again according to Scriven) define truth; rather they define prima facie truth (i.e., truth on first appearance) which is not the same as truth. Scriven goes on to say, “…eventually, paradigms are rejected as too far from reality and they are always governed by that possibility[i.e.,that they will be rejected] (page 253).”
So why is it important to understand paradigms. They frame the inquiry. And evaluators are asking a question, that is, they are inquiring.
How inquiry is framed is based on the components of paradigm:
- ontology–what is the nature of reality?
- epistemology–what is the relationship between the known and the knower?
- methodology–what is done to gain knowledge of reality, i.e., the world?
These beliefs shape how the evaluator sees the world and then guides the evaluator in the use of data, whether those data are derived from records, observations, interviews (i.e., qualitative data) or those data are derived from measurement, scales, instruments (i.e., quantitative data). Each paradigm guides the questions asked and the interpretations brought to the answers to those questions. This is the importance to evaluation.
Denzin and Lincoln (2005) in their 3rd edition volume of the Handbook of Qualitative Research 
list what they call interpretive paradigms. They are described in Chapters 8 – 14 in that volume. The paradigms are:
- Positivist/post positivist
- Constructivist
- Feminist
- Ethnic
- Marxist
- Cultural studies
- Queer theory
They indicate that each of these paradigms have criteria, a form of theory, and have a specific type of narration or report. If paradigms have criteria, then it makes sense to me that the data derived in the inquiry formed by those paradigms would have criteria. Certainly, the psychometric properties of validity and reliability (stemming from the positivist paradigm) relate to data, usually quantitative. It would make sense to me that the parallel, though different, concepts in constructivist paradigm, trustworthiness and credibility, would apply to data derived from that paradigm–often qualitative.
If that is the case–then evaluators need to be at least knowledgeable about paradigms.
