Tag Archives: Qualitative Data

Collecting outcome data

Posted on by

I know it is Monday, not Tuesday or Wednesday. I will not have internet access Tuesday or Wednesday and I wanted to answer a question posed to me by a colleague and long time friend who has just begun her evaluation career.

Her question is:

What are the best methods to collect outcome evaluation data.Data 3 2_2

Good question.

The answer:  It all depends.

On what does the collection depend?

  • Your question.
  • Your use.
  • Your resources.

If your resources are endless (yeah, right…smiley ), then you can hire people; use all the time you need; and collect a wealth of data. Most folks aren’t this lucky.

If you plan to use your findings to convince someone, you need to think about what will be most convincing. Legislators like the STORY that tugs at the heart strings.

Administrators like, “Just the FACTS, ma’am.” Typically presented in a one-page format with bullets.

Program developers may want a little of both.

Depending on what question you want answered will depend on how you will collect the answer.

My friend, Ellen-Taylor Powell, at the University of Wisconsin Extension Service has developed a handout of data methods (see: Methods for Collecting Information).  This handout is in PDF form and can be downloaded. It is a comprehensive list of different data collection methods that can be adapted to answer your question within your available resources.

She also has a companion handout called Sources of Evaluation Information. I like this handout because it is clear and straight forward. I have found both very useful in the work I do.

Whole books have been written on individual methods. I can recommend some I like–let me know.

Focus Group Questions

Posted on by

Welcome back!   For those of you new to this blog–I post every Tuesday, rain or shine…at least I have for the past 6 weeks…:) I guess that is MY new year’s resolution–write here every week; on Tuesdays…now to today’s post…

What one thing are you going to learn this year about evaluation?

Something about survey design?

OR logic modeling?

OR program planning?

OR focus groups?focusgroups

OR…(fill in the blank and let me know…)

A colleague of mine asked me the other day about focus groups.

Specifically, the question was, “What makes a good focus group question?”

I went to Dick Krueger and Mary Anne Casey’s book (Focus Groups, 3rd ed. Sage Publications, 2000).  On page 40, they have a section called “Qualities of Good Questions”. These make sense.They say: Good questions…

focus group book--krueger

  1. …sound conversational
  2. …use words participants would use.
  3. …are easy to say.
  4. …are clear.
  5. …are short.
  6. …are open-ended.
  7. …are one dimensional.
  8. …include good directions.

Let’s explore these a bit.

  1. Since focus groups are a social experience (albeit, a data gathering one), conversational questions help set an informal tone.
  2. If participants don’t/can’t understand your questions (because you use jargon, technical terms, etc.), you won’t get good information. Without good information, your focus group will not help answer your inquiry.
  3. You don’t want to stumble over the words, so avoid complicated sentences.
  4. Make sure your participants know what you are asking. Long introductions can be confusing, not clarifying. Messages may be mixed and thus interpreted in different ways. All this results information that doesn’t answer your inquiry.
  5. Like being clear, short questions tend to avoid ambiguity and yield good data.
  6. To quote Dick and Mary Anne, “Open-ended questions are a hallmark of focus group interviewing.” You want an opinion. You want an explanation. You want rich description. Yes/No doesn’t give you good data.
  7. Using synonyms add richness to questioning–using synonyms confuses the participant. Confused participants yields ambiguous data. Avoid using synonyms–keep questions one-dimensional keeps questions clear.
  8. Participants need clear instructions when asked to do something in the focus group.  “Make a list” needs to have “on the piece of paper in front of you” added.  A list in the participants head may get lost and you loose the data.

Before you convene your focus group, make sure you have several individuals (3 – 5) who are similar to and not included in your target audience review the focus group questions. It is always a good idea to pilot any question you use to gather data.

Ellen Taylor-Powell (at University of Wisconsin Extension) has a Quick Tips sheet on focus groups for more information. To access it go to: http://www.uwex.edu/ces/pdande/resources/pdf/Tipsheet5.pdf