Monthly Archives: April 2014

Routine vs. Exceptions

Posted on by

Having just read Harold Jarche’s April 27, 2014 blog, making sense of the network era, about personal knowledge mastery (PKM), I am once again reminded about the challenge of evaluation. I am often asked, “Do you have a form I could use about…?” My nutrition and exercise questions notwithstanding (I do have notebooks of those), this makes evaluation sound like it is routine, standardized, or prepackaged rather than individualized, customized, or specific. For me, evaluation is about the exceptions to the rule; how the evaluation this week may have similarities to something I’ve done before (after all this time, I would hope so…), yet is so different; unique, specific.

You can’t expect to find a pre-made formsurvey 2 for your individual program (unless, of course you are replicating a previously established program). Evaluations are unique and the evaluation approach needs to match that unique program specialness. Whether the evaluation uses a survey, a focus group, or an observation (or any other data gathering approach), that approach to gathering data needs to focus on the evaluation question you want answered. You can start with “What difference did the program make?” Only you, the evaluator, can determine if you have enough resources to conduct the evaluation to answer the specific questions that result from what difference did the program make.  You probably do not have enough resources to determine if the program led your target audience to world peace; you might have enough resources to determine if the intention to do something different is there. You probably have enough resources to decide how to use your findings. It is so important that the findings be used; use may be how world peace may be accomplished.

demographics 4There are a few commonalities in data collection; those are the demographics, the data that tell you what your target audience looks like. Things like gender, age, marital status, education level, SES, probably a few other things depending on the program. Make sure when you ask demographic information that a “choose not to answer” option is provided in the survey. Sometimes you have to ask; observations don’t always provide the answer. You need to make sure you include demographics in your survey as most journals want to know what the target audience looked like.

Readers, what makes your evaluations different, unique, special? I’d like to hear about that. Oh and while you are at it…like and share this post, if you do.

 

Excel and sampling

Posted on by

My friend, Susan, who promised instructions on how to use Excel to select a sample, wrote a post on that very topic. excel and random sampling

She tells me that using screen shots made providing instructions easier so she posted it in a tutorial. Thank you, Susan, for adding this information. Now there is no reason for not selecting a random sample from your large population. Whether the sample will respond or not is out of your control.

Response rates are another thing, to be covered later.

Making a difference–analytics?

Posted on by

A reader commented that I need to be attending to my analytics not just reading my comments. Hmmm…

My question is: what do analytics tell me about making a difference–by providing an educational forum that changes people am I making a difference? Keep in mind that I am an evaluator and that the root for the word evaluation is “value”. So I wonder, do the analytics tell me about the merit, worth, value of this educational intervention?

What will the analytics really tell me about the readers? What will the comments tell me that the analytics don’t? Will the analytics tell me what difference this blog has made in the readers. Will analytics tell me about intention to change? How will analytics help me write posts to which more people will respond; make me more of an authority in my posting?

I DO NOT KNOW.

If someone, any one out in cyber space knows the answers (readers?), I’d love to hear from you. I blog weekly; sometimes more than weekly (like this week because, although I had the post written, I didn’t get it posted before I left the office so I posted it when I came back). I check my blog regularly for comments. I approve those which provide thoughtful meaningful responses for other readers as well as for me.

Another reader suggests that I look at the number of readers who have established an RSS feed or established a subscription. Hmmm…Not sure what that will tell me. I’ll talk to the IT folks for an answer to that question.

I would certainly appreciate any thoughts from readers.

More on Survey Sampling

Posted on by

ramdom sample (used with permission.)

After last week’s post on random sampling, I received a comment from a friend. She recommended some tools that might help calculate sample size especially when the population is different from the list Dillman offers. It is called Macorr. It has fields for the same variables that Dillman lists AND the population size can vary. Very important since Extension programs that repeat may not have a nice round number in the population.

She also says that you can calculate a random sample in Excel. She has to send me the directions on how to do that. I offer that to those of you who are much more adapt at Excel than I am.

When she sends it to me, I will post it. In the meantime, we wait or use the resources already available.

New Topic.

A comment was made: How important is it to have geographical representation in your random sample?

Theoretically, the random sample allows all individuals in a population to have the same chance of being in the sample. Because of that chance, there is also an excellent likelihood that the geographic representation will be distributed to represent the population. Of course, you have to decide before you sample, to what questions you want answers. If geographic areas may affect the outcome, then I would suggest the following. If you want to make sure that a particular area is represented (i.e., mixed metropolitan and rural areas), you can stratify on the type of representation you want.  I’m doing this in an evaluation I will be undertaking this summer and fall. We hypothesize that the metropolitan areas are different from the mixed metropolitan/rural areas and both are different from the rural areas. The evaluation team stratified on the density question and are randomly selecting in the three areas. I’ll let you know how the stratification worked.