One response I got for last week’s query was about on-line survey services. Are they reliable? Are they economical? What are the design limitations? What are the question format limitations?
Yes. Depends. Some. Not many.
Depends. Cost of postage for paper survey (both out and back) vs. the time it takes to enter questions in system. Cost of system vs. length of survey. These are things to consider.
Because most people have access to email today, using an on-line survey service is often the easiest and most economical way to distribute an evaluation survey. Most institutional review boards view an on-line survey like a mail survey and typically grant a waiver of documentation of informed consent. The consenting document is the entry screen and often an agree to participate question is included on that screen.
Are they valid and reliable?
Yes, but…The old adage “Garbage in, garbage out” applies here. Like a paper survey, and internet survey is only as good as the survey questions. Don Dillman, in his third edition “Internet, mail, and mixed-mode surveys” (co-authored with Jolene D. Smyth and Leah Melani Christian), talks about question development. Since he wrote the book (literally), I use this resource a lot!
What are the design limitations?
Some limitations apply…Each online survey service is different. The most common service is Survey Monkey (www.surveymonkey.com). The introduction to Survey Monkey says, “Create and publish online surveys in minutes, and view results graphically and in real time.” The basic account with Survey Monkey is free. It has limitations (number of questions ; limited number of question formats ; number of responses ). And you can upgrade to the Pro or Unlimited for a subscription fee ($19.95/mo or $200/annually, respectively). There are others. A search using “survey services” returns many options such as Zoomerang or InstantSurvey.
What are the question format limitations?
Not many–both open-ended and closed ended questions can be asked. Survey Monkey has 15 different formats from which to choose (see below). I’m sure there may be others; this list covers most formats.
- Multiple Choice (Only one Answer)
- Multiple Choice (Multiple Answers)
- Matrix of Choices (Only one Answer per Row)
- Matrix of Choices (Multiple Answers per Row)
- Matrix of Drop-down Menus
- Rating Scale
- Single Textbox
- Multiple Textboxes
- Comment/Essay Box
- Numerical Textboxes
- Demographic Information (US)
- Demographic Information (International)
- Date and/or Time
- Descriptive Text
Oregon State University has an in-house service sponsored by the College of Business (BSG–Business Survey Groups). OSU also has an institutional account with Student Voice, an on-line service designed initially for learning assessment which I have found useful for evaluations. Check your institution for options available. For your next evaluation that involves a survey, think electronically.