Last week I suggested a few evaluation related resolutions…one I didn’t mention which is easily accomplished is reading and/or contributing to AEA365.  AEA365 is a daily evaluation blog sponsored by the American Evaluation Association.  AEA’s Newsletter says: “The aea365 Tip-a-Day Alerts are dedicated to highlighting Hot Tips, Cool Tricks, Rad Resources, and Lessons Learned by and for evaluators (see the aea365 site here). Begun on January 1, 2010, we’re kicking off our second year and hoping to expand the diversity of voices, perspectives, and content shared during the coming year. We’re seeking colleagues to write one-time contributions of 250-400 words from their own experience. No online writing experience is necessary – you simply review examples on the aea356 Tip-a-Day Alerts site, craft your entry according to the contributions guidelines, and send it to Michelle Baron our blog coordinator. She’ll do a final edit and upload. If you have questions, or want to learn more, please review the site and then contact Michelle at aea365@eval.org. (updated December 2011)”

AEA365 is a valuable site.  I commend it to you.

Now the topic for today: Data sources–the why and the why not (or advantages and disadvantages for the source of information).

Ellen Taylor Powell, Evaluation Specialist at UWEX, has a handout that identifies sources of evaluation data.  These sources are existing information, people, and pictorial records and observations. Each source has advantages and disadvantages.

The source for the information below is the United Way publication, Measuring Program Outcomes (p. 86).

1.  Existing information such as Program Records are

• Available
• Accessible
• Known sources and methods  of data collection

Program records can also

• Be corrupt because of data collection methods
• Have missing data
• Omit post intervention impact data

2. Another form of existing information is Other Agency Records

• Offer a different perspective
• May contain impact data

Other agency records may also

• Be corrupt because of data collection methods
• Have missing data
• May be unavailable as a data source
• Have inconsistent time frames
• Have case identification difficulties

3.  People are often main data source and include Individuals and General Public and

• Have unique perspective on experience
• Are an original source of data
• General public can provide information when individuals are not accessible
• Can serve geographic areas or specific population segments

Individuals and the general public  may also

• Introduce a self-report bias
• Not be accessible
• Have limited overall experience

4.  Observations and pictorial records include Trained Observers and Mechanical Measurements

• Can provide information on behavioral skills and practices
• Supplement self reports
• Can be easily quantified and standardized

These sources of data also

• Are only relevant to physical observation
• Need data collectors who must be reliably trained
• Often result in inconsistent data with multiple observers
• Are affected by the accuracy of testing devices
• Have limited applicability to outcome measurement