Weak evaluation? Monitor & improve your project

It is important to reflect on your project from the very beginning

It’s difficult to know whether or not a program works if your method of evaluation isn’t reliable or valid. A poor method of evaluation can, and often will, lead to misleading results. Is your educational program accomplishing its stated objectives? This important question is not always easy to answer, yet this information is crucial to the success and financial viability of educational programs. Traditional methods of program evaluation – usually involving printed surveys and participant interviews – provide valuable information, but often require a significant investment of both time and money.

For example, imagine a team received a grant to educate a community about the antioxidant benefits of oranges and orange juice. The project team hoped to measure an increase in the consumption of oranges and orange juice at the end of the intervention. The program had a team of educators who specialized in the delivery of culturally appropriate educational materials and community outreach. As a result, the materials and outreach were well received by the community. Although the program staff thought the materials were working, their paper and pencil evaluations were inconclusive. After spending hours processing the data into a usable format, they found that a particular group within their sample had increased their consumption of oranges and orange juice. They suspected the group in question consisted of older adults. Unfortunately, they did not collect data with respect to age and no longer had the funds to modify and reprint their evaluation forms. As a result, their results were not publishable and their chances of obtaining additional funding were dramatically reduced.

Look at things from many vantage points. This orange is whole, peeled, cut, & X-rayed.

If the team had designed a better evaluation or had used techniques that would have made the evaluation format easier to modify, the outcome might have been completely different. Reliable and valid evaluation strategies can make the difference between sweet success and coming up with lemons.

Without good evaluation, you may have sour surprises.

With the proper knowledge and insight, however, program evaluation can be genuinely helpful both during and after a project. Understanding and using a variety of approaches to evaluation makes generating useful and accurate data possible while also adequately reflecting the effectiveness of the program in question. Solid evaluation techniques can also ensure that a program does not do unintentional harm to the user. If, for example, a smoking cessation program went unmonitored or unevaluated, there would be no way of knowing whether participants started smoking more often.

Although traditional evaluations are time-consuming, the process has been revolutionized by technology. Today’s tools can help project teams design, implement, manage, and report evaluations in ways that are efficient both financially and in terms of time and effort. Indeed, the difference between the old methods and the new are as striking as the difference between farming your food and shopping at a grocery store. Moreover, technology enables projects to gather better information sooner, allowing project teams to more nimbly adjust their programs while satisfying their stakeholders in the process. If a project uses an external evaluator, technological tools can provide more and better data so that he or she can provide more sophisticated analyses.

Strong evaluation can make the difference between sweet success and coming up with lemons.

Developing an evaluation plan can help project teams and stakeholders bring critical insight to the table. Certain methods of evaluations can assess a program during its development in order to guide it towards success in the future. Other methods can help narrow the focus of a program, so that it delivers the information or service most needed by the target population. There are many ways to look at a problem, and an evaluation plan can help determine which approach is ideally suited to everyone’s needs.

See recipes for improving evaluation >>