The Value of Evaluation

Submitted by Nancy Roob on January 14, 2010 - 9:27am

In our experience, an independent evaluation benefits an organization in other important ways in addition to assessing a program externally. The data collection and analysis an evaluation requires provide an opportunity and means to improve a program internally. The information an evaluation gleans enables an organization to determine what works and what doesn’t, and to adjust its program(s) accordingly. Preparing for and undergoing an evaluation has prompted several of our grantees to design or improve performance measurement systems and to use these as management tools. The discipline of evaluation, understanding a program’s strengths, limitations, and areas needing improvement, makes an organization more realistic, sophisticated, and effective.

It’s important to note that not every program requires an evaluation to prove its effectiveness. Many organizations, including some EMCF grantees, have adopted and implemented models that have already been evaluated. In such instances it is critical to ensure that the model is replicated with fidelity. Funders and nonprofits can use ongoing performance tracking data to document that the program is reproducing its original outcomes. Many a promising program has been compromised by cutting corners.

Monday's post: Evidence and Evaluation: Getting from Promise to Proof.

Tuesday's post: How Can an Organization Build Its Evidence Base?

Wednesday's post: A Single Standard of Evidence?

Tomorrow: How philanthropy and government can work together.

Topics: Evaluation, Metrics
»

Comments

More sophisticated thinking about 'doing an evaluation'

Submitted by Paul Duignan on January 17, 2010 - 4:36pm.

Hi Nancy, thanks for your thoughtful posts on evidence and evaluation. It seems to me that one issue we need to get clear on is the type of evaluation that we are expecting is done on programs. Usually the instruction is issued from above, or pressure is felt, to 'Do an evaluation' on a program. And increasingly this is being interpreted as 'Do an impact/outcome evaluation' (ie. be able to attribute changes in high-level outcomes to the program in question). In my view there are two mistaken ways of dealing with this pressure.

1. To just say that you will do an impact/outcome evaluation without having worked out what is appropriate, feasible and/or affordable in regard to impact/outcome evaluation.

2. Just say that your area of work is too complicated and that and that you are not going to do an impact/outcome evaluation without any further examination of the issue.

I think that the best approach is for all programs to undertake a formal technical analysis of the appropriateness, feasibility and affordability of doing an impact/outcome evaluation and to document this. This then forms part of the basis for working out what is the appropriate mix of monitoring and evaluation for that project.

In order to do this formal analysis we need a framework and that is what I am trying to provide in my work. If you have a moment check out the article Duignan, P. (2009). Impact evaluation - when it should, and should not, be used. Outcomes Theory Knowledge Base Article No. 242. ( http://knol.google.com/k/paul-duignan-phd/impact-evaluation-when-it-shou... ).

Regards,

Paul Duignan, PhD. (Follow me on my Outcomes Blog; Twitter; or via my E-Newsletter).