In an ideal world, an experimental evaluation or randomized controlled trial (RCT), which compares participants in a program to a randomly assigned control group, provides the highest standard of proof and level of confidence that a program works. In the real world, an RCT is not always feasible, for a variety of reasons:
- It can be time consuming and costly.
- Denying services to a control group may raise ethical issues.
- A program may not be mature enough operationally or an organization’s performance management and measurement systems insufficiently developed for such robust research.
- A program may be so complex and multifaceted, or its objectives so broad, that designing an RCT is extremely difficult. The Clark Foundation and those of our grantees that provide a single service, be it home visitations by nurses to low-income first-time mothers or an afterschool mentoring program, have found RCTs less daunting than have grantees that offer an array of services to the entire community in which they are based. With these multi-service organizations, as we call them, the Clark Foundation is still exploring the best ways to marshal evidence on their programs’ behalf.
Under some of the above circumstances, a third-party, quasi-experimental evaluation that compares participants to a control group that is not randomly assigned may represent the highest proof point a program is capable of reaching. Developing evidentiary standards for effective programs is a work in progress, and the best measure in one domain—healthcare or youth development, for example—may not apply without adjustments to another, such as alternate energy sources or community development.
Tomorrow: The Value of Evaluation.
Monday’s post: Evidence and Evaluation: Getting from Promise to Proof.
Tuesday’s post: How Can an Organization Build Its Evidence Base?
Nancy Roob