Press "Enter" to skip to content

QUIS ÆSTIMABIT IPSOS ÆSTIMATORES? (WHO EVALUATES THE EVALUATORS?)

Foundations hold troves of important research and evaluations. But how do they decide what parts of this to publicize, how, and for what audiences?

(Pedantry warning: The pidgin Latin in the headline of this post is a butchery of a famous line from Juvenal’s SatiresQuis custodiet ipsos custodes? or, Who will watch the watchmen? One of the privileges of blogging at an elite university is that you get to write headlines in Latin.)

In his January presentation to the Foundation Impact Research Group (video here), Chris Oechsli, president of The Atlantic Philanthropies, shared some of his struggles in defining that foundation’s legacy. Because Atlantic plans to end most or all of its grantmaking next year, thus putting all of its endowment to use (close to $8 billion in the end) and then closing its doors, it has more reason to think about legacy than do perpetual foundations, whose contributions to the public good are forever evolving. In Atlantic’s case, its achievements will soon be completed and beyond revision.

One part of that legacy, Mr. Oechsli suggested, will be the body of learning the foundation leaves behind: research reports, white papers, videos, policy memoranda, speeches, and, most obviously, evaluations. Evaluations tend mostly to be hefty academic documents, full of evidence and carefully reasoned conclusions, but far from light reading. Mr. Oechsli says that Atlantic has hired an official Evaluation Translator to make some of this material easier to digest. The foundation’s evaluation and communications teams are closely integrated to ensure that lessons are pushed out to interested audiences.

But amid all the reams of documents, how do you know what bits to translate, or who might be interested? Is there time to sift through all that material to determine what’s really interesting, and to whom? Mr. Oechsli speculated, as an example, that analyses of failures are too scarce and would make useful reading — and he’s surely right on both counts. But beyond that? Foundations are not always the best judges of what’s interesting in their own work (their annual reports, for example, have raised frequent questions about cost-effectiveness).

In a parenthetical remark, almost sotto voce, Mr. Oechsli described the challenge as one of “evaluating the evaluations,” acknowledging with a subtle shrug that the job is slightly mind-bending.

Still, it’s an excellent discipline for any foundation (and there are a lot of them) that considers knowledge and learning to be among its most important achievements. “Evaluating the evaluations,” in any thorough and rigorous way, is probably beyond Atlantic’s ability in the short time it has left. But we should give the foundation points for trying — and for spotlighting one rich body of philanthropic assets that remains, in most cases, chronically under-exploited.

Tony Proscio