Module 16: Current Evaluation Policy

To further highlight the importance of program evaluations, most major development agencies have evaluation policies that guide and focus their program evaluations. A summary of a few of these extensive policy documents are provided here.

United States Agency for International Development (USAID)(1)

USAID’s evaluation policy was implemented in 2010 as a direct response to diminishing quality and quantity of program evaluations within the agency. The agency saw evaluations submitted to the document clearinghouse drop from 500 to only 170 over the course of 15 years, despite increased program spending. The policy creates a sharp distinction between process and impact evaluations, stating that process evaluations will be part of the standard institutional evaluations for every USAID project, while impact evaluations will be reserved for select, innovative programs where the program theory has not been scientifically proven.(2) The policy also calls for all USAID evaluations to be conducted by a third party, in an attempt to reduce bias in or misuse of evaluation findings. Also, the agency’s emphasis on transparency and increased access of evaluation findings to the global community is consistent with the objectives and best practices for evaluation dissemination. Supporters and critics of the policy both take issue with some of the very apparent loopholes that exist within the document; many times “unusual circumstances” allow for country programs to conduct their own evaluation (as opposed to contracting it to a third party) or do not require disclosure of evaluation findings.(3) Without a strong definition of "unusual circumstances," this ambiguity could allow a lapse in the rigor and transparency of USAID evaluations.  

United National Children’s Fund (UNICEF)(4)

After a peer-review of UNICEF evaluation strategies in 2006, the executive board met to refocus existing evaluation policy. The peer-review resulted in a number of recommendations that the Executive Board used to revise the policy. Some items suggested by the peer-review group included: a standardized evaluation policy, more predictable budgets for evaluation, and supporting and strengthening in-country offices to conduct evaluations. The full peer-review report is available online (UNICEF Peer Review of Evaluation Function) and serves as a great example of action items intended for evaluation client use. Like the USAID policy, a specific amount (3-5%) of program funds was to be allotted for the purpose of evaluation. Transparency and rigor in study design were also emphasized through the UNICEF document, though the Executive Board did not, like in the USAID policy, call for third party evaluators. While the UNICEF board wants ‘independent’ evaluations, the term independent is not explicitly defined.

Department for International Development (DFID)(5)

Similar to how the UNICEF policy was developed, DFID’s external review board recommended the development of an evaluation policy to standardize and subsequently strengthen the quality of program evaluations. This policy coincided with the knowledge that the DFID programming budget would be increasing significantly over the next few years, with the expectation that additional funding would be used effectively.  As a result, it was determined that improved evaluations could provide evidence of success and identify areas of need for funding prioritization.

DFID has developed evaluation policy as they would a health program, including objectives and indicators to measure progress towards full policy implementation. For example, one of the objectives is to “meet Paris Declaration commitments to commission more evaluation jointly;” in order to measure progress towards this goal, DFID plans to ensure that “at least half of DFID’s Country Programme Evaluations are carried out together with country and/or international partners by 2012/13." The emphasis on DFID’s policy document is putting policy into practice, as can be seen from the operationalization of the department’s policy commitments. A viable example of a working policy document, DFID has since rolled out the policy implementation process. This quality differentiates DFID’s policy from that of both USAID and UNICEF. 

Evaluation Policy Conclusions

Many similarities exist between the policies described above. First, each of the agencies has created evaluation polices with the understanding that evaluation is an invaluable resource in future programming efforts; lessons learned is a theme across all of these major agency regulations. Second, many elements of a successful evaluation dissemination and utilization plan are found throughout the policies: making results widely accessible, ensuring the utilization of the information in the evaluation reports, and certifying that the evaluation itself is credible and independent from the client. Future organizational research may choose to follow the model of these changes in order to determine the level of success and variety of factors that influence evaluation policy implementation.

Footnotes

(1) U.S. Agency for International Development. (2011). Evaluation: Learning from experience. USAID Evaluation Policy. Washington, DC: USAID. https://www.usaid.gov/sites/default/files/documents/2151/USAIDEvaluationPolicy.pdf.

(2) Savedoff, W. (2011). USAID announces serious commitment to evaluation and joins 3ie. Washington, DC: Center for Global Development. https://www.cgdev.org/blog/usaid-announces-serious-commitment-evaluation-and-joins-3ie.

(3) Birdsall, N. (2011). The new USAID evaluation policy is not geting enough attention. Washington, DC: Center for Global Development. https://www.cgdev.org/blog/new-usaid-evaluation-policy-not-getting-nearly-enough-attention.

(4) United National Children’s Fund Executive Board. (2007). UNICEF evaluation policy. United National Economic and Social Council. https://www.unicef.org/evaluation/files/2008_Evaluation_Policy_English.pdf.

(5) Department for International Development. (2009). Building the evidence to reduce poverty: The UK’s policy on evaluation for international development. London, UK: Department for International Development. https://reliefweb.int/sites/reliefweb.int/files/resources/86E6BFEE2E9607BB4925765B001CC86B-dfid_jun2009.pdf.