Welcome to the United Nations
Language:
  1. Dev - Evaluation Handbook

Dev - Evaluation Handbook

Introduction

This Evaluation Handbook aligns with the United Nations Secretary General’s commitment to strengthening the evaluation capacity of the UN Secretariat to better inform programme planning and design and enhance reporting on programme performance. Evaluation is also fundamental to the United Nations Office of Counter-Terrorism (UNOCT) efforts to ensure the Office is accountable for the funds received and transparent about progress in achieving its intended results. Equally important is that evaluation enables decisionmakers to learn from successes and challenges, better enabling the achievement of programmes that are transformative, relevant, timely, gender sensitive, human rights-responsive, inclusive, and impactful.  

The UNOCT Evaluation Policy provides the overall framework for the planning and undertaking of evaluation. As the custodian of the evaluation function, the Evaluation Compliance Unit (ECU) has responsibility for implementing this policy with accompanying guidelines and supporting continuous improvement of the quality and credibility of evaluations in line with the United Nations Evaluation Group (UNEG) Norms and Standards and with international good practice.

 

Purpose and Audience

The purpose of this Evaluation Handbook is to provide concrete guidance on how the Evaluation Policy is to be carried out across the Office. The intended audience is all UNOCT staff, and in particular programme and project managers who have responsibility for managing evaluations, as well as their programming units and senior management. It is also an important resource for independent evaluators who need to understand not only the guiding principles, standards, and processes for evaluations within the UNOCT context, but also where the intervention under review sits within the structure of the organization and the UN system. The UNOCT Evaluation Handbook is not a manual for how to design and conduct evaluations. It is expected that contracted evaluators will have this expertise, and there are plenty of resources that provide detailed information on basic methods and on more complex and emerging methodological approaches - several of these are referenced at the end of this document. 

 

Structure and Contents

This handbook contains eight chapters. It is designed so that the chapters can be read in sequence or as stand-alone sections. It is intended to be updated as needed to respond to changes in UNOCT policy and approaches as well as to changing needs for evaluation guidance.

 

 

Resources

The Annexes contain practical information as well as tools and templates for use in the evaluation planning and implementing phases and are referenced throughout the chapters of the handbook. Particularly useful for those overseeing evaluations are management checklists for each of the three evaluation categories; these cover each step and the roles and responsibilities of those supporting the process along the way. Evaluators should note that the report templates are required to be used for each deliverable.

  • Annex 1 – Glossary of Terms
  • Annex 2 - Examples of Evaluation Questions
  • Annex 3(a) - Evaluation Management Checklist for Centralized Evaluations
  • Annex 3(b) - Evaluation Management Checklist for Independent Programme/Project Evaluations
  • Annex 3(c) - Evaluation Management Checklist for Internal Evaluations
  • Annex 4 - Evaluation Report Quality Checklist
  • Annex 5 - UNEG Pledge of Ethical Conduct in Evaluation
  • Annex 6 – Comparisons of Tasks and Responsibilities by Type of Evaluation
  • Annex 7 – Templates
  • Annex 7(a) - Evaluation Selection Criteria Template
  • Annex 7(b) - Evaluation Proposal Template
  • Annex 7(c) – Terms of Reference Template
  • Annex 7(d) - Stakeholder Mapping Matrix Template
  • Annex 7(e) - Evaluation Matrix Template
  • Annex 7(f) – Inception Report Template
  • Annex 7(g) - Evaluation Report Template: Centralized Evaluations
  • Annex 7(h) - Evaluation Report Template: Independent Programme/Project Evaluations
  • Annex 7(i) - Evaluation Report Template: Internal Evaluations
  • Annex 7(j) – Management Response & Tracking Template
  • Annex 7(k) – Evaluation Brief Template

Chapter 1: Evaluation and Why It Matters

Evaluation and Why It Matters provides a basic introduction to evaluation and discusses how it differs from other types of assessment and how it links to results-based management. The chapter also explains why ‘measuring impact’ and ‘impact evaluations’ do not always mean the same thing, and what UNOCT’s International Hub on Behavioral Insights to Counter Terrorism (BI Hub) is doing to increase the rigor of programme design and evaluation processes.   

Chapter 4: Planning, Costing and Preparing for Evaluations

Planning, Costing and Preparing for Evaluations goes into more detail on the steps to be followed before the evaluation begins. It is primarily directed to evaluation managers as it covers the evaluation proposal, scheduling, further guidance on budgeting, how to develop the terms of reference (ToR) for the evaluation, and recruitment of the evaluation team. The Planning Phase and the Preparation Phase are the initial phases of any evaluation process.

Chapter 7: Quality Assurance and Assessment

Quality Assurance and Assessment highlights the features of a credible evaluation process. Both evaluation managers and evaluators are encouraged to review this chapter periodically as it succinctly covers the most important things to look for when conducting quality control of data collection and of each deliverable. 

Chapter 2: Evaluation in the UN System

Evaluation in the UN System gives an overview of the standards, expectations, and ethical principles that all evaluations carried out by UN entities are required to follow, including the requirements for integrating human rights, gender equality, leave no one behind, and disability inclusion. It also situates this work within the Sustainable Development Goals (SDGs) and the ‘Delivering As One UN’ agenda and provides guidance on conducting joint evaluations. This is an important chapter for all audiences. 

Chapter 5: Implementing and Managing Evaluations

Implementing and Managing Evaluations covers the steps involved in the third and main phase, and as such, it is important for all stakeholders who participate in evaluation processes, including the evaluation team. The Implementation Phase encompasses the initial briefing of the evaluators and others most closely involved in the evaluation process, the inception/design stage of the evaluation, the data collection and analysis stage, and the development of the draft and final evaluation reports, as well as the evaluation brief. 

Chapter 8: Practical Guide

Practical Guide presents fundamental aspects of good evaluation practice. It will be most useful to those who are less familiar with the field of evaluation, helping to demystify terms such as theory of change, evaluation matrixes, stakeholder mapping, sampling frameworks, and most significant change. It introduces different types of evaluation design and a range of data collection and analysis methods.

Chapter 3: UNOCT Evaluation Function

UNOCT Evaluation Function lays out the guidelines for how the Office’s evaluations are structured, planned, and implemented. It introduces the Evaluation Policy, the Evaluation Compliance Unit, and the different categories and types of evaluations within UNOCT, as well as roles and responsibilities for each. This chapter is of importance to Programme and Project Managers as it also explains the preliminary evaluation planning and budgeting process.

Chapter 6: Using Evaluations Results

Using Evaluation Results discusses the final phase; the Follow-up Phase that needs to take place once the evaluation report has been delivered. The steps include planning the activities and who should be involved, the development of the Management Response (which addresses the recommendations emerging from the evaluation), the formal presentation of results by the evaluators, and tips for effectively communicating results.