Print This Page
Print This Page |
Evaluation activities, measures and criteria
| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z |
| A (Top) | |
| Accountability | Assumption |
| Activity | Attribution |
| Analytical method | |
| B (Top) | |
| Baseline | Beneficiary |
| Benchmark | Bias |
| C (Top) | |
| Case study | Contribution |
| Causal relationship | Control group |
| Cluster evaluation | Cost-benefit analysis |
| Conclusions | Cost-effectiveness |
| Condition | Criteria |
| Content analysis | |
| D (Top) | |
| Data | Data source |
| Data collection method | Discretionary external evaluation |
| Data collection tool | Discretionary internal evalution/Self-evaluations |
| E (Top) | |
| Effect | Evaluation scope |
| Effectiveness | Evaluation team |
| Efficiency | Evaluator |
| End-user | Evidence |
| Evaluability | External factor |
| Evaluation | External evaluation |
| F (Top) | |
| Feedback | Formative evaluation |
| Focus group | |
| G (Top) | |
| Goal |
| I (Top) | |
| Impact | Input |
| Indicator | Interim performance measure |
| Indicator of achievement | Internal evaluation |
| Indirect effect | In-depth evaluation |
| L (Top) | |
| Learning | Lesson learned |
| Legislative mandate | Logical framework |
| M (Top) | |
| Mandatory external evaluation | Methodology |
| Mandatory internal evaluation/Self-assessments | Methodology form |
| O (Top) | |
| Objective | Outlier |
| Outcome | Output |
| P (Top) | |
| Project | Project evaluation |
| Project document | Proxy indicator |
| Q (Top) | |
| Qualitative data | Quantitative data |
| R (Top) | |
| Rating system | Result |
| Relevance | Revised baseline |
| Reliability | Revised target |
| S (Top) | |
| Sample | Subprogramme |
| Self-monitoring | Survey |
| Stakeholder | Synthesis |
| T (Top) | |
| Target | Terms of reference |
| Target group | Triangulation |
| U (Top) | |
| Universe | |
| V (Top) | |
| Validity | Variable |
Accountability (Back / Top)
Obligation for a manager of resources to demonstrate that work has been conducted in compliance with the established plans, budgets, rules and standards and to report fairly and accurately on performance results. It includes responsibility for the justification of expenditures, decisions or results of the discharge of authority and official duties, including duties delegated to a subordinate unit or individual. The effective discharge of accountability is predicated on clearly defined responsibilities, performance expectations, limits of authority, and clarity on how the exercise of responsibility and authority will be monitored and assessed. One of the main functions of monitoring and evaluation is to contribute to strengthening accountability by providing objective information on the veracity of a manager’s reporting.
See also: “Audit”. “Evaluation”, “Inspection”
Activity (Back / Top)
Action taken or work performed to transform inputs into outputs.
See also: “Input,” “Output”, “Result”
Analytical method (Back / Top)
A means to process, understand and interpret data.
See also: “Data collection tool”
Assumption (Back / Top)
Hypothesis about risks, influences, external factors or conditions that could affect the progress or success of a project or a programme. Assumptions highlight external factors, which are important for the success of project or programme, but are largely or completely beyond the control of management.
See also: “External factor”, “Condition”, “Effect”, “Logical framework”
Attribution (Back / Top)
A causal link between observed (or expected to be observed) changes and a specific intervention. Attribution refers to that which is to be credited for the observed changes or results achieved. With regard to attribution for the achievement of accomplishments/results, evaluations aim to ascertain a credible link between outputs and achieved accomplishments. In assessing this link, attribution takes account of the effects of other interventions that are independent of the effort being evaluated.
See also: “Contribution”, “External Factor”, “Causal Relationship”
Baseline (Back / Top)
Data that describe the situation to be addressed by a programme, subprogramme or project and that serve as the starting point for measuring performance. A baseline study would be the analysis describing the situation prior to the commencement of the programme or project or the situation following initial commencement of the programme or project to serve as a basis of comparison and progress for future analyses. It is used to determine the accomplishments/results and serves as an important reference for evaluation.
See also: “Target”
Benchmark (Back / Top)
Reference point or standard against which performance or achievement can be assessed. A benchmark often refers to an intermediate target to measure progress within a given period as well as to the performance of other comparable organizational entities.
See also: “Interim performance measures”
Beneficiary (Back / Top)
The individual, group, or organization, whether targeted or not, that benefit, directly or indirectly, from the implementation of a programme, project or output.
See also: “End-user”, “Target group”
Bias (Back / Top)
Anything that produces systematic error in an evaluation finding. Bias may result in over- or under-estimating the object of evaluation or assessment.
See also: “Attribution”,”Causal relationship”, “Evidence”
The examination of the characteristics of a single case (such as an individual, an event, a programme or some other discrete entity). A sample of multiple cases can also be examined to look for commonalities and to identify patterns. Case studies are often used to gather qualitative information in support of findings obtained through quantitative methods.
See also: “Qualitative data”, “Lesson learned”, “Learning”
A logical connection or cause-effect linkage ascribed to the relationship between accomplishments/results and efforts to achieve them or between final results and their impact on the target beneficiaries. Generally the term refers to reliably plausible linkages.
See also: “Assumption”, “Attribution”, "Bias”, “Evidence”, “Logical framework”, “Indirect effect”
An evaluation of a set of related outputs, projects and/or programmes or subprogrammes.
See also: “Evaluation”
Conclusions present reasoned judgments based on a synthesis of empirical findings or factual statements corresponding to specific circumstances. Conclusions point out the factors of success and failure of the evaluated projects and programmes, with special attention paid to the intended and unintended results and impacts, and more generally to any other strength or weakness. Conclusions draw on data collection and analyses undertaken, through a transparent chain of arguments.
See also: “Finding”, “Lesson learned”, “Terms of reference”
The situation that exists, and that the evaluator seeks to describe and explain. Condition can describe how well a programme is operating or the problem it is facing.
See also: “Finding”
A systematic approach to analyzing themes in audio, visual, electronic or print communication. Selected material is reviewed and assessed on the basis of predetermined criteria (such as the reflection of key messages, accuracy, prominence, and reference to sponsoring organization).
See also: “Data”, “Data collection method”
A selected subgroup of beneficiaries who do not receive the same treatment, input or training, but share characteristics similar to the target group. Thus, differences between the control group and the target group can be measured and evaluated.
See also: “Target group”
The link between the activities carried out by various organizational units to arrive at final products or services delivered to end-users to accomplish a desired result.
See also: “Attribution”, "Causal relationship”
A specialized analysis which converts all costs and benefits to common monetary terms and then assesses the ratio of results to inputs against other alternatives or against some established criteria of cost-benefit performance. It often involves the comparison of investment and operating costs with the direct and indirect benefits generated by the investment in a project or programme.
See also: “Cost-effectiveness”, “Efficiency”
Comparison of the relative costs of achieving a given result or output by different means. It focuses on the relation between the costs (inputs) and results produced by a project or programme. A project/programme is more cost effective when it achieves its results at the lowest possible cost compared with alternative projects with the same intended results.
See also: “Cost-benefit analysis”
The standards used to determine whether or not a programme or project meets expectations.
See also: “Objective”
Specific quantitative and qualitative information or facts that are collected.
See: “Data collection tool”, ”Data collection method”, “Data source”
The mode of collection to be used when gathering information and data on a given indicator of achievement or evaluation. Collection methods include the review of records, surveys, interviews, or content analysis. The term is one of the elements of the indicator methodology form for reporting through IMDIS.
See also: “IMDIS”, “Methodology form”
Instrument used to collect information for use in performance assessment, self-evaluation and external evaluation. Examples are mail, telephone, in-person and web-based surveys, direct or participatory observation, interviews, focus groups, expert opinion, case studies, literature search, and content analysis of internal and external records. The data collection tools must be strong enough to support the findings of the evaluation.
The origin of the data or information collected. Data sources may include informal and official records, individuals, documents, etc. This term is one of the elements of the indicator methodology form for reporting through IMDIS.
See also: “IMDIS”, “Methodology form”
An evaluation requested by programme managers inthe United Nations Secretariat and managed and conducted by entities outside the programme being evaluated such as theJoint Inspection Unit (JIU) or the Office of Internal oversight Services (OIOS), or any external evaluator. In this type of evaluation, the programme manager requests the JIU, OIOS or the external evaluator, to design, manage and/or conduct the evaluation, while the programme manager's role is as the "evaluaee".
Discretionary external evaluations are primarily useful for programme managers who wish to improve the performance of their programmes on the basis of objective assessments, though they may also be useful to support decision-making at the governance level.
See also: "External evaluation", "Internal evaluation", "In-depth evaluation"
These are optional, non-mandatory evaluations conducted by programme managers for their own use. They address issues that are over and above those covered by Mandatory Internal Evaluation/Self-assessments. They are normally not required to be reported upon at the intergovernmental level. While manages may contract external consultants and specialists to help with the exercise, these evaluatons are considered internal or self-evaluations because the programme managers remain responsible for the design and/or management of the evaluation, and are ultimately responsible for the quality of the reports and for using the results to improve operations. Such self-evaluations are useful when formulating best-practices and lessons-learned and can cover cross-cutting issues relevant to a number of subprogrammes.
See also: "External evaluation", "Internal evaluation", "Mandatory internal evaluation/Self-assessments"
Intended or unintended change caused directly or indirectly by the delivery of an output, project or programme.
See also: “Assumption”, “Attribution”
The extent to which a project or programme attains its objectives, expected accomplishments and delivers planned outputs.
A measure of how well inputs (funds, expertise, time, etc.) are converted into outputs.
See also: “Effectiveness”, “Impact”
Recipient of an output or accomplishment.
See also: “Stakeholder”, “Beneficiary”
Extent to which an activity or a programme can be evaluated in a reliable and credible fashion.
See also: “Evaluation”, “Terms of reference”
A process that seeks to determine as systematically and objectively as possible the relevance, effectiveness and impact of an ongoing or completed programme, project or policy in the light of its objectives and accomplishments. It encompasses their design, implementation and results with the view to providing information that is credible and useful, enabling the incorporation of lessons learned into both executive and legislative decision-making process. Evaluation is often undertaken selectively to answer specific questions to guide decision-makers and/or programme managers, and to provide information on whether underlying theories and assumptions used in programme development were valid, what worked and what did not work and why.
See also: “Cluster evaluation”, “Ex-post evaluation”, “External evaluation”, “Formative evaluation”, “In-depth evaluation”, “Internal evaluation”, “Joint evaluation”, “Meta-evaluation”, “Participatory evaluation”, “Project evaluation", “Self-evaluation”, “Strategic evaluation”, “Summative evaluation”, “Thematic evaluation”
A framework that establishes the focus of an evaluation in terms of questions to address, the issues to be covered, and defines what will be analyzed and what will not be analyzed. The scope defines the parameters of the evaluation and is presented in the “Terms of Reference”.
See also: “Terms of reference”
Group of specialists responsible for the planning and conduct of an evaluation. An evaluation team produces the evaluation report.
See also: “Evaluation”, “Terms of reference”
An individual involved in all stages of the evaluation process, from defining the terms of reference and collecting and analyzing data to developing findings and making recommendations. The evaluator may also be involved in taking corrective action or making improvements.
See also: “Evaluation team”
The information presented to support a finding or conclusion. Evidence should be sufficient, competent and relevant. There are four types of evidence: observations (obtained through direct observation of people or events); documentary (obtained from written information); analytical (based on computations and comparisons); and self-reported (obtained through, for example, surveys).
See also: “Assumption”, “Attribution”, “Bias”, "Causal relationship”, “Indirect effect”
Event and/or condition that is beyond the control of those responsible for an activity but that has an effect on the success or failure of the activity. It may be anticipated in the form of assumptions or they may be unanticipated.
See also: “Assumption”, “Attribution”, “Causal relationship”, “Logical framework”An evaluation performed by entities outside of the programme being evaluated. Generally, it is intergovernmental organs that commission such evaluations and receive final reports on them. As a rule, external evaluation of a project, programme or subprogramme is conducted by entities free of control or influence by those responsible for the design and implementation of the project and programmes.
In the United Nations Secretariat, there are two types of external evaluation, namely: (1) Mandatory External Evaluation and (2) Discretionary External Evaluation. (Under exceptional circumstances, the Secretary-General may commission high level external evaluation undertaken by renowned authorities or experts)
See also: "Evaluation", "Internal evaluation", "Mandatory external evaluation", "Discretionalry external evaluation"
A person who helps members of a group to conduct an evaluative meeting and achieve the result for which they came together. A Facilitator should have no vested interest in the outcome of the meeting, nor dictate its substance.
See also: “Results-based management (RBM)”, " Participatory evaluation”
A process consisting of the transmission of relevant information from monitoring and evaluation exercises to targetted users so as to facilitate learning and decision-making. Such information usually comprises findings, conclusions, recommendations and lessons learned.
See also: “Lesson learned”, “Learning”, “Best practice”
A group of individuals having some common interest or characteristics, brought together by a moderator, who uses the group and its interaction as a way to gain information about a specific or focused issue.
See also: “Data collection method”, “Facilitator”
Sometimes known as interim evaluation, it is conducted during implementation phase of projects or programmes to improve their performance. Formative evaluations may also be conducted for other reasons such as compliance, legal requirements or as part of a larger evaluation initiative. It is intended for managers and direct supporters of a project.
See also: “Evaluation”
The higher-order aim to which a programme is intended to contribute: a statement of longer-term intent.
See also: “Objective”, “Accomplishment“, "Result ”
The overall effect of accomplishing specific results. In some situations it comprises changes, whether planned or unplanned, positive or negative, direct or indirect, primary and secondary that a programme or project helped to bring about. In others, it could also connote the maintenance of a current condition, assuming that that condition is favourable. Impact is the longer-term or ultimate effect attributable to a programme or project, in contrast with an expected accomplishment and output, which are geared to the biennial timeframe.
See also: “Effect”, “Evaluation”, “Ex-post evaluation”
A measure, preferably numerical, of a variable that provides a reasonably simple and reliable basis for assessing achievement, change or performance. A unit of information measured over time that can help show changes in a specific condition.
See: “Logical framework”, “Results-based management (RBM)”, "Methodology form"
Used to measure the extent to which expected accomplishments have been achieved. Indicators correspond to the expected accomplishment for which they are used to measure performance. One expected accomplishment can have multiple indicators.
See also: “Expected accomplishment”, “Performance management”, " Performance measure”
The unplanned changes brought about as a result of implementing a programme or a project.
See also: “External factor”, “Assumption”
Personnel, finance, equipment, knowledge, information and other resources necessary for producing the planned outputs and achieving expected accomplishments.
See also: “Output ”
Data collected for the indicators of achievement during the biennium. They are used as benchmarks to determine whether progress is being made towards the intended results.
See also: “Performance measure”, “Baseline”, “Target”.
Evaluation that is managed and/or conducted by entities within the programmes being evaluated. There are two types of internal evaluation, namely:
(1) Mandatory Internal Evaluation (Self-assessmetns)
(2) Discretionary Internal Evaluation (Self-evaluation)
See also: "External evaluation", "Self-evaluation”, "Mandatory internal evaluation/Self-assessments", "Discretionary internal evaluation/self-evaluation", “Accomplishment account”, “Results-based management (RBM)”
Refers to the scope of the evaluation and is not to be considered a category or type of evaluation.
Both internal and external evaluation can be "in-depth" when they take a comprehensive and broad-ranging review of the work of a programme or organizational entity. In-depth evaluation can be undertaken by external entities or by programme managers. The objective is to make recommendations that will help the Senior Managers, intergovernmental bodies or the General Assembly to formulate decisions aimed at increasing the overall relevance, effectiveness and impact of the programmes of the United Nations.
See also: “Evaluation”, "External evaluation", "Internal evaluation"
Reflection on experience and results of monitoring and evaluation to identify how a situation or future actions could be improved and then using this knowledge to make actual improvements. This can be individual or group-based – the latter would be considered “organizational” learning when it transcends the group and becomes part of the corporate knowledge base. Learning involves documenting experiences, the process of synthesis and the application of lessons learned to future actions.
See also: “Feedback”, “Best practice”
A request for action addressed to the Secretary-General or an executive head acting on his behalf in a resolution or decision adopted by a competent United Nations intergovernmental organ.
Generalization derived from evaluation experiences with programmes, projects or policies that is applicable to a generic situation rather than to a specific circumstance and has the potential to improve future actions. A lesson learned summarises knowledge at a point in time, while learning is an ongoing process.
See also: “Best practice”, “Learning”
Management tool (also known as a logframe) used to identify strategic elements of a programme or project (objective, expected accomplishments, indicators of achievement, outputs and inputs) and their causal relationships, as well as the assumptions and external factors that may influence success and failure. It facilitates planning, implementation, monitoring and evaluation of a programme or project.
See also: “Budget fascicle”, “Results-based management (RBM)”
Mandatory external evaluation (Back / Top)
An evaluation mandated by an Intergovernmental body that is managed and performed by an entity outside the programme being evaluated. Mandatory external evaluations are mandated by bodies such as the Committee for Programme and Coordination (CPC), functional commissions, regional and sectoral intergovernmental bodies and other technical bodies. Mandatory external evaluatons are generally managed and conducted by the Joint Inspection Unit (JIU) as part of its UN-wide mandate, and by the Office of Internal Oversight Services (OIOS), as well as by appointed external evaluators.
Mandatory external evaluations are primarily useful for oversight and support to decision-making at the governance level, while they also may be useful for programme managers.
See also: "External evaluation", "Internal evaluation", "In-depth evalaution"
Mandatory internal evaluation/Self-assessments (Back / Top)
These are compulsary assessments performed by programme managers once every two years. These assessment are required for all United Nations Secretariat programmes and are framed by the logical frameworks in the approved bienniual programme budget documents. They are reported on through the biennial Programme Performance Report (PPR). They use information generated by measuring the extent to which Expected Accomplishments were achieved through the collection of indicator data and use of IMDIS tool as a means to record progress and report results and accomplishments. They may also use information generated from other types of evaluation and assessment, as appropriate.
See also: "External evaluation", "Internal evaluation"
A set of analytical methods and techniques appropriate for evaluation of the particular activity. It could also be aimed at collecting the best possible evidence needed to answer the evaluation issues and analytic questions.
See also: “Baseline”, “Benchmark”, “Data”, “Data collection tool”, “Indicator”, "Terms of reference"
A set of fields developed and incorporated into IMDIS for each indicator of achievement that allow subprogrammes and programmes to identify and document the elements of the indicator that will be measured during the biennium, identify data sources, determine data collection and verification methods, fix the periodicity of measurements, create or identify a presentation format and identify external factors that could distort or influence measurements. This should be done early in the biennium to ensure proper data collection and reporting of results.
See also: “IMDIS”
Description of an overall desired achievement involving a process of change and aimed at meeting certain needs of identified end-users within a given period of time. A good objective meets the criteria of being impact oriented, measurable, time limited, specific and practical. The objective is set at the next higher level than the expected accomplishments.
See also: “Goal“, “Expected accomplishment”, “Logical framework”
In the United Nations Secretariat, “outcome” is used as a synonym of an accomplishment or a result.
See also: “Accomplishment”, “Expected accomplishment”, “Logical framework”, “Result”, “Results-based management (RBM)”
A subject or other unit of analysis that has extreme values. Usually outliers are excluded from statistical tabulations because they are considered unrepresentative or unreliable.
See also: “Data”
A final product or service delivered by a programme or project to end-users, such as reports, publications, servicing of meetings, training, advisory, editorial, translation or security services, which a programme is expected to produce in order to achieve its expected accomplishments and objectives. Outputs may be grouped into broader categories.
See also: “IMDIS”, “Logical framework”, “Additional output”, “Discretionary action”
Planned activity or a set of planned, interrelated activities designed to achieve certain specific objectives within a given budget, organizational structure and specified time period. Within the Secretariat, projects are used in technical cooperation activities.
See also: “Programme”
Evaluation of an individual project designed to achieve specific objectives within specified resources, in an adopted time span and following an established plan of action, often within the framework of a broader programme. The basis of evaluation should be built into the project document.
See also: “Evaluation”
A formal document covering a project, which sets out, inter alia, the needs, results, outputs, activities, work plan, budget, pertinent background, supporting data and any special arrangements applicable to the execution of the project in question. Once a project document is approved by signature, the project represents a commitment of resources.
See also: “Project evaluation”, "Project/Programme cycle management”
Proxy indicator is used when it is difficult to identify direct indicators to measure the result. Proxies are indicators that may tell us indirectly whether a result has been achieved.
See also: “Indicator of achievement”, “Causal relationship”, “Indicator”, “Data”
Information that is not easily captured in numerical form (although qualitative data can be quantified). Qualitative data typically consist of words and normally describe people's opinions, knowledge, attitudes or behaviours.
See also: “Data”
Information measured or measurable by, or concerned with, quantity and expressed in numerical form. Quantitative data typically consists of numbers.
See also: “Data”
Forming and validating a judgment on the relevance, performance and success of a programme or project through the use of a scale with numeric, alphabetic and/or descriptive codes.
See also: “Data”
The extent to which an activity, expected accomplishment or strategy is pertinent or significant for achieving the related objective and the extent to which the objective is significant to the problem addressed.
See also: “Evaluation”
Consistency or dependability of data and evaluation judgements, with reference to the quality of the instruments, procedures and analyses used to collect and interpret evaluation data. Evaluation information is reliable when repeated observations using similar instruments under similar conditions produce similar results.
See also: “Evaluation”
The measurable accomplishment/outcome (intended or unintended, positive or negative) of a programme or project. In the Secretariat practice, “result” is synonymous with accomplishment and outcome.
See also: “Accomplishment”, “Expected accomplishment”, “Outcome”, “Effect”, “Impact”, “Results-based management (RBM)”
An adjusted baseline recorded in IMDIS in cases where the actual figures recorded in the previous biennium differ from the estimated baselines in the proposed programme of work.
See also: “Baseline”, “Target,”, “IMDIS”
An adjusted target recorded in IMDIS in cases where the estimated baselines and targets in the proposed programme of work have to be adjusted to capture actual achieved results from the previous biennium and realistic achievements for the current biennium.
See also: “Baseline”, “Target,”, “IMDIS”
The selection of a representative part of a universe in order to assess parameters or characteristics of that universe. Random sampling is the selection of a group of subjects (the sample) from a larger group (the population or universe), so that each individual or other unit is chosen entirely by chance.
See also:“IMDIS”, “Results-based management (RBM)”, “Monitoring”
Agencies, organizations, groups or individuals who have a direct or indirect role and interest in the objectives and implementation of a programme or project and its evaluation. In participatory evaluation, stakeholders assume an increased role in the evaluation process as question-makers, evaluation planners, data gatherers and problem solvers.
See also: “End-user”, “Beneficiary”
A subprogramme consists of activities within a programme aimed at achieving one or a few closely related objectives as set out in the strategic framework. The subprogramme structure shall correspond, to the extent possible, to an organizational unit, normally at the division level.
See also: “Budget fascicle”
Systematic collection of information from defined respondents through the administration of a structured questionnaire in person, by telephone, by mail, or by web. A survey can be administered to the universe of all respondents or to a sample of that universe.
The process of identifying relationships between variables and aggregating data with a view to reducing complexity and drawing conclusions.
See also: “Data”
A specified objective that indicates the number, timing and location of what is to be achieved.
The main beneficiaries of a programme or project that are expected to gain from the results of that programme or project. They are closely related to its impact and relevance.
See also: “End-user”, “Beneficiary”, “Stakeholder”
Written document presenting the purpose and scope of the evaluation or inspection, the methods to be used, issues to be addressed, the resources, schedule, and reporting requirements.
See also: “Evaluation”
The use of three or more methods to conduct an evaluation or substantiate as assessment. By combining multiple data sources or methods evaluators seek to overcome the bias that comes from single informants and single methods.
See also: “Data”, “Evaluation”, “Bias”, “Best practice”
A group of persons (or other subjects of study) that the evaluation wishes to describe or generalize about. Also called “population”.
The extent to which the data collection methods or tests accurately measure what they are supposed to. Valid evaluations are ones that take into account all relevant factors, given the whole context of the evaluation, and weigh them appropriately in the process of formulating conclusions and recommendations.
See also: “Data”
See also: “Methodology form”