Monitoring/Evaluation of Policies

Theme: Statistics, Data and Evaluation, and Monitoring
Programme Monitoring and Evaluation; The Disability Perspective in the Context of Development

Previous | Contents | Next

I. INTRODUCTION

A. Purpose of the report

Since its founding, the United Nations (UN) has been deeply concerned with both human rights and development. Throughout most of its history, UN has also focused on issues concerning disabled persons. In some sense, the effectiveness of policies to promote all human rights for disabled persons and the involvement of people with disabilities in all aspects of social and economic development constitutes a basic measure of the extent to which policies in the areas of human rights and development are indeed effective.

The adoption of the World Programme of Action concerning Disabled Persons[1], by General Assembly resolution 37/52 of 3 December 1982 commits the UN "...to promote effective measures for prevention of disability, rehabilitation and the realization of the goals of 'full participation' of disabled persons in social life and development, and of 'equality'. This means opportunities equal to those of the whole population and an equal share in the improvement in living conditions resulting from social and economic development."[2] Thus, the UN recognized the need to integrate disability issues into human rights and development efforts.

The World Programme of Action anticipates the need for ongoing monitoring and evaluation activities to assure Programme effectiveness.[3] Indeed, the Programme recognizes that it is essential to for the UN system to conduct periodic assessments, with the establishment both of baselines to measure progress towards its goals and of appropriate indicators for evaluation in consultation with Member States. Reviews of the Programme are to take place every five years, with a view towards potential revisions. The first and second reviews took place in 1987 and in 1992 during the United Nations Decade of Disabled Persons (1983-1992).[4]

This report builds upon the third review and appraisal of the World Programme of Action concerning Disabled Persons, which was considered by the General Assembly at its fifty-second session in 1997.[5] The reports focuses on activities during the period of1993 to 1997, which constitutes the first five years after the United Nations Decade of Disabled Persons (1983-1992). The report has two broad purposes: First, progress towards and obstacles to achievement of the purposes of the World Programme of Action concerning Disabled Persons since 1992 will be reviewed and appraised. Second, options for improving the implementation of World Programme monitoring, including implications of developments in statistical information and indicators on disability, will be demonstrated.

B. The approach to this report

Any review and appraisal such as this one is at least somewhat influenced by some of the values of the time in which it is written. While a goal of complete objectivity is desirable, it is important at least to document the particular points of view that prevailed in the analyses that have been conducted. This report takes several different approaches to review and appraisal, as well as to the issues of monitoring and of indicators. The emphases are as follows: (1) a long-range historical perspective, (2) the mainstreaming of disability issues into general policies, (3) the importance of definitions, (4) the practical use of data and indicators for World Programme implementation and (5) the use of case studies to demonstrate effective analytical approaches. A brief review of each may inform the reader of the ideas underlying the conclusions drawn.

First, although the period under review constitutes the five years since 1992, events during that period are seen as part of long-range historical trend. This is important, because, as this review demonstrates, the last fifteen years have witnessed a major shift in thinking on disability issues. To document exclusively the last five years might be misleading, because many of the developments are a function of events in the preceding ten or, perhaps, even twenty years. Hence, a historical review is provided in this document.

Second, as indicated in the first paragraph in this report, disability issues are viewed as a critical component of overall policies for human rights and development. The point of view represented is that the extent to which these policies are applicable to persons with disabilities is an indicator of the effectiveness of their overall implementation. However, the incorporation of disability issues into these policies may have the effect of altering these policies. Thus, the focus on mainstreaming goes beyond simple integration of disabled persons into various social and economic sectors.

Third, perhaps more than in other issues, the particular definitions employed for disability have a profound impact on policy formulation, implementation and monitoring. Indeed, perhaps the one firm conclusion that can be drawn about disability is that there is no general agreement as to what it is, particularly in an international context. The World Programme of Action employs definitions from the World Health Organization's (WHO) International Classification of Impairments, Disabilities and Handicaps (ICIDH)[6]. However, the World Programme does not necessarily employ one definition of disability at any particular time. With its focus on mainstreaming, this report tends to employ a social viewpoint on disability, but even this viewpoint is subject to different interpretations. The impact of this definitional ambiguity goes beyond activities related to data collection and analysis to the basic question of identifying exactly who is the target of disability policy. To some extent, the issue is mitigated when one views the targets of human rights and development as all people and disability, defined in many ways, is viewed as an element that should not be employed to limit the applicability of these policies to all persons. One then focuses on disability not as a problem but as an opportunity to further expand the applicability of policies. Even with this approach, one must understand the particular definition of disability used in any particular situation to implement effectively both disability-specific and general policies. This theme will be stressed time and again in this report.

Fourth, data and indicators are viewed as tools for effective World Programme implementation. This is not to claim that the only use for data is to monitor formal policies. There are many uses of data. Moreover, the socio-economic phenomena on which data are collected are subject to many different influences apart from disability programmes. Those other uses of data are not discussed extensively in this report. Rather, the important point to remember is that suggestions made here for analyzing data for World Programme implementation may not be applicable for other uses of disability data.

Finally, a case studies approach is employed to demonstrate the use of data for monitoring and evaluation. The World Programme of Action clearly anticipated periodic evaluations, employing data-driven baselines and indicators. However, such an approach has not been institutionalized in a formal way at the international level. It would be premature and probably impossible for this report to recommend a definitive approach for monitoring the World Programme of Action. What the report does present is an approach to organizing and using data for Programme monitoring and evaluation using the experiences of certain countries as case studies. The case studies demonstrate the feasibility of such activities but also allow for many alterations as evaluation systems develop. This approach has served the United Nations well in the past. In 1986, case studies employing data from Egypt, Iraq, Jordan, Lebanon and the Syrian Arab Republic demonstrated that an international effort to systematically compile and analyze data on disability was highly feasible.[7] This demonstration resulted in the creation and dissemination by the United Nations Statistics Division of the widely used UN Disability Statistics Data Base (DISTAT).[8] Similarly, it is hoped that this use of case studies will play a role in assisting in development of a system for monitoring the World Programme of Action employing global disability indicators.

C. Organization of this report

The report is in three parts. The first purpose of this report, to appraise progress and obstacles in World Programme implementation constitutes the first part. The second purpose, to present options for programme monitoring is covered in the second and third parts dealing with monitoring issues and global disability indicators, respectively.

Such a division is necessary for at least two reasons. First, approaches to monitoring encompass many factors larger than the establishment of indicators, including both conceptual issues and issues in data development independent of the indicators. However, the monitoring approach chosen can have a huge impact on the indicators selected. Hence, the approach must be selected first based on factors other than the criteria for selecting the indicators. Second, once an approach to monitoring is selected, the development of indicators is a complex undertaking that must take into account a variety of aspects related to both concepts and data.

The review of the progress and obstacles in Programme implementation consists of a historical review of UN efforts in the field of disability through the end of the UN Decade for Disabled Persons in 1992 and a general discussion of trends in policies and programmes from the disability perspective since 1992 prior to the current review and appraisal. The historical review does not purport to be a definitive history of the involvement of the United Nations in disability issues, which is covered in another publication.[9] Rather, the focus is placed on the World Programme of Action, with brief discussions of events leading up to its adoption and its implementation during the United Nations Decade of Disabled Persons (1983-1992). In a similar manner, the general discussion of trends provides an overview of issues that have influenced implementation of the Programme worldwide and of both disability-related and general policy instruments adopted by the United Nations since 1992. The actual appraisal focuses on specific characteristics of policy instruments that may have influenced Programme implementation and on measures of progress and obstacles.

The discussion of monitoring is broken into issues related to conceptualizing monitoring activities and issues related to development of data. In determining options for conceptualizing monitoring, the desired and expected results of Programme activities and resource constraints are considered as parameters for setting priorities in both activities and monitoring those activities. In some sense, the approach is replicated in determining data options, with data availability and integration of disability data with other data collection efforts considered as parameters in establishing data priorities.

In 1995, by resolution 50/144, the General Assembly "encourage[d] the Secretary-General … to continue … efforts to facilitate the collection and transmission of relevant data to be used to finalize, in consultation with member States, the development of global disability indicators …". The section of this report on development of global disability indicators reviews and evaluates information currently available for development of indicators. Finally, through the use of the case studies, issues in the construction of indicators are considered in the hope of establishing a systematic approach to the development of global disability indicators.

D. Sources of data

The data sources for the current report include country-level data gathered by the Statistics Division of the United Nations for inclusion in DISTAT, the August 1995 results from the survey of the Special Rapporteur on Disability of the Commission for Social Development on implementation of the "Standard Rules on Equalization of Opportunities for Disabled Persons"[10], the International Labour Organization (ILO) survey on monitoring Convention Number 159 (on vocational rehabilitation and promotion of employment of persons with disabilities) and the United Nations Educational, Scientific and Cultural Organization (UNESCO) survey on special needs education, and official UN documents from a variety of sources. The sources for information are employed both to evaluate progress achieved and obstacles encountered in implementation of the World Programme in implementation, as well as to evaluate the options for improving the implementation of Programme monitoring.

The sources of data are of two general types. The first is country-level censuses and surveys, which provide demographic and socio-economic data about populations in countries. The second is administrative data collections specifically related to programmes, where there may not be information on persons without disabilities at all and individuals may not even be the unit of analysis. The purpose of the specific data collection cited is an important factor in analyzing the data for policy purposes.

Methods used to evaluate this information involve two approaches. The first compares the socio-economic situation of people with disabilities to that of the entire population to assess progress towards equalization of opportunities for all people. The second focuses on specialized indicators that pinpoint those elements of the disability experience where action needs to be taken. As will be demonstrated, the two approaches sometimes correspond to the two different types of data sources but they also correspond to two different ways of examining "Handicap". It is important for the reader to note both the data source and the approach to the data source.

Notes:

[1] United Nations General Assembly, World Programme of Action concerning Disabled Persons (A/37/351/Add.1 and Add.1/Corr.1 annex) [http://www.un.org/esa/socdev/diswpa00.htm].

[2] Ibid., para 1.

[3] Ibid., paras 193-201.

[4] United Nations General Assembly, "Implementation of the World Programme of Action concerning Disabled Persons; report of the Secretary-General (A/42/561)"; and "Implementation of the World Programme of Action concerning Disabled Persons; report of the Secretary-General (A/47/415 and Corr.1)".

[5] United Nations General Assembly, "Review and Appraisal of the World Programme of Action concerning Disabled Persons; report of the Secretary-General (A/52/351)" [http://www.un.org/esa/socdev/disrawp0.htm].

[6] WHO, International Classification of Impairments, Disabilities and Handicaps; a manual of classification relating to the consequences of disease (Geneva, 1980, reprinted 1993) [http://www.who.int.icidh].

[7] United Nations, Development of Statistics of Disabled Persons; case studies (United Nations Publication, Sales No. E.86.XVII.17).

[8] DISTAT version I, completed in 1988, contains disability statistics from national household surveys, population censuses, and population or civil registers from 55 countries, in United Nations, Disability Statistics Compendium (United Nations Publication, Sales No. E.90.XVII.17); see also United Nations, United Nations Disability Statistics Data Base, 1975-1986; technical manual (United Nations Publication, Sales No. E.88.XVII.12).

[9] "The United Nations and Disabled Persons; the first fifty years" at http://www.un.org/esa/socdev/dis50y00.htm (28 August 1999).

[10] United Nations General Assembly, "Final report of the Special Rapporteur of the Commission for Social Development on monitoring the implementation of the Standard Rules for the Equalization of Opportunities for Persons with Disabilities (A/52/56, annex)" [http://www.un.org/esa/socdev/dismsre0.htm].

Previous : Next