Developing and testing model 2000 census questionnaires

Laurie Lewis, Adviser on Population Statistics

Background

The post-ICPD shift away from priority support to national censuses to RH, brought with it the need for more innovative approaches to strategic planning and funding for the 2000 census round in the Pacific. During the 1990 round, the Caribbean islands had developed a cooperative approach toward the design and processing of censuses which we in the CST felt might have some application to the Pacific. Without much success, we tried to gather information on the achievements and failures of this exercise. Nevertheless, for the Pacific, we thought a start could be made through the convening of a regional meeting of heads of statistics to apprise them of the current UNFPA position and to develop a more rational pan-Pacific approach to census-taking, one that would help make better use of the scarce human and financial resources available to the island countries.

Some of the participants during the Meeting on Strategies for the 2000 Round of Population and Housing Censuses, Nadi, Fiji, December 1997

With financial assistance from UNFPA and collaboration with United Nations Statistics Division, the Secretariat for the Pacific Community (formerly South Pacific Commission), and the University of the South Pacific, a Meeting on Strategies for the 2000 Round of Population and Housing Censuses in the Pacific was held in Nadi, Fiji in December 1997.

The 17 national statisticians or their representatives attending this meeting drew up some very far-reaching recommendations on how to maintain the momentum the meeting had provided and pursue a more regional approach to census taking. For the CST the most important of these recommendations was that the UNFPA/CST should develop model population and housing questionnaires for the 2000 census round. In its consideration of a CST paper on population and housing census questions/topics, the meeting approved core topics for inclusion in the 2000 round, and it was felt these would provide the basis for the models.

Development and Testing of Model Forms

At the meeting, the representative of the Australian Bureau of Statistics (ABS) offered its services to develop draft forms. To support work on developing draft forms, conducting field tests in the Pacific, printing and distribution of model forms, and convening an Expert Group to consider further directions and activities for the regional approach, the UNFPA Sub-Regional Office in Suva provided some funding.

To date development is on target. Using ABS facilities, a draft was prepared in Canberra early in June. Four field tests have since been conducted by the CST Adviser on Population Statistics.

The Samoa Pre-Test

The first pre-test of the draft model census questionnaire was carried out during 11-15 August in Samoa, a Polynesian country. The intention was to test the draft questions, particularly their word order and the ease of translation into the local language. For the fieldwork, two areas were selected, one urban (in Apia), the other rural (in northwest Upolu). Twenty fieldworkers were instructed to take note of any problems encountered, either in the English or Samoan version of questions. In all, about 400 interviews were completed. Following the pretest, an open discussion was held to reflect on some of the difficulties experienced in the pretest.

These fell broadly into three groups: those exposing problems in the wording or order of the draft questions that would lead to amendments to the model questionnaires; those that were of a local character that would lead to a departure (for Samoa) from the model questionnaires; and those dealing with other aspects of design, including provision for write-in responses, codeability of responses and issues relating to the use of the questionnaires for direct key entry.

Second Pre-test in Vanuatu

Capture of responses, coding and appropriateness of the forms for data entry were tested in Vanuatu, a Melanesian country, in late September. It was also necessary to test some of the modifications made to the models as a result of the first pre-test in Samoa. Moreover, the mission provided a useful opportunity to test the questions in a more complex social setting, using the three lingua franca with translations into some of the local languages. In particular it was important to test the changes made to the economic activity questions. Following the Samoa test, a separate question on work in agriculture and fishing was added, designed to lead into a more detailed classification of agricultural occupations and to capture in the labour force women (and sometimes men) who shared their time between home duties and unpaid agricultural work.

Pre-census testing in the PNG highlands

As a result of these tests, and some informal work in PNG, a large number of modifications were made to the forms at the ABS, where the model files are housed on Pagemaker format.

Final Tests in Micronesian Countries

The interviews proceeded smoothly with few problems related to the questionnaire itself. However, a few changes were proposed in the presentation of pre-coded questions and some of these were incorporated into the next version. One such example is the question on date of construction of the housing unit. This has been difficult in several of the countries in which tests have been conducted, and space permitting, pre-coded response categories such as less than 1year, 1-2 years, 3-4 years, 5-9 years and more than 10 years, will be substituted. Additionally, several comments in the discussions that followed related to the inappropriateness of some of the response categories for Kiribati, e.g. for religion. It is recognized that the structure of some variables, including administrative geographic units and religion, will vary from country to country. The inclusion of response categories for such questions is intended to provide only a notional set to facilitate the construction of the model forms.

From the work done in Kiribati, it appeared that the booklet style of forms was well-suited to data reporting, recording, coding, and control. The approach does present some difficulties for direct data entry, but these can be mitigated through the use of special code sheets. Other options include alternative data entry software and scanning, but these were not considered during this round of pretests.

In any case, any model system developed should contain sufficient flexibility to enable countries to easily adapt model versions to suit their own needs. To illustrate this point, an individualised version of the model forms is currently being developed and it should be possible in time to expand the option still further.

The very small staff at the Office of Planning and Statistics (OPS) in the Republic of the Marshall Islands (RMI) limited the amount of work possible on the models. Fortunately, the three earlier pretests had gone far in resolving the major form development issues and my concerns in RMI were not crucial. In the Kiribati test in the use of code sheets, it was felt that a further streamlining of processing could be achieved if the responses were coded directly to code sheets, eliminating the need to enter codes onto the actual questionnaires. The adviser thought it would be useful to test this proposal in RMI. To run a small test the OPS staff completed a number of interviews. The completed forms were coded, some directly to code sheets, others onto the questionnaires. Direct coding to code sheets did seem to reduce overall time, though no conclusion can be made from the small trial; the writer believes that the risk of introducing error cautions against the adoption of this practice. In any case, the results confirmed the Kiribati findings that the use of code sheets facilitated data entry.

Model Census Questionnaire

More importantly, the visit to the RMI provided a useful opportunity to compare core questions on the models with standardized census forms developed by the US Bureau (USB) of the Census for use in the former US associated States and the so-called insular areas. A number of these entities, the Commonwealth of Northern Mariana Islands, the Federated States of Micronesia, American Samoa and RMI, had been represented at the Nadi census strategy meeting and had supported the idea of model questionnaires for the entire Pacific region. It would be useful to see the extent to which the core questions included on the two sets of forms could be reconciled.

The broad comparison is shown below. The good news is that most of the questions included on the Pacific region models are also included on the USB models. However, the special emphases given to more precise measurement of fertility and mortality, and subsistence/market orientation and women’s unpaid work in agriculture are not properly reflected in the USB model. More generally, the USB model forms appear far too lengthy and too complex for the Pacific environment. This comment applies especially to RMI, given the shallow resource base and the shortage of time to plan and prepare for the census, and to recruit and train fieldworkers. In both 1981 and 1988, the censuses were clearly deficient in a number of respects related to both coverage and content. There is little doubt that if the forms were overloaded in detail, under-coverage would occur and the quality of the key statistics would suffer. Moreover, the shift from a de jure census in 1981 to de facto in 1988 has affected comparability. The Pacific model is designed on the assumption that countries will be adopting a de facto approach to their 2000 census round, although this assumption is in no way intended to suggest that this approach is well-suited to all countries. The USB model, in contrast, is based on usual residence and it is worth noting that if adopted, the change would introduce further problems of comparability with the last census.

The adviser’s recommendation is to use the Pacific models for the core items, since this would give RMI most of what a census should provide and, at the same time, promote regional standardization. Additional questions could be added, but this should be done sparingly and only after meetings of the census technical and user committees, both properly briefed on the costs and dangers of being over-ambitious.

An Expert Group meeting of selected Pacific island countries, regional organisations, and multi- and bilateral donors is proposed to follow the final testing programme, allowing some time to consider comments on the drafts, to allow ABS to make further amendments, and to print and distribute quality copies of the model questionnaires.

Already, a number of achievements have been observed. Close cooperation among the island nations of the Pacific, despite large variations in size, economic development and census history is clearly the most important. The acceptance almost overnight of statistical standards consistent with the recommendations of the UNSD and Regional Commission will also go far to contribute to strengthening of Pacific statistical services. A good illustration is the impetus this work gives to the discussion of the creation of a common database for use by all UN agencies under the UN Resident Coordinator System. Other recommendations of the December 1997 meeting, for example the call for common software for census data processing and the development of a common tabulation plan, will all start from the model questionnaires.

Many questions remain to be answered on the future of the regional initiative. One of these being posed by countries participating in the testing programme, for example, relates to the facility that the regional programme will provide to map from the model to national questionnaires, to take account of local variations and provide options. The proposed Expert Group could address these questions and establish more permanent institutional arrangements for development and technical assistance. It will also be necessary to consider the addition on non-core questions or topics in a way that would not contravene the principle of regional standards. This will necessarily involve considerations of related technical issues such as the creation of a common data directory, standards for coding, editing, data entry, tabulations and analysis. It will also require a huge input for documentation of procedures and preparation of manuals.

There is a long way to go in ensuring a regional strategic approach to the 2000 censuses and beyond. But a great start has been made.