QUALITY OF CARE:
Is the Client Centred Focus Sufficient for Programme Quality?

Table of Contents

Preface

I. Introduction

Bibliography

The UNFPA Country Support Team for the south Pacific, based in Suva, Fiji, is one of eight regional technical support teams established by the United Nations Population Fund to provide countries with technical assistance and information backstopping to meet country needs in the population field. In fulfilling this function, apart from field missions, the Country Support Teams also try to foster active communication and open discussion with national experts to promote a more holistic approach to population programmes.

This Discussion Papers series has been initiated by the CST (Suva) in an attempt to establish a dialogue among national population programme personnel on the integrated and co-ordinated multidisciplinary approach to population. Hence, CST Discussion Papers are not particularly addressed to academic audiences but to practitioners.

In this paper, Dr. Praema Raghavan-Gilbert, Adviser on Reproductive Health and Gender, addresses quality of care and management issues in reproductive health programmes. She suggests that the client-centred quality of care models currently in use are too narrow in scope to identify management causes of poor quality of services and advances a programme quality assessment model to focus on organizational issues. The context of the paper is limited to family planning simply because the original field work was done before the Cairo Conference and under the sponsorship of an international family planning organization. However, the theoretical logic and analytical framework of the Programme Quality Assessment Tool are compatible with the whole domain of reproductive health programmes. It will be a useful and appropriate tool for managing integrated primary health care in the post-ICPD era.

18 June 1999
Stephen Chee

Director

Executive Summary

Over the last decade, two important events triggered a flurry of interest in improving quality of care in reproductive health care programmes. The first was the publication of a conceptual framework for describing quality of care by Judith Bruce (1989). The second came in 1990, when the USAID-led subcommittee on Quality Indicators in Family Planning Service Delivery adopted and modified the Bruce framework, provided illustrative indicators for the six elements of the framework through its Evaluation Project (1992), and encouraged family planning agencies to experiment with needs-specific approaches to assess program quality. Much of the current research in assessment of family planning quality of care evolved out of these two critical incidents.

While the Quality of Care (QOC) models advocate allowing the client to speak for herself and to assess client-defined needs, the steep status and power asymmetries between provider and client in developing countries represent a troubling barrier to this. The inevitable ‘courtesy bias’ renders ensuing client ‘evaluation’ of the service experience mostly invalid. Client empowerment will increase gradually, with rising female literacy, higher lifestyle expectations and increasing awareness of the right to be treated with dignity and care, especially in public sector programs in the interim. The service delivery organisation must protect the client from insult and injury, both technical and emotional. To ensure this, the organisation must strengthen all critical inputs and processes that affect the service delivery production and the service experience in family planning programmes. These critical inputs and processes are the indicators and criteria of quality that stakeholders should examine to ensure that organisational standards of service delivery are met.

This paper shifts the focus from the QOC paradigm, grounded in the Bruce model of the 1980s, to the post-ICPD paradigm. Among its other affirmations the ICPD programme of action (ICPD POA) declares that programmes should:

1. Recognise that appropriate family planning methods for couples and individuals vary according to their age, parity, family size preference and other factors, and ensure that women and men have information and access to the widest possible range of safe and effective methods to enable them to exercise free and informed choice;

2. Provide accessible, complete and accurate information about various family-planning methods, including their health risks and benefits, possible side effects and their effectiveness in the prevention of the spread of HIV/AIDS and other sexually transmitted diseases;

3. Make services safer, affordable, more convenient and accessible for clients and ensure, through strengthened logistical systems, a sufficient and continuous supply of essential high-quality contraceptives. Privacy and confidentiality should be ensured;

4. Expand and upgrade formal and informal training in sexual and reproductive healthcare and family planning for all health-care providers, health educators and managers, including training in interpersonal communications and counselling;

5. Ensure appropriate follow-up care, including treatment for side effects of contraceptive use;

6. Ensure availability of reproductive health services on site or through a strong referral mechanism;

7. In addition to quantitative measures of performance, give more emphasis to qualitative ones that take into account the perspectives of current and potential users of services, through means including effective management information system and survey techniques for the timely evaluation of services;

8. Emphasise breast-feeding education and support services, which can simultaneously contribute to birth spacing, better maternal, and child health and higher child survival.

Many of the goals outlined above depend on infrastructural systems and organisational arrangements that are not visible to clients. An organisation strengthened through the application of an appropriate service quality management model enables free and informed choice, through complete and accurate information in IEC (information, education and communication) and client-sensitive counselling services. The model should examine service safety and accessibility and should seek to improve service reliability and provider competency.

The Programme Quality Assessment (PQA) model introduced in this paper allows the organisation to do this. It enables the organisation to assess its resources, management processes, support and accessibility systems against normative standards, which in turn enable continuous service quality improvement in the post-ICPD era. The PQA model can address concerns raised in quality of care standards in reproductive health and family planning programmes by identifying weaknesses in programme structure, inputs and processes that currently may hinder men and women from acquiring adequate information and access to safe and effective services. This comprehensive programme quality assessment (PQA) model and its implementing tool (PQAT) examines managerial and interpersonal aspects of care in FP. In keeping with the new paradigm shift the PQA model enables the organisation to deliver free and informed choice, complete and accurate information, safer and more accessible services, strengthened logistical systems, privacy and confidentiality, formal and informal training, appropriate follow-up, strong referral mechanisms, effective management information system, and timely service evaluation.

The domain in this paper is limited to FP because the work was done before the Cairo conference for an international family planning funding agency. However, the underlying logic and structure of the PQAT are compatible with the demands of reproductive health care services and it can be adapted to this domain. It will be a useful and appropriate tool for managing integrated primary health care in the post-ICPD era. Three case studies from the Pacific conducted in the early nineties are presented to support its claim as a reliable and robust tool for managers.

Interest has been growing in recent years to improve the quality of healthcare services particularly in public sector programmes. While in developed countries this in part reflects the rise in awareness of consumer rights and provider accountability, in developing countries it also reflects an awareness that people deserve more than nominal services with minimum standards of technical care. In certain areas of healthcare such as family planning (FP) services notably characterized by the elective nature of usage by the ‘well’ patient, it is increasingly recognized that improvements in quality are likely to enhance utilization and the success of programmes (Jain 1989). Conversely, many researchers have questioned whether quality is the missing link in reducing the unmet needs of populations for FP and reproductive health (RH) services (Forsberg et al 1992, Kumar et al 1989).

Despite these concerns, stakeholders in many countries view quality of care research with suspicion. As with other approaches to assessment, quality evaluation can be perceived as "unwelcome external criticism" (Hull 1996), further compounding the difficulty in gathering information about programmes perceived to be performing poorly. This is to some degree a real issue because the reward system in FP programmes, even with the logical framework (Logframe) approach continues to be performance based on quantitative indicators (objectively verifiable indicators, or OVIs). An organizational commitment and widespread adoption of the ICPD POA on voluntarism and choice both as a right and a gender entitlement would largely defuse this sovereignty issue of external evaluation standards being imposed on national organisations.

The quality movement in healthcare evolved from the consumer industry, which was interested primarily in increasing profit by reducing wastage through rejection and rework of products that did not meet customer needs and specifications. The service industry expanded on these concepts by studying the difficulties created by the special characteristics of the service experience. The attention to the needs of the internal customer i.e. the people within the organisation producing the service was central to improving quality in the service industry. During the 1980s, healthcare, also a service industry responded to these findings and supported many interventions seeking to understand the user perspective in reproductive health and family planning. The client-centred framework for examining quality of care in FP services, promoted by Bruce of the Popuation Council in 1989, was based on earlier work by Donabedian (1988). The narrow focus of the client-centred approach of the quality of care (QOC) models is both its weakness and its strength. A quality experience cannot result if systems within the organization are weak. Empathetic services alone cannot compensate for weaknesses in supplies, skills and knowledge in the enabling systems. For sustainable quality, a comprehensive and structured way for management to examine the full phenomena of service quality is important. Clear organisational goals in quality must be flagged for the mid-level managers in their day to day work to attain programme quality.

Even though the consumption of FP services is not well understood by social scientists because contracepting behaviour is socio-culturally and personally complex, they continue to seek explanations for low utilization of services. Programme managers however are interested in tools and skills to measure, monitor and improve the quality of services for which they are responsible. The expectation is that this would increase service utilization. Often, elaborate but impractical monitoring and evaluation tools are centrally provided which gather either incomplete information or information that mid-level mangers cannot use easily and promptly to rectify problems.

Managers and supervisors must have tools to help them identify programme strengths and inadequacies. These tools must assist them to understand the cause of the problem i.e. the "why" of an inadequacy? If the "why" cannot be identified, then the problem cannot be fixed in a way that would prevent a recurrence. Periodic stockouts of supplies due to poor inventory control systems are one such example in many programmes. While acquiring commodity loans can bridge the gap, only logistics training and improved logistics management is likely to rectify the problem.

This supports Walter Shewart’s thinking on quality, who argued in 1931 that efforts in a company should be directed at finding and fixing problems in the processes of work. Crosby (1986), too, makes a very strong case for Doing It Right the First Time (DRIFT), as do Quality gurus Deming (1986) and Juran (1989) for reducing variance in products and processes. Their rightful contention is that early attainment of quality in a process or programme is less costly and wasteful of scare resources.

While good interpersonal and counselling skills promoted in the QOC model may ameliorate some of the negative service experience, the client-centred models are unable to move upstream from these to seek out the root cause of weaknesses that contribute to poor quality of care. The organisation alone has this information to fix these problem at its origin through policy decisions and resource allocation.

In healthcare, underutilized services are an inefficient drain on scarce resources and have serious opportunity and negative image costs, which may be difficult to correct later in the community.

The direct relationship between high fertility and high maternal and child deaths in developing countries is well documented (Besley 1989). Although population growth rates and fertility rates have fallen in all countries the quest for improving access and utilization of reproductive health and family planning continues unabated. Recent FHS & DHS surveys worldwide have suggested that many FP programmes are weak in information, education and communication (IEC) and accessibility. However, anecdotal evidence also suggests that women either abandon contraceptive methods because of limited choice, weak counselling or dehumanized services. Although stakeholders value this information, demographic surveys are unable to inform stakeholders about clients' perceptions about poor quality of care.

Managers experience additional difficulties because researchers and policy makers have yet to reach a consensus about the goal of service quality. The debate among researchers continues about whether the correct output measure of "quality" is service utilization or contraceptive practice or whether the end point is "the reduction or elimination of unwanted pregnancy and healthy, safe and wanted children".

Figure 1: Conceptual Framework of Family Planning Programme Impact on Fertility in the Context of Supply and Demand

Figure 1 portrays the complex links between quality and fertility, and traces fertility outcomes back through contraceptive practice and service utilization. The model portrays quality as ONE of several service outputs that influence both the supply and demand of FP services. As the figure reveals, FP supply factors themselves operate in a system that is influenced by external forces and endogenous factors such as organisational structure and operations management. Many variables influence service quality and this clearly illustrates the logical flaws in using contraceptive practice, service utilization or fertility outcomes as surrogate measures for programme quality. Improving quality of care is an end in itself from a rights perspective. Because understanding of FP programme quality management is emergent, only experience will confirm the positive relationship between high service quality and utilization. It is however encouraging that researchers have identified direct relationships between service quality and utilization in the health sector (Morehead 1974, Berwick, et al 1990) and between service quality and utilization in the service industry (Zeithaml, et al 1990).

Key stakeholders in all RH/FP programmes currently seek to improve quality of care. They realize that management information on quality indicators is required for client attraction and client retention as well as for making good decisions about resource allocation. While the political support is there, the long term commitment to gather and use this information to improve quality of care is less obvious. This is less forthcoming because the process of improving quality of care is a slow and incremental one. This process has been estimated by quality experts to take between three to six years to become embedded in an organisation’s psyche (Juran, 1989, Deming 1986, Crosby 1986). While some of the wavering of commitment is due to resource constraints, it is also partly because different stakeholders also hold special perspectives on quality.

The key features of the delivery systems and the characteristics of the organisational and community actors must first be examined to understand differences in perspectives. The key systemic features of family planning (and primary healthcare) are, first, high interdependence among the specialized functions within a complex service delivery system which comprise of physical and financial access, support services, interpersonal, technical and management skills; and secondly, large information gaps among those diverse entities with a stake in service quality – namely the client (the external customers), the providers and managers (the internal customers), the community, the health and funding agencies, policymakers and special interest groups such as feminists or anti-FP groups.

Donor interest in quality, historically, was affected (often negatively) by concerns related to impact, efficiency and equity. Even today, programme evaluation indicators continue to lean towards quantitative measures specified in the OVIs of the Logframe approach (though target free). Donors today are concerned that lower quality will lead to underutilization of services and greater inefficiency. However, they are slow to incorporate measures of quality either because of inexperience and unfamiliarity with the domain or because of doubts about the research validity of quality of care indicators chosen. This confusion is aggravated by a weak understanding about two conflicting paradigms in evaluation, i.e. the "utilization-focussed" programme evaluation paradigm (Patton 1986) which seeks management information to improve programme effectiveness, and the "verification of truth" paradigm held dear by evaluation researchers. Donors are now convinced that wasteful care is either directly harmful to health or harmfully displaces more beneficial care (Donabedian 1985; Calla 1991).

SDP Managers: Non clinician managers of service delivery points (SDP) tend to feel that access, effectiveness in technical competence and efficiency are the most important dimensions of quality (Brown et al 1993). Less importance is assigned to the inter-personal dimensions of service. Clinician managers of clinics tend to be reluctant to support wider quality efforts for they view quality as within the technical domain of the physician (Berwick et al ’90).

Providers tend to focus on technical competence, effectiveness and safety. They need and expect (as internal customers) effective and efficient technical, administrative and supportive services to provide high quality service. They tend to underestimate the importance of their role in the interpersonal experience of the client and the effect this experience has on the client’s perception of service quality (Brown et al 1993). Physicians may be a major barrier to improving service quality because of their reluctance to support quality efforts that include the clinical areas (Berwick et a l 1990).

Clients (patients and communities) often focus on accessibility, interpersonal process, continuity of provider, effectiveness and amenities as the most important dimensions of quality (Zeithaml et al 1990, Brown et al 1993; Bruce ’89). They cannot adequately assess technical competence as they have neither the technical expertise nor the organisational information about inherent weaknesses in the technical support services. They assume a moral dimension that technical quality is adequate until a critical incident occurs.

Organisational Managers are responsible for providing support services to meet internal customer needs and may not fully understand the workings of the front line field staff and how their own actions (or lack of action) may impact on the client’s perception of quality. Clinic and service staff may not fully comprehened the intricacies and constraints of the central level operations.

Policy makers tend to focus on macro-level objectives and are shielded from the effects of their collective decisions on individuals. Policy makers are politically vulnerable and risk averse and are usually convinced by efficiency and risk analyses. Therefore, they often prefer safer measures of organisational efficiency (making the numbers) as proof of accountability to higher levels. This attitude will change only when the reward system within the healthcare sector evolves to value high quality of care as an end in itself.

There is no single definition of a multifaceted and complex attribute called "quality". The characteristics of this attribute are a kaleidoscope of properties depending from whose perspective it is viewed, the complexities of which have been already been explored.

The definition advanced by the International Organisation for Standardization (ISO), in ISO-8402(1986): Quality Vocabulary, has three key embedded concepts that encapsulates its complexity and multi-dimensionality. Quality is defined as the totality of features and characteristics of a product or service that bears on its ability to satisfy stated or implied needs. Definitions that range from customer satisfaction, value for money, fitness for use, collective attitude of mind, systematic approach to excellence, constancy of purpose and guarantee of confidence are some definitions advanced in the Total Quality Management (TQM) literature. The industrial TQM model emphasizes the organisation’s ability to meet the specifications of the physical product, i.e. "conformance to specification". By comparison, the service TQM model, including healthcare, is concerned with the organisation’s ability to meet the clients’ expectations, which include dimensions such as, reliability, responsiveness and competence.

Unfortunately production of service delivery differs from production in manufacturing on three characteristics of service that must be fully understood: intangibility, co-production and simultaneous production, and consumption of the product (the service). Service is an intangible experience produced jointly by the interaction of the provider and the client. Since it does not share the "hard" characteristics (specifications) of a physical product, it cannot be made, inspected and stored ahead of time (like a refrigerator or computer). In FP a physical product such as an intra-uterine device (IUD), with its product specifications, may be involved in a service transaction, but the personal service of IUD insertion demands the simultaneous presence of both the client and the provider.

Research in service quality has confirmed that it is the client’s perception of whether an experience of acceptable quality occurred at the time of service production that decides the continuation of the relationship between the client and the provider. This acceptability is particularly awkward to measure as the client carries a mental model of expectations that is often influenced by the "chemistry" between people, some of which is communicated verbally or non-verbally between both parties. Positive perceptions of quality of care are particularly important in the promotive and preventive services where the services are elective and the patient is not seeking relief from pain or distress. The simultaneity of service production and consumption leaves little time for corrective action on deficiencies revealed to the client during the interaction. Therefore, all deficiencies in the enabling systems (the internal customer support systems comprising of the field managers and support services team) must be corrected before the provider interacts with the client, the (external customer). This is shown figure 2.

Extensive research undertaken by the service industry to understand the service quality dimensions important to the external customer has revealed ten dimensions of quality in the TQM service model. They are given below in the order of importance, with the greatest value being placed on reliability of service and the lowest value being placed on physical facilities.

1. Reliability: consistency of performance and dependability

2. Responsiveness: the willingness or readiness of the employees to provide service and to the timeliness of service

3. Competence: possession of required knowledge and skills

4. Access: approachability and ease of contact

5. Courtesy: politeness, respect, consideration, and friendliness of contact with people

6. Communication: adjusting language for different consumers and listening to customers

7. Credibility: trustworthiness, believability, and honesty as well as having the customer’s best interest at heart

8. Security: freedom from danger, risk, or doubt

9. Understanding the customer: the effort to understand the customer’s needs

10. . Tangibles: the physical evidence of the service.

 

Much work has emanated within the last decade from the Bruce (1989) Quality of Care (QOC) framework. This framework remains a trestle for conceptualizing client –centred QOC models. The framework identified six elements of quality as salient for improving quality of care. They were modified by the USAID Sub-Committee on Quality Indicators (1990) and are given below:

1. Choice of Methods
2. Information given to clients, and counselling
3. Technical Competence
4. Interpersonal relations
5. Mechanisms to encourage continuity and follow up
6. Appropriateness and Acceptability

The QOC framework has created a heightened awareness about the important but often neglected interpersonal aspects of the care-giving process. However, it pays little attention to the enabling systems within the organisation that are responsible for the production of a high quality service experience for the client. The client centred QOC framework proposed by Bruce is a valuable and timely work, but has weaknesses that are being studied by various researchers.

A conceptual weakness exists because of the underlying assumption that clients in developing countries wield both considerable economic and social power, and therefore improvements in quality of service based on a client-centered approach will result in greater client retention in programmes. This claim is being tested by family planning programmes, using many quality assessment models (Katz et al 1993). However, experience is insufficient to substantiate this claim with any degree of confidence. Currently, only limited empirical evidence about access, choice and competence support this argument in family planning research (JHPIEGO, 1991; Jain, 1992). However increased client satisfaction is known to build ‘loyalty’ and therefore continuing use of service is a plausible extrapolation derived from lessons in other service industries (Parasuraman, 1987; Berry et al 1990; Zeithaml et al 1990).

The knowledge asymmetries between client and provider are too marked in developing countries for the client to assess the competence of clinical processes. Here, the wide social distance between the client and the medical establishment also limits any kind of useful feedback about deficiencies in the interpersonal process. Unavailability or poor quality of "continuing use" data too poses major methodological problems for all family planning programmes in developing countries.

Additionally, in most developing countries, where alternate sources of service are either scarce, absent or financially inaccessible, the market forces so necessary to allow free expression of empowerment are not yet in play. Major health sector reforms may soon test this "market-based" hypothesis.

A practical weakness of the QOC model is that Bruce does not indicate how to operationalize the assessment of the elements in the QOC framework. This is addressed partially by the work of the USAID Subcommittee on Quality Indicators in family planning service delivery (USAID Taskforce Report, 1990), the Situational Analysis Study (Fisher et al 1983) and the Evaluation Project (Bertrand et al 1993). Operationalisation of the six elements of the QOC model was unwieldy because of the 42 evaluation indicators later suggested in the Evaluation Project Report. The ability to operationalize the concepts in the QOC framework and its key indicators is critical in assisting field and programme managers to improve programme quality in the day to day management of clinics.

The absence of management and infrastructure factors in the framework is a major weakness of the QOC model, when applied to quality improvement in developing countries. Any quality management model must examine management inputs and processes. These are central to total quality management theory (Shewart, 1931; Crosby, 1979; Deming, 1986; Juran, 1989). In developing countries, unlike in many industrialized (and often litigation-prone) societies, there is often no legal requirement to fulfill strictly enforced internal or even regulatory organisational standards for licensing before commencing family planning clinic operation. Clinics in these settings generally function with widely varying degrees of operational readiness and controls, depending largely on their programme heritage.

An explanation of programme heritage is perhaps useful here. This construct may be unique to developing country family planning programmes and has a special bearing on the pressing need for standards. Family planning programmes in many developing countries are often started and supported with external funds, partly because of local political and cultural sensitivity to family planning intervention. Also, national health resources are usually very limited. Typically, over the life of a family planning project, different external funding organisations with varying missions and quality norms will have provided technical assistance in critical areas such as training and IEC activities. In this context, there is an obvious and critical need to specify organisational standards and the criteria of quality indicators.

The binary (yes/no) scoring template is found in many QOC models. While helping to identify that a problem exists, this technique does not provide sufficient quantitative nor qualitative data for informed decisions. It says "yes," a problem exists in this dimension of quality or "no" a problem does not exist in that area. It is unable to measure the degree (by how much or the ‘gap’) to which a defect exists relative to a standard, and cannot seek the root cause (the ‘why’) of the deficiency. Understanding the ‘gap’ and the ‘why’ will provide valuable information for management decisions in the TQM model.

Nearly all family planning programmes assessed for service quality over the last decade by the writer had weak management and supervisory systems, mainly because they were not guided by formal organisational standards. As a result, field managers continue to experience great difficulty in monitoring, in a consistent and systematic way, those organisational processes that are critical for quality in service production and delivery. Seldom were both management and technical systems formalized, either through appropriate written standards of practice (SOP), or through training and retraining as part of a feedback loop to close the performance gap. Often, central-level management support was limited to vague commitments to "undertake all necessary corrective actions" because of difficulty in prioritization of resource allocation. This remains a major weakness in sustainable quality improvement efforts in family planning programmes throughout the developing world.

An alternate QOC model was deemed necessary to overcome these inherent service quality management problems in developing country programmes. The author, cognizant of the deficiencies of the Bruce QOC model, expanded the QOC framework by including organisational as well as interpersonal processes in designing the Programme Quality Assessment model and developed its implementation tool, the Programme Quality Assessment Tool (PQAT) Raghavan-Gilbert 1991. The PQA model, as operationalised through its tool, helps the organisation set and formalise relevant standards in client-centred and organisational procedures.

The PQA model which includes management inputs and processes as well as the client-provider interaction are presented below in a comprehensive table. It seeks to explain why the indicators of programme quality and their key criteria were chosen and to show how this information can be used to improve quality.

Figure 3: Programme Quality Indicators showing justification for selection (Raghavan-Gilbert 1997)

Unlike the Bruce Model, at the implementation stage, it is a tool that assists managers in the identification of programme inputs and processes that fall short of organisational standards of quality in critical activities. This identification guide’s continuous corrective action to close the performance gaps, and assists local managers to meet organisational standards. The tool facilitates identification of quality deficiencies, provides answers also to the ‘why’ and ‘by how much’ questions about them, and helps to narrow the performance gap between intention (service specification) and actuality (service performance).

However, there is considerable congruence with the eight critical dimensions of quality in the Quality Assurance (QA) model, which like the PQAT is a TQM model (Brown, et al 1993). The dimensions of Quality in the QA model are given below.

1. Technical competence (clinical, non-clinical and management skills) of providers, managers and support staff;

2. Access to service (geographic, economic, socio-cultural, linguistic and organisational factors such as hours, appointment systems etc.);

3. Effectiveness (service delivery norms, clinical guidelines, potential harm vs. net benefits);

4. Interpersonal relations (provider-client, manager-provider, health team- community) because inadequate interpersonal relations can reduce effectiveness of technically competent services;

5. Efficiency (product and service affordability, optimal vs. maximal care, maximal resource utilization) because wasteful services increase costs;

6. Continuity (uninterrupted service, access to timely referral, accurate record keeping) because absence of continuity compromise effectiveness, decrease efficiency and reduce interpersonal relations;

7. Safety (infection, harmful side effects and jury to client or health risk to provider) and

8. Amenities (physical appearance of facilities, personnel and materials, comfort, cleanliness and privacy).

The comprehensive PQA model has been used successfully by the author in six countries and nine programmes in the early nineties.

The PQAT has been used in Africa, the Middle East, and Asia. The summaries below portray the results of assessments of family planning services in three island countries in the South Pacific. All share certain similarities: former colonies, they are remote from the industrialized world. However, levels of social development are very different. In each nation, the sites represent a mix of urban and rural clinic locations, and both government-run and independent clinic operation (Raghavan-Gilbert, 1998).

A discussion of the use of the PQAT in three countries in the Pacific follows.

Figure 4: Findings from Country A

Country A had achieved the lowest level of economic and social progress since independence of the three cases. Figure 4 reflects inadequate headquarters support to the field: note the pervasive pattern of inadequacy in facilities, access, standards, MIS, supervision, monitoring, follow up, safety and competence. The single urban clinic (Clinic 1) assessed as adequate in a majority of the indicators was run by an NGO and located in the capital. Clinic 5 located 50 kilometres from the nearest referral hospital had no trained staff. Despite generally high unmet need for family planning services in the area, this clinic had the lowest utilisation of the six sampled.

A programme-wide pattern of inadequacies related to technical competence and safety was demonstrated by poor case management, weak technical knowledge and low skills caused by a lack of formal training. Other inadequacies were the unreliable supply of commodities and consumables; poorly organized and maintained physical facilities; poor access to IUD services and information; and weak management and support functions related to the provision of SOP and protocols, IEC, MIS, supervision, monitoring and client follow-up. This pattern reflects major weaknesses in the central leadership and management of the family planning programme and signals the need to address programme quality at the organisational level.

Weaknesses related to SOP and protocols, MIS, monitoring and follow-up were endemic in all six clinics. As measured by the PQAT, only Clinic 1 (an IPPF affiliate) produced high quality service. Clinic 2 tested better than the remaining four, and was one of two served by family planning-trained providers (the other served Clinic 4).

Figure 5: Findings from Country B

Country B was somewhat more developed, due partly to its role as a popular tourist destination. Nevertheless, similar patterns emerge:

Clinic 1, located in the main city, was operated by an NGO. Clinic 2, in the central government hospital in the same city, had good utilisation figures despite its poor showing in the assessment mainly because of its affiliation to and location within the central hospital. Clinics 3 and 5 (also run by the government) suffered from a lack of trained staff, while utilisation was low.

System-wide problems existed in systems that provided support to the internal customer (field managers and providers) for commodities, consumables and equipment. Throughout the national family planning programme, all centrally provided support functions related to the provision of SOPs and protocols, IEC, MIS, supervision, monitoring and client systems were poor.

Clinics 1 (an FPA/ IPPF affiliate) and 4 (an integrated rural PHC clinic with trained providers) showed higher PQAT service quality measures. While both clinics were served by family planning trained staff, the urban NGO/FPA clinic revealed much higher attendance rates than other public sector clinics assessed. While urban Clinic 2 offered poor quality services, its attachment to the base hospital in the nation’s capital led to high attendance figures. Rural Clinics 3 and 5 offered inadequate quality of service except for medical backup, physical facility and access. Every Department of Health (DOH) clinic had serious service quality problems caused by weak central management support and systems.

Figure 6: Findings from Country C

Country C was larger and had more highly developed infrastructure than the other two, which is reflected in much higher performance on access and medical backup, far stronger performance in capital-intensive areas such as logistics and equipment, and generally better performance on skills-based areas such as competence, safety, supervision, and MIS.

Technical competence was inadequate in all the clinics except the IPPF-affiliated clinic. This was particularly so in relation to basic family planning knowledge, which is important for case management, in IUD provision, and counseling. This effectively limited the range of methods offered to the client. Major weaknesses were also identified in management support functions related to the provision of SOPs and protocols, IEC outreach planning, implementation and IEC materials and to MIS inadequacies, caused by the lack of on-the-job training by supervisors. In Country C, choice, infection control, medical backup, commodities, consumables and equipment were all adequate. Formal structured family planning supervision at the program and clinic level was weak, but stronger than in Country A or Country B, because of good Ministry of Health (MOH) infrastructure down to the village level. On-the-job training by supervisors is lacking, largely because supervisors, who had received no family planning training, lacked confidence. Client follow-up systems were weak in most clinics, as supervisors were unaware at that time of the importance of follow-up to continuation of contracepting behaviour.

All urban and rural clinics tested well on those quality indicators that were independent of central level support. Clinic 4, the subdivision health centre in the nation’s capital, offered the highest quality of services as per the PQAT. It also showed the highest utilization figures. Clinic 3 and Clinic 5, the only two clinics served by family planning trained nurses, had utilization figures that were nearly as high.

Figure 7: Comparative Findings of 3 Pacific countries

Figure 7 presents a comparison of the 3 countries served by the regional organisation, South Pacific Alliance for Family Health (SPAFH), as key stakeholder and client in this assessment.

Although the required action varied by country, common problem areas emerge, such as technical competence, standards, management information systems, and outreach. At a central level, the regional organisation had problems in providing management support to improve service quality in the following areas: 1) Training in basic family planning knowledge and skills was weak in all three island nations and affected technical competence, and 2) SOPs and protocols were not available in any of the countries. The regional organisation had yet to develop and provide Standards of Practice (SOP) and Protocols to its member countries. IEC outreach and MIS were weak in member countries with a need for central support and directions to improve the situation to meet quality standards. Client follow-up systems were currently inadequate in member countries because of lack of central direction and support.

The client-centred quality of care models currently in use are too narrow in scope to identify management causes of poor quality services. The Programme Quality Assessment model is a viable example of a tool that extends beyond the client-centred focus of QOC models to focus on organisational issues. This comprehensive tool is able to identify a number of inadequacies in the enabling systems of the organisation both centrally and at the periphery, which would have been missed by the client-centred QOC approaches.

The paper shifts the focus from the QOC paradigm, grounded in the Bruce model of the 1980s, to the post-ICPD paradigm, with its emphasis on quality of care standards in reproductive health and family planning programmes in particular, and on gender and population development programmes in general. The PQA model enables free and informed choice, complete and accurate information, safer and more accessible services, strengthened logistical systems, privacy and confidentiality, formal and informal training, appropriate follow-up, strong referral mechanisms, effective management information systems, and timely service evaluation.

The conceptual framework and the logic of the Programme Quality Assessment model and tool definitely has the potential for application to service quality assessments in reproductive health care services (Raghavan-Gilbert et al 1998). It has the potential, if used correctly, to make a contribution to improving quality of reproductive health care services beyond family planning. The indicators of quality in the PQAT while generally consistent with all primary health care services, will need to be fine-tuned to capture additional dimensions of quality that are of critical importance to the stakeholders in RH programmes. It is important to remember that RH care unlike family planning care-seeking behaviour, is not purely elective. The importance of examining the organisational and client-centred inputs and processes that enable the production of a high quality reproductive health services are as important in RH as in FP.

The general principles of quality measurement and quality management are endorsed by most programme managers. Yet these are often not well understood in the field, at the operational, or even in the headquarters levels of many programmes in less industrialized countries. This gap is due to a lack of exposure to quality theory, philosophy, and techniques compounded by a dearth of practical management tools to support timely operational decisions. These symptoms can be addressed simultaneously by providing a system of quality management training combined with broad and comprehensive quality tools built on strong theory.

Belsey, M (1989). Priority issues in maternal and child health for the 1990s. DHS World Conference Proceedings, 3: 2173-2178

Berry L. L.; Parasuraman, A (1990). The service-quality puzzle, In: Clark, G. (ed.) Managing Service Quality: An IFS Executive Briefing, IFS Publications, UK

Bertrand, J. T, Magneni, R. J, Knowles, J. C. (eds.). (1993) Handbook of Indicators for Family Planning Program Evaluation, The Evaluation Project, Tulane University

Bertrand, J (1994, Jan). "The evaluation Project promotes consensus on family planning quality indicators", Q. A. Brief. 3(1): 1-2

Berwick, D. M.; Godfrey, A. B.; Roessner, J. (1990) Curing Health Care: New Strategies for Quality Improvement, A Report on the National Demonstration Project on Quality Improvement in Health Care, San Francisco: Jossey-Bass

Brown, L. D.; Franco, L. M.; Rafeh, N.; Hatzell, T. (1993). Quality assurance of healthcare in developing countries. Quality Assurance Methodology Refinement Series, Bethesda, Maryland, Center for Human Services, Quality Assurance Project, 4 (27) USAID Contract No. DPE-5992-A-00-0050-00

Bruce, J. (1989, May). Fundamental elements of the quality of care: a simple framework, The Population Council, Working Papers. (1)

Calla, C (1991, Jan). Concepts of quality of care in family planning: programmatic implications for the Family Planning Services Division, Office of Population. [Unpublished working paper], 5 (24)

Crosby, P. B (1979). Quality is Free. The art of making quality certain, New American Library.

Crosby, P. B (1986). Quality without Tears: The Avenue of Hassle-free Management, McGraw-Hill International edition.

Deming, W. E. (1986). Out of this Crisis, Cambridge University Press, Cambridge, 507 pages

Donebedian, A (1985). The Methods and Findings of Quality Assessment and Monitoring. Explorations in Quality Assessment and Monitoring, Health Administration Press. 3

Donebedian, A (1988, Sept). The Quality of Care: How can it be assessed?. JAMA, 260(12): 1743-1748

Fisher, A. et al (1983). Handbook for Family Planning Operations Research, The Populations Council

Forsberg, Birger C., Fernando C. Barros, and Cesar G. Victoria 1992. "Developing countries need more quality assurance: how health facilities surveys can contribute". Health Policy and Planning 7, 2: 193-196

Hull, V. J (1996). Improving Quality of Care in Family Planning: How far have we come? South and East Asia, Regional Working Papers, (5), The Population Council, Jakarta, Indonesia

Jain, A. K (1989, Feb). Fertility Reduction and the Quality of Family Planning Services. Studies in Family Planning, 20 (1) : 1-16

Jain, A. K (1992). Definition and impact of quality. Managing quality of care in population programs, edited by Anrudh K. Jain., West Hartford, Connecticut, Kumarian Press,: 10-9

JHPIEGO (1991): John Hopkins Program for International Education in Gynecology and Obstetrics; "Evaluation of the reproduction health and education skills training project for anglophone West African nurse-tutors," [Unpublished working paper]: 2

Juran, J. M (1989). Leadership for Quality: An Executive Handbook, The Free Press, New York: 376 s

Katz, K.; Hardee, K.; Villinski, M (1993). Quality of Care in Family Planning: A Catalog of Assessment and Improvement Tools, Family Health International, Durham, North Carolina

Kumar, S; Jain, A.; Bruce, J. (1989, Oct). Assessing the Quality of Family Planning Services in Developing Countries. The Population Council, Programs Division Working Paper

Morehead; Mildred, A.; Donaldson, R (1974, Apr). Quality of Clinical Management of Disease in Comprehensive Neighbourhood Health Centers. Medical Care 12: 301-15

Parasuraman, A (1987). Customer-oriented corporate cultures are crucial to service marketing success. Journal of Services Marketing

Patton, M. Q (1986). Utilization-focused Evaluation, Sage Publications, 2nd edition

Raghavan-Gilbert (1991), A Program Quality Implementation Assessment Tool. Proceedings of the American Public Health Association: Population and Family Planning Section, Atlanta

Raghavan-Gilbert, P. (1997), "Service quality management in family planning: the programme quality assessment model, a multi-purpose management tool", Ph.D. Dissertation, University of Exeter, UK.

Raghavan-Gilbert, P.; Philips, D. R.; Gilbert, A. L (1998). Design and Implementation of a Program Quality Assessment Tool: Three Case Studies of Primary Health Services in Developing Countries. International Journal of Quality and Reliability Management Vol. 15 No. 8/9, MCB University Press UK.

Shewart, W.; Economic Control of Quality of Manufactured Product (Van Nostrand, 1931), Reprinted in 1980 by the American Society for Quality Control, reprinted in 1986 by CEEP Press, Washington, D.C.

USAID (1990), "Report of the Subcommittee on Quality Indicators in Family Planning Service Delivery," Task Force on Standardisation of Family Planning Program Performance Indicators, US Agency for International Development, Washington, DC.

Zeithaml V. A.; Parasuraman A.; Berry, L. L (1990). Delivering Quality Service, Balancing customer perceptions and expectations, The Free Press, New York: 205