Impact evaluation of agricultural assistance projects in the Gaza Strip – FAO case study

DESIGNING & MANAGING INTERVENTIONS TO ACCOMMODATE COUNTERFACTUAL IMPACT EVALUATION:   

A CASE STUDY IN THE GAZA STRIP 

OSRO/GAZ/912/UK, 2010

Food and Agriculture Organisation of the United Nations (FAO-West Bank & Gaza Strip)

Abstract

The following case study documents and discusses FAO’s first attempt in the West Bank and the Gaza Strip to implement an intervention which includes a rigorous impact evaluation component. The report discusses the methodologies used to design an intervention which considers impact evaluation. The interdependency in the design of the package of assistance, the beneficiary selection process, the impact evaluation design and the implementation of the activities must be recognised. The profiling of beneficiaries establishes a set of criteria to inform the design of a standardised package of assistance. Equally important, the design of an impact evaluation requires that the package of assistance is standardised and the beneficiaries of similar types of socio-economic profile and poverty level are randomly selected. Finally, a randomly selected comparison group of the same socio-economic profile as the beneficiary group (that did not receive the same package of assistance), were selected in order to assess whether the intervention directly impacted key household indicators or whether, and to what extent, identified changes are influenced by external factors.

Introduction

On the 27 December, 2008, Israel launched a military offensive, Operation Cast Lead on the Gaza Strip (December 2009 – January 2010). The impact of Operation Cast Lead left an estimated USD 269 million in damages to the Sector. Additionally, the Agriculture Sector in Gaza has been severely hampered by the almost complete blockade in place since 2007. The Agricultural Sector has traditionally served as a shock absorber for communities and has the potential to significantly improve the quality of life, food security and nutrition of an estimated 1.5 million people. With a little over two-thirds of the population deemed food insecure and vulnerable to food insecurity, damage to the livelihoods of farmers has meant households affected by Operation Cast Lead have become vulnerable to further shocks due to the loss of their productive assets and risk falling deeper into poverty.  

In June 2009, a total of 46 percent of agricultural land was assessed to be inaccessible or out of production¹. Since the last military offensive, households have claimed land reclamation certificates for submission to the Ministry of Agriculture (MoA) in order to be eligible for land rehabilitation projects. Within a limited operational space, humanitarian agencies have attempted to mitigate further deterioration of farmer’s and fisher folk’s livelihoods².  Investments in simple inputs and the increase of access to land can generate substantial immediate improvements in the livelihoods of food insecure farming households. However, such interventions rarely measure the impact on savings and spending patterns of households.  

In December 2009, the Food and Agriculture Organization of the United Nations (FAO) with the support of the Government of the United Kingdom and Northern Ireland Department for International Development (DFID) embarked on a project with the aim to provide immediate support for the endangered livelihoods of food insecure farmers, herders and fisher folk alongside improving the impact of FAO interventions by embedding a rigorous impact-evaluation component in its interventions. Project OSRO/GAZ/912/UK, ‘Immediate Support for Endangered Livelihoods of Food Insecure Farmers, Herders, and Fisher folk in the Gaza Strip’ (hereinafter Project 912), became the first FAO intervention in the West Bank and Gaza Strip (WBGS) to include this impact evaluation.

The Project Team

The project team consisted of 7 core members including support staff from three organisations: the Job Creation Programme of the Office of the President of the Palestinian National Authority (JCP); Al-Sahel for Institutional Development and Communications (Al-Sahel) and FAO.  

The role of the project team was as follows: JCP was responsible for the selection of potential beneficiaries and to implement the farmland reclamation packages with a workforce of labourers. Al-Sahel was responsible for conducting the survey upon which the impact evaluation was based. Lastly, FAO was responsible for the project design, partner selection, procurement and supervision of the distribution of inputs, conduct the impact evaluation, and manage the overall project.  

The team was further broken down into activity-specific sub-teams which consisted of the project design, surveying, intervention implementation and impact evaluation. As shown in the team-structure diagram below, the Project Manager was the focal point for coordination and communication, supervising the sub-team leads and the overall project.   In the diagram, the lead for the activity specific sub team is listed first in bold.

The activities falling under the project were designed according to the following: intervention package design; beneficiary selection; counter factual impact evaluation design; and intervention implementation. The following sections are organised according to the chronological order of activities and includes a description of the activity, considerations and decisions and the actions it entailed and an analysis of the activities providing guidance on the successful implementation for future activities which consider an impact evaluation in the design.

Intervention Package Design

Project 912 was designed mainly in response to the destruction of farmland that took place during Operation Cast Lead, which also destroyed the main productive assets of many farming households. This posed both a significant loss to household’s main source of income and to their food source. Therefore, the main objective of Project 912 was to contribute to the process of reclaiming these productive assets by providing farmers with a farmland-reclamation package.  

The project aimed further to protect the livelihoods of an additional 128 food insecure households who received a partial land rehabilitation package and an additional 50 herders and fisher folk by providing a small-livestock farm package. These beneficiaries were located in the Al Mawasi locality which spreads across the Rafah and Khan Yunis coast. Beneficiaries of the project were further extended to 128 labourers selected from the JCP, who carried out the work on farmland reclamation. The labourers worked for two months at the local daily wage rate.³ 

The type and scale of the intervention package was initially designed based on an informal need assessment carried out by FAO’s agronomist in the Gaza Strip. The farmland package was designed to provide beneficiaries with two dunums of vegetables that would be, in a matter of two months, available for consumption and/or sale at the local market. The standardised farmland reclamation package involved clearing two dunums of land, installing an irrigation network, fertilizing the land and sowing it with the beneficiary’s preselected choice of seasonally appropriate corn, okra, squash, kidney beans, and/or water melon seeds.

The package included one day training workshops that were conducted by JCP’s local agronomists.  The training informed beneficiaries about crop, fertiliser, compost, irrigation network management and safety practises in the use of pesticide. The feedback received by JCP was positive with participants particularly grateful for advice on how to minimise costs.

The small livestock package was designed for herders and fisher folk. It consisted of two pregnant sheep and 100 days worth of fodder (200 kg of barley and 120 kg of wheat bran). Although the main work for fisher folk is fishing, herding is a common source of food and income now due to the restrictions put on fishing by the ongoing blockade of Gaza.  

The two pregnant ewes were expected to provide beneficiaries with milk, wool, and compost , all of which could be used either for domestic consumption or sold on the local market, Once the sheep gave birth, the lamb could be consumed, sold or kept on the farm.

Due to differences in intervention packages provided to farmers and to herders and fisher folk, it was decided to conduct the impact evaluation only on the intervention packages provided to farmers. All components of both the farmland reclamation and livestock packages followed strict technical specifications outlined by FAO’s agronomist in Gaza, with technical support from FAO headquarters (HQ).

Discussion

There are two main categories of intervention targeting designs: self-targeting and indicator-contingent administrative targeting (Barrett 2002).

Both categories of intervention design aim to provide beneficiaries of a particular type and magnitude of poverty with an appropriate type and magnitude of intervention. This match is often difficult to achieve and requires careful consideration of the type and magnitude of both poverty and intervention.

The first step in the intervention package design should be the profiling of intended beneficiaries. Profiling of intended beneficiaries involves identifying indicators that would deem a beneficiary eligible for assistance based on their socio-economic characteristic and in relation to their level of poverty. In order to standardize a package of intervention, the selection of beneficiaries must all be match to the same socio-economic characteristics and level of poverty. However, profiling requires appropriate national or district level statistics available or funds to conduct one. In any intervention there will be a trade off between the accuracy of information and time and budget resources available to collect information. In the case of Project 912, the statistics were already available through the Socio-Economic and Food Security Survey 2 (SEFSec)⁴ Monitoring System which partially laid the ground work for an impact evaluation to be undertaken.

Within Project 912, the profiling of intended beneficiaries was conducted by the FAO Food Security Analyst who used her/his experience in poverty and vulnerability assessment to define the intended beneficiaries according to the degree of poverty. The profile of beneficiaries was based on data collected for the SEFSec survey according to the typical household composition of food insecure households⁵. Once the profiles of intended beneficiaries were created and the local context considered, the food security analyst and the agronomist worked together to identify the appropriate intervention package that best fits the beneficiary profiles.  

The informal nature of the initial needs assessment conducted presented potential challenges in appropriate targeting and design of the intervention, mainly due to the fact that specific household characteristics and indicators were needed. Although the assessment considered poverty levels, it did not provide an in-depth assessment of household characteristics including: income, gender of head of household, number of members, number of those unemployed, number children, women, elderly, members with disabilities and the number of dunums of land households owned. This information would have shown how specific household characteristics affected the magnitude of household poverty and was needed for the design of an appropriate standardised package of assistance.

This implies that that the impact of standardised intervention packages would vary among households due to significant differences in household characteristics. In other words, there was potential for a mismatch between the type and magnitude of poverty and the selected intervention, particularly because the extent of the mismatch on the distribution of the type and magnitude of poverty among the beneficiaries.  

To minimise this mismatch, the project team chose to tailor intervention packages to proportionally match unique household profiles.  In the case of farmers, the amount of farmland reclaimed was proportionate to the magnitude of poverty. Similarly, with the herders and fisher folk, the number of sheep provided to each household was based proportionate to household poverty levels. Alternatively, if the provision of a set number of sheep is preferred, then only the beneficiaries with a similar type and magnitude of poverty could have been selected for the intervention.

Profiling could also be used to ensure that intervention components go toward their intended use. For example, the chances of the sheep and/or the lamb being prematurely consumed, sold, or traded are likely to be higher amongst food-insecure households than among households who are food secure; suggesting that poor, but relatively food-secure, herders and fisher folk were more likely to use the sheep and lamb for their intended purpose, i.e. as productive assets, then relatively food-insecure herders and fisher folk.

Additionally, determination of the duration of an intervention needs to rely on strategic consideration of the time required to actualize the intervention’s objective of providing sustainable income generation. For example, in Project 912, provision of fodder over a longer time period may help to ensure the sustainability and even expansion of the small livestock package.

When an impact evaluation is planned, it is important that the package is standardized based on beneficiaries which are selected according to the type of poverty and the scale.

Beneficiary Selection Design

The geographic location of the planned intervention was selected by the project team according to a three-step process. The first step was to use the results of a FAO Resilience Analysis⁶ model conducted in 2009 to identify the localities with households least capable of dealing with adverse change. In order to avoid overlap with other projects, the second step was to exclude localities where similar intervention packages were recently or currently being implemented. In the case of farmland-reclamation beneficiaries, the third step was to select from the remaining localities those with the greatest number of households with destroyed farmland. For beneficiaries receiving the livestock package, the third step was to select from the remaining localities those including herder and fisher folk communities.

An additional step for selecting the geographic location for farmland reclamation was to avoid farmland in the Israeli-designated 500-meter⁷ buffer zone, which is located on the Gaza side of the border with Israel. Although the land is farmable and fertile, any activity in the area provokes Israeli gunfire and/or incursions. For this reason, farming in the buffer zone has largely ceased. Since the address documented in the potential beneficiary list was sometimes of the farmer’s home and sometimes of her/his farmland, it was left to the ad hoc survey conducted by Al-Sahel to filter out potential beneficiaries with farmlands located in the buffer zone.  

Seven localities were identified with three in South Gaza (Al Mughraqa, Sheikh Ijleen, Juhor ad Dik) and four in Khan Yunis (Abasan Kabira, Abasan Saghira, Al Fukhkhari, and Khaza’a). As afore-mentioned, evaluation was only to be conducted on farmland-reclamation packages.

Discussion

The first challenge addressed was that the selection of farmland-reclamation beneficiaries did not allow for a randomized evaluation. Following some discussion an agreement was made that helping poor farmers reclaim destroyed land was an integral characteristic of the project and that, as result, due to time, budget constraints, and ethical reasons, it would have been difficult to randomly select the farmland-reclamation beneficiaries. This challenge was overcome through the application of appropriate statistical techniques (the importance of randomness in impact evaluation and the techniques applied in adjusting for its absence in the beneficiary-selection process are discussed in greater detail in the COUNTERFACTUAL IMPACT EVALUATION section).

The second challenge was selecting the most qualified 350 beneficiaries from the provided list of 400 potential beneficiaries. A number of ranking techniques exist in statistics, which allow for the ranking of beneficiaries according to pre-selected criteria and, in turn, the selection of the first 350 most qualified as the final beneficiaries. Applying the criteria, however, would have required criteria-specific household data that, at that point had not yet been collected.

The option of using the data from the to-be-conducted survey was considered and rejected when the team realized that there would not have been enough time between the completion of the survey and the beginning of the implementation for a proper application of a set of criteria. However, to negotiate this time constraint, an agreement was made to use an arbitrary chronological ordering of beneficiaries in the spreadsheet provided by JCP as the ranking methodology. This, however, did not ensure that the farmland-reclamation beneficiaries were all eligible, because while the list consisted of beneficiaries who submitted a destroyed-farmland certificate to the MoA, it had no information on whether the farmland had since been reclaimed.  

Furthermore, Al Sahel in its survey-testing stage uncovered a significant number of duplicates in the farmland-reclamation beneficiary list. It turned out that a number of households allocated land among their members in order to ensure higher chance of getting more farmland reclaimed. After the destruction of farmland during Operation Cast Lead, households in Gaza were also asked to fill out certificates with the MoA. During reclamation, those households with larger destroyed farmlands found it unfair that the compensation was not proportional to the size of destroyed land. Therefore, following Operation Cast Lead, many household-heads separated their farmlands among various household members to account for their destroyed-land size.

This, in turn, created additional challenges. For one, it was important for impact evaluation to have package provisions standardized, either by having a precise formula for providing the package proportionally to each beneficiary or by providing one standard package, irrespective of farmland characteristics. Secondly, the questionnaire was designed with the household as the unit of study. Having numerous beneficiaries as household members of the same household unit implied that package provision could not be standardized and that the beneficiaries would be reporting on incomplete or overlapping incomes and spending practices. What further made everyone take these challenges seriously was a JCP rough estimate that suggested that more than half of the potential farmland-reclamation beneficiaries may be duplicates.

By this time the questionnaire was finalized and Al Sahel had completed its pre-testing of the survey in the field. This opened a small window to utilize the survey to verify the eligibility of the beneficiaries. It was agreed that Al Sahel would use filtering questions in the survey to ensure that each household was a ‘farmer household’ and is only represented once on the list and that none of the farmlands selected to receive the reclamation package have yet been reclaimed.⁸ 

An agreement was also made that JCP would expand the potential beneficiary reserve to ensure there being surplus potential beneficiaries in case of high rates of replacement. Shortly after, JCP provided additional 113 beneficiaries from the same six localities, bringing the number of beneficiaries in the reserve from 50 to 163.

To avoid further delay in starting the intervention implementation, the decision was made to ask JCP’s local agronomists to offer training to all 400 potential beneficiaries on the list. If those who attended the training are later excluded from the final beneficiary list complained, they would be invited to participate in Project 912 as a worker or be added to another JCP project.

Fortunately, the duplication problem was relatively minor. In total, 25 duplicates were identified and replaced with potential beneficiaries from the reserve (Al Sahel 2010). A few other potential beneficiaries were also replaced because surveying indicated that they either did not live in the selected localities or were no longer farming. In the end, 350 final beneficiary households were identified for the farmland-reclamation intervention.

As Al Sahel surveyed the chosen localities, they verified the beneficiaries on the list and in real-time provided those eligible to JCP for the start of farmland reclamation. According to JCP, all eligible beneficiaries agreed to participate in the intervention.

Impact Evaluation Design

According to the World Bank’s Independent Evaluation Group, counterfactual impact evaluation is a comparison of post intervention outcomes of key welfare indicators with what the indicators would have been in the absence of the intervention. Counterfactual impact evaluation is preferably conducted in a controlled environment, as is often done in clinical trials, where a researcher can apply a randomisation procedure, in selecting the subjects to be studied. Utilising allocation concealment measures to ensure that subject allocations are hidden from researcher and the subject, the researcher can then apply a randomization procedure to allocate participants into two groups; one group who are recipients of the intervention and another  who are the non recipient comparison group.   

The procedure requires that the beneficiary group and the comparison group both consist of the same characteristics. This allows the researcher to record key information about the subjects in both groups before and after the intervention. It also allows the researcher to determine whether the intervention has had a direct impact on the household receiving the intervention. Thus the beneficiary selection process is integral to the design of the impact evaluation.

Due to operational conditions in the Gaza Strip the randomisation of the sample for the impact evaluation was not possible, meaning a potential sample-selection bias exists. A number of evaluation methods have been developed in response to this challenge in order to assist the researcher in drawing inferences that are still acceptably free of bias. These evaluation methods are either called pseudo- or non- experimental, depending on whether the comparison group is or is not randomly selected, respectively.

Another source of bias is called contamination and occurs either when the intervention in question has spill over effects onto the comparison group due to, for example, the two groups’ close geographic proximity, or when another intervention during the studied period affects the comparison group.⁹ 

Methodology¹⁰ 

In the case of Project 912, the impact-evaluation team took three steps with the surveying team and intervention-implementation team toward minimizing the chances of duplication. The first step was to exclude localities where similar intervention packages were recently being implemented. The second step was to make sure that members from households receiving the farmland-reclamation package were not amongst the workers paid by JCP to implement the intervention. The third step was to inquire in the questionnaire about whether the studied households benefited from any other intervention during the studied period.

While the selection of the beneficiary group did not follow a randomized procedure, the impact-evaluation sub-team was able to make sure that the selection of the control group did, with the households in the comparison group also being farmers and from the same seven localities.

This allowed the impact-evaluation sub-team to select two different pseudo-experimental evaluation techniques for tackling the potential sample-selection bias problem stemming from the lack of randomness in the treatment group. One of the evaluation techniques is called propensity-score matching while the other is called difference-in-differences.¹¹ 

Questionnaire

The questionnaire used in Project 912 was an adaptation of the Socio-Economic Food Security (SEFSec) survey¹². The SEFSec survey was originally designed in 2008 by FAO to assess the level of food insecurity in the West Bank and Gaza Strip (WBGS). The survey makes inquiries into a household’s: livelihood and food access status; overall expenditure patterns, with emphasis on expenditure on food; general coping strategies and needs; and access to various humanitarian interventions.

For the purposes of Project 912, the SEFSec survey was augmented to include an agricultural module. The extension makes a detailed inquiry into a household’s access to land, land use, and planting and production practices, with a focus on income and cost of inputs; allowing for analysis on the land and crop level (de la O Campos 2010). In total, the questionnaire consisted of 13 sections and a total of 104 questions.

The impact-evaluation sub-team’s ability to evaluate the intervention’s impact greatly depends on the success of the questionnaire design, i.e. the questionnaire’s ability to capture the impact through quantifiable questions/answers. Accordingly, sets of questions were carefully selected and worded to facilitate data collection.

The SEFSec survey was well fitted to accommodate the impact evaluation and resilience analysis. Its only potential weaknesses are its length and the

complexity of the impact-evaluation-related questions that many surveyed households found difficult to answer (Al-Sahel 2010). However, these questions were adjusted during the pilot test of the questionnaire in order to facilitate the data gathering process.

Surveying ¹³ 

The surveying process was headed by an Al- Sahel surveyor who first engaged in familiarizing the enumerators with the questionnaire. The surveyor then tested the questionnaire and the enumerators’ familiarity with it by piloting the questionnaire with several dozen randomly-selected potential beneficiaries. In the process, the questionnaire underwent eight revisions, during which questions were adapted to the local context and the chronological order of some questions was reworked (Al-Sahel 2010).

The many revisions at such a late point were necessary partially due to the fact that the involvement of a local FAO agronomist in the questionnaire-design process was delayed, underlining once again the importance of engaging relevant team members in appropriate sub-teams at suitable times.

Surveying of the beneficiary and comparison groups was to occur twice – once before the intervention and once after (after the fall harvest in September 2010). The random selection of the comparison group, which took place in the same six localities as the beneficiary group, was based on Palestinian Central Bureau of Statistics’ (PCBS) survey maps and followed a sampling procedure designed by an Al-Sahel Statistician. Although it was decided that only 600 farmer households were needed for the control group, 618 were randomly selected to account for the possibility that some of the randomly-selected households may have already been selected as potential beneficiaries.

As described in the BENEFICIARY SELECTION section, the pre-intervention round of surveying was also used to verify the eligibility of potential beneficiaries to receive the intervention. Al-Sahel surveyed the two groups area by area (with each locality being comprised of a number of areas), each time surveying the comparison group in a particular area immediately after surveying the potential beneficiary group in that same area.  

Overall, the surveying process took six weeks, with no more than one week elapsing between the surveying of the two groups in one area. Surveying of the control group occurred much quicker than expected because nearly ten times more households in the selected localities were eligible to be surveyed than initially expected (Al-Sahel 2010).  

Households on the comparison-group list that were already surveyed as part of the potential-beneficiary list were not surveyed again. They were recorded for the sake of documentation and replaced by another household from the comparison group list. Altogether, 10 enumerators, 2 supervisors, and 5 survey auditors conducted the pre-intervention survey under close supervision of the Al-Sahel Surveyor. The team of enumerators was intentionally small in order to minimize the potential for human error. Other surveying techniques were implemented by the Al-Sahel Surveyor to minimize human error, such as having each enumerator spend a similar amount of time in each locality.

Something that was not minimized is the speed with which the news of the intervention spread. Fortunately, this implied that every household approached to be interviewed as part of the comparison group hopefully agreed. Unfortunately, it also implies that knowledge of the intervention could have potentially further biased already-biased answers to income-related questions. The expectation generated by the survey that their completion of the survey may result in them receiving a package could have exacerbated this issue (de la O Campos 2010).

Following the survey, the data was entered into a database and underwent a thorough eight-day data-cleansing process.¹⁴ The data-cleansing process was time-consuming due to the complexity of many of the impact-evaluation related questions. In order to triangulate date, more than half of the surveyed households had to be contacted by phone during the data-cleansing process in order to clarify answers and ensure their consistency (Al-Sahel 2010).  

The post-intervention round of surveying is to be conducted by Al-Sahel later in 2010, after beneficiaries have a chance to harvest and benefit from consumption and/or sale.  

Analysis

The impact-evaluation team began preliminary data analysis immediately after the completion of the data-cleansing process. Preliminary analysis involved identifying relevant indicators, generating propensity scores, and matching households with similar scores in the treatment and control groups.

Following the post-intervention round of surveying of the beneficiary and comparison groups, the impact-evaluation team will begin applying the two evaluation techniques. The attrition rate at that time will need to be assessed and, if significant, explored to test the extent the drop-outs are a random subset of the initial groups¹⁵. Finally, the impact-evaluation team will test whether the impact of the intervention on intended indicators was statistically significant and, if yes, what was its extent.

Intervention Implementation ¹⁶ 

Implementation of the farmland reclamation and small livestock packages involved procuring the inputs and supervising their distribution, with both processes conducted by FAO. In the case of the farmland-reclamation packages, the implementation also involved JCP reclaiming the farmland and installing and sowing the irrigation networks and seeds, respectively.  

JCP coordinated machinery and labour on 350 two-dunam lots of farmland, spread out throughout the seven target localities. The most technical activity of the implementation process was the installation of the irrigation networks. The other activities were relatively straight forward.

Procurement of inputs was required to meet with FAO’s technical specifications for quality assurance and follow a procurement process outlined in FAO’s strict guidelines.  Although this process is necessary to ensure quality of project implementation, it also presented some delays to project implementation. A number of beneficiaries planted their own seeds without waiting for the technical clearance of FAO procured seeds out of concern for missing the planting season, as well as having to deal with the growth of weeds that began to appear as a result of rain falling on recently-ploughed but unplanted farmland.  

The impact-evaluation team agreed that the cost of seeds was an insignificant expense in the entire farmland-reclamation package and, therefore, while a record should be kept of which farmers have planted their own seeds, no other actions that would distinguish them from the other beneficiaries should be taken. It was also agreed that the farmers who planted their own seeds would still receive the seeds previously assigned to them as part of the farmland-reclamation package.

In the case of the livestock farm package, implementation was conducted solely by FAO and involved procuring sheep and fodder, verifying specifications of the sheep (which included sending blood tests to Ramallah), and supervising the distribution of the sheep and the fodder. The entire process went smoothly.

Discussion

The implementation of Project 912 included a procurement component, which involved a multi-step input- and supplier- selection and approval process that required close coordination among the FAO Gaza, Jerusalem, and Rome offices, and a multi-step preparation- and installation- process that required JCP coordinating machinery and workers among the 350 lots of farmland and across six different localities.  

Both components involved working in an environment characterized by the blockade of Gaza, which implied additional and often significant uncertainty and constraints. With this in mind, it is important that all implementation-affecting processes were carefully planned out in advance and are streamlined to assure that only uncontrollable delays have the chance to jeopardize the intervention’s successful and timely completion.

Conclusion

Project 912 is a good example of how scientifically-sound impact evaluation can be conducted in a relatively scientifically-unsound context. Despite some initial challenges related to intervention design and in input procurement, Project 912’s objectives were successfully met.  

The DISCUSSION sections throughout this case study highlight the importance of careful intervention design and management. To emphasize this importance, the FINAL REMARKS focus on the two subjects.

Intervention Design

Intervention designers should carefully consider the apparent type and magnitude of poverty, the suitable type and magnitude of intervention, and the mechanisms for timely and feasible delivery of the intervention to eligible beneficiaries.

This is true for any intervention, even those excluding an impact-evaluation component. However, interventions including impact evaluation, as exemplified throughout this case study, must keep impact evaluation in mind at each and every step of the intervention design process, as any changes in any part of the intervention design are likely to directly or indirectly affect impact evaluation.  

Intervention Management

Project 912 highlights the heavy interdependence between the implementation and impact-evaluation activities of the intervention and the importance of close and timely coordination and communication among the activity-specific sub-teams.

Many of the challenges that emerged during the implementation of Project 912 could have been reduced, if not entirely avoided, by engaging appropriate team members and at the right times.

While it is said that ten minutes of planning may save a hundred minutes in implementation, effective planning alone cannot ensure close and timely coordination and communication during intervention implementation, especially considering the increased levels of uncertainty and constraints arising from the Israeli imposed blockade.

References

Al Sahel (2010) Ad-Hoc Agriculture Household Survey in the Gaza Strip: Summary Report of the Survey Implementation, Ramallah: Al Sahel for Institutional Development and Communications

Barrett, Christopher B. (2002) “Food Security and Food Assistance Programs”, Handbook of Agricultural Economics, Volume 2, Edited by B. Gardner and G. Rausser, Atlanta: Elsevier Science B.V.

Besley, Timothy and Kanbur, Ravi (1990) The Principles of Targeting, Working Paper, Washington, D.C.: World Bank

Bonilla, Garcia A. and Gruat, J. V. (2003) Social protection: a lifecycle continuum investment for social justice, poverty reduction and development, Geneva: International Labor Organization

de la O Campos, Ana Paula (2010) Report on the Baseline Household and Agricultural Survey, Rome: Food and Agriculture Organization of the United Nations

Devereux, Stephen and Sabates-Wheeler, Rachel (2004) Transformative social protection, IDS Working Paper 232, Brighton: Institute of Development Studies

White, Howard (2006) Impact Evaluation – The Experience of the Independent Evaluation Group, Washington, D.C.: Independent Evaluation Group, World Bank

List of Acronyms

Al-Sahel Al Sahel for Institutional Development and Communications

DFID Government of United Kingdom and Northern Ireland Department for International Development

FAO Food and Agriculture Organisation of the United Nations

JCP Job Creation Programme of the Office of the President of the Palestinian National Authority

MoA Ministry of Agriculture

PCBS Palestinian Central Bureau of Statistics

SEFSec Socio-Economic and Food Security Survey

WBGS West Bank and Gaza Strip

WFP World Food Programme

Notes

¹ EUNIDA, Final Report: Damage Assessment and Needs Identification in the Gaza Strip, produced for the European Commission, March 2009.

² Since January 2009, fisher’s access to fishing grounds has been restricted to 3 nautical miles (nm) from the shore. The previous fishing zone was 6-9 nm before “Cast Lead”, 12 nm from Bertini Commitments, and 20 nm under the Oslo Accords.

³ The local daily wage in the Gaza Strip is USD 12 for unskilled labourers and USD 24 for skilled labourers. 

⁴ WFP/FAO, Socio-Economic Food Security Survey 2 – Gaza Strip, November 2009

⁵ WFP/FAO, Household Food Security Profiling in the Gaza Strip, January 2010

⁶ The resilience framework attempts to understand the socio-economic situation of a country in a more systemic manner, organizing information on the following building blocks: income and food access (IFA); assets (A); access to basic services (ABS) and social safety nets (SSN); as well as two cross-cutting dimensions – stability (S) and adaptive capacity (AC). This approach also provides the baseline for conducting rigorous impact evaluation of different programs and policies. Currently, the resilience analysis is conducted using household surveys in the WBGS, Kenya, Ethiopia, and possibly Sudan in 2010.

⁷ Formally, the buffer zone extends 300 meters from the Israeli border. However, numerous incidents involving Israeli gunfire at farmers attempting to farm their land close to the buffer zone informally extended the buffer zone to 500 meters.  

⁸ For a definition of what is a ‘farmer household’ and a discussion on the filtering questions, please see de la O Campos (2010).

⁹ This paragraph relies on White (2006).

¹⁰ With questions related to impact-evaluation methodology and/or resilience analysis please contact Erdgin Mane (FAO Econometrician; Erdgin.Mane@fao.org) or Ana Paula de la O Campos (FAO Economist; AnaPaula.DeLaOCampos@fao.org).

¹¹ For further details of evaluation techniques see FAO’s “Methodology Note on the Baseline Survey to Inform Programme Evaluation Design,” May, 2010.

¹² With questions related to the SEFSec survey please contact Rana Hannoun (FAO Food-Security Analyst; Rana.Hannoun@fao.org).

¹³ With questions related to the surveying process please contact Amer Madi (Al Sahel Project Coordinator; Amer@alsahelidc.com).

¹⁴ This paragraph relies on Al-Sahel (2010).

¹⁵ Attrition rate is the rate of households “lost” when the second round is conducted (either because they moved to another place and were unable to find, or enumerator was unable to locate household, or the household was not willing to participate, etc.). If we get too many of those, the rate is high, we have to assess if there is a consistent (not random) reason for it and act accordingly (i.e. picking a new sample, or adding to the sample).

¹⁶ With questions related to the implementation process please contact Badr El-Hussini (JCP General Manager; badr@jcp.ps).