Implementation of the LLIN campaign
Activities which formed the core basis of the universal LLIN campaign in Ghana included pre-registration of persons and sleeping places, door-to-door distribution of LLINs by volunteers including hang-up activities, and post-distribution ‘keep-up’ behaviour change communication activities.
Trained volunteers registered the number of people and sleeping spaces in households in the districts to benefit from the Hang-Up Campaign. This was used as the basis for the allocation of the LLINs and related items. In the first instance, the number of LLINs was calculated as number of household members divided by two; if this was greater than or less than the number of sleeping spaces then the number of LLINs was adjusted to equal the number of sleeping spaces.
Trained volunteers were informed of the final LLIN allocations to households in their community and collected the LLINs from the nearest pre-positioning site. The volunteers then delivered the allocated number of LLINs to each household. Where they were permitted to enter, they helped the occupants to hang the nets above each sleeping space in the household. Where they were not permitted to enter, they were instructed to provide rope and nails. In both cases the volunteers also delivered behaviour change communication (BCC) messages about use.
The volunteers who conducted the household registration and hang-up were recruited from the communities in which they worked. They relied on local knowledge of community members and especially household heads to identify registered households for LLIN distribution. Each hang-up team (made up of three volunteers) had a member who had participated in the household registration in the area (s) where the team carried out the hang-up.
Study setting
This paper focuses on evaluation of the LLIN Campaign in Brong Ahafo, Central and Western regions, which were selected for pragmatic reasons, based on timing of data collection relative to campaign implementation and funding support. The selected regions cover all three of Ghana’s ecological zones: Central and Western regions are in southern Ghana and include parts of the coastal and forest ecological zones; Brong Ahafo is in central Ghana and includes parts of the forest and savannah ecological zones. There are nevertheless socio-economic and cultural differences between these regions and those in the north or east of the country. Likewise, since the selection of Brong Ahafo, Central and Western regions for this evaluation was not random it is not possible to collate the regional results to produce a statistically representative national average. This would also be hard to interpret as universal campaign implementation was phased across the country, starting in Eastern region in December 2010 and ending in Greater Accra Region in October 2012.
Overview of evaluation design
An uncontrolled before-after design was chosen for this evaluation, with attribution of effects of the LLIN Campaign through collection of data on source of nets owned by households and a thorough process evaluation [7]. A mixed methods approach was taken, involving quantitative pre- and post-campaign household surveys, post-campaign in-depth interviews and focus group discussions with key stakeholders, and a costing analysis.
Because the same implementation strategy for the campaign was used in all ten regions of Ghana, it was not possible to use a randomized controlled trial or even a controlled before-and-after design to allow for the individual components to be evaluated separately. Therefore, the incremental cost effectiveness of the ‘hang-up’ component was estimated using reported variation in the implementation of hang-up activities and LLIN use. To facilitate this comparison, detailed questions on exposure to each element of the campaign process were included in the post-campaign household survey questionnaire.
LLIN distribution in Central and Western regions took place in November-December 2011; distribution in Brong Ahafo was in May-June 2012. The post-campaign household survey was conducted in September-October 2012, approximately 11 months after LLIN distribution in Central and Western regions and five months in Brong Ahafo region.
Collection of costs data
Financial and economic costs were collected from the societal perspective, meaning that direct and indirect costs to both LLIN providers and recipients were incorporated. An ingredients approach was used to identify all resources required to deliver LLINs through a mass universal campaign with hang-up activities [8].
Financial costs were obtained retrospectively from the financial reports and accounts of the implementation partners. Research and evaluation costs were not included. Costs were measured in Ghanaian Cedis (GHC) or United States Dollars (USD), depending on the currency of the original expenditure. Costs in GHC were converted to USD according to the average exchange rate for the year of the expenditure (1 USD equivalent to 1.42 GHC in 2010, 1.52 GHC in 2011, and 1.81 GHC in 2012). All costs were adjusted for inflation and are presented as 2012 USD using the consumer price indices available from the International Monetary Fund [9].
Economic costs recognize that the cost of using resources means that these resources are unavailable for productive use elsewhere, and include costs such as donated goods or volunteered time spent on the intervention. Information on the time that volunteers spent on campaign activities, what they would have been doing if not working on the campaign and estimates of their usual income was collected during focus group discussions. The time spent by GHS personnel and other partners on training and supervision for which they did not receive direct salary support was also valued and included as an economic cost.
Capital goods with an expected lifespan of more than one year were annualized using a discount rate of 3% according to the guidelines of the World Health Organization [10]. The lifespan of cars and motorbikes used in the campaign was estimated to be 5 years, based on information from UNICEF. An average useful lifespan of three years was assumed for the LLINs [3]. Other one-off costs of the campaign were also treated as capital costs, including sensitization, household registration and LLIN distribution and hang-up; essentially these are investments, which are expected to last as long as the useful LLIN lifespan. These costs were annualized across the average LLIN lifespan of three years using a discount rate of 3%. Consistent with the running costs of the UNICEF Ghana office, overheads were included at a fixed value of 7% of all financial provider-level expenditure. Economic costs are presented as the average annual economic cost over the effective lifespan of the LLIN.
Questions on whether the household had made some contribution to the volunteer for hang-up and the value of the contribution were included in the household survey questionnaire, along with questions to investigate the amount of time any member of the household had to wait at home for the LLIN hang up visit.
Measurement of campaign effectiveness
The number of LLINs distributed in each region was obtained from GHS district reports. The effect of the campaign on LLIN ownership and use was measured using household surveys. Pre-campaign baseline data on LLIN ownership and use is provided at regional level by the Multiple Indicator Cluster Survey (MICS) conducted in September-December 2011 [11]. Post-campaign data on LLIN ownership and use was collected in Brong Ahafo, Central and Western regions in September-October 2012 using the same survey design and standardized questionnaire as MICS to allow direct comparison. Additional questions on households’ exposure to each phase of the LLIN Campaign were also included in the post-campaign questionnaire, including whether campaign LLINs were hung by the volunteer or not. Briefly, the post-campaign household survey followed a two-stage cluster sample design. At least 543 households were required in each of the three evaluation regions to give an estimate of the proportion of children under five who slept under an LLIN the night before the survey to within 8% precision (further details to be reported elsewhere; Awini et al., personal communication).
Data processing and analysis was conducted in Stata 12.0, using ‘svy’ commands to ensure confidence intervals were appropriately adjusted according to the design of the surveys. Determinants of post-campaign LLIN use were explored using logistic regression; explanatory variables with a p-value of <0.1 in univariable analyses were included in the final multivariable model.
Cost effectiveness analysis
The increase in number of individuals that slept under an LLIN was estimated using data from the pre- and post-campaign household surveys and population data from the 2010 Ghana Census for each region. The measures of effect were:
-
∎ Additional number of persons using an LLIN – difference between number of individuals sleeping under an LLIN pre-campaign and the number sleeping under an LLIN post-campaign.
-
∎ Additional number of children under five years using an LLIN – difference between the number of children under five sleeping under an LLIN pre-campaign and the number sleeping under an LLIN post-campaign.
-
∎ Additional number of all-cause under five deaths averted – estimated number of deaths averted each year from use of LLIN was predicted based on the change in the number of children under five years sleeping under an LLIN following the campaign and the pooled estimate of 5.5 child deaths averted per 1000 children protected by insecticide-treated nets (ITNs) [1].
For each of these outcomes, an incremental cost effectiveness ratio (ICER) was calculated i.e. the additional effect (person using an LLIN or death averted) for additional cost of the LLIN Campaign compared to no campaign.
Sensitivity analyses
The cost analyses involve a number of important assumptions. To investigate the effect of these assumptions on the results, each was varied in turn in one-way sensitivity analyses. Estimates of the useful lifespan of an LLIN vary by setting, therefore a lower value of two years and upper value of five years were explored [3]. These changes in LLIN lifespan were applied to all annualized campaign costs. The financial cost of the LLINs decreased considerably over the period of the LLIN Campaign. Cost estimates of USD 3.25 and USD 4.80 per LLIN used in the sensitivity analysis were median 2012 prices for a rectangular net (as distributed in the LLIN Campaign) or a conical net, respectively [12]. To explore the importance of assumptions regarding the predicted mortality impact of the LLIN Campaign (such as the influence of transmission intensity), the upper and lower uncertainty limits for the pooled estimate of 5.5 child deaths averted per 1,000 children protected by an LLIN were used (95% CI: 3.39, 7.67) [1]. The discount rate was also varied with lower and upper values of 0% and 10% [13].
Estimating the incremental cost-effectiveness of hang-up activities
The study design did not allow direct comparison of a universal LLIN campaign with and without hang-up activities. However, the effect of the hang-up was derived by comparing LLIN use in households in which volunteers had hung-up one or more LLINs with use in households where the volunteer did not hang the net (s). Costs were estimated of a universal campaign without hang-up visits to households. In this alternative scenario, LLIN distribution was conducted from a fixed-point without any household follow-up visits for hang-up: it was assumed that the same volunteers involved in household registration attended a fixed distribution point for three days to deliver nets to recipients; the number of volunteers needed for registration and fixed-point distribution is approximately one-third that for door-to-door distribution. No time was included for house-to-house visits by volunteers. Supervision of distribution was reduced from ten days to three days. All other costs were kept constant.
The effectiveness on LLIN use of this campaign strategy was assumed to be lower, based on data from the post-campaign household survey data which found that LLIN use amongst individuals and children under five living in households where all or some of the nets were hung by a volunteer was significantly higher than LLIN use amongst those living in households where nets weren’t hung by the volunteer.
Thus, the ICER of a campaign with hang-up activities compared to one without was calculated. The sensitivity of this predicted ICER to the assumption about the additional effect of hang up was explored by varying the odds ratio of LLIN use by children under five living in households where nets were or were not hung by a campaign volunteer.
Ethical considerations
Approval for the evaluation was granted by Ghana Health Service. Ethics approval was granted by the Observational/Interventions Research Ethics Committee of the London School of Hygiene & Tropical Medicine. Appropriate authorities were informed in each region and district involved in the evaluation. Individual informed written consent to participate in the study was obtained from heads of households and key informants.