Costing the distribution of insecticide-treated nets: a review of cost and cost-effectiveness studies to provide guidance on standardization of costing methodology

Technical characteristics

Types and clarity of cost measures

Financial costs of the service provider were included in all but one study, which was costed from the patients' viewpoint. Considerable variation was observed between studies in the items identified and measured. No general template could be identified by which authors had decided which inputs to include. Few studies detailed all the inputs, their unit cost, quantities consumed and reasons for inclusion/exclusion in the analysis. This observed lack of consistency and often detail could not be attributed to authors not consulting the relevant health economics literature. More than half of the studies referred to papers on costing or to one of the leading texts in this field. Most frequently quoted were Phillips et al [15], Creese and Parker [16], the first and second editions of Drummond et al. [17] and Gold et al. [18].

Insufficient attention was given to the cost of reaching specific target groups; in the case of ITNs these are generally children below the age of five or pregnant women. Two papers explicitly recognized this cost. Guyatt et al. [19] calculated two costs, one for delivery of nets to all users and a higher one for delivery to pregnant women (i.e. accounting for the proportion of nets that had been taken-up by other users). An intervention trial in northern Ghana considered it necessary to distribute nets to all family members to ensure that children were protected and considered this in the cost estimate [20].

Studies varied greatly in the way in which financial costs were adjusted to obtain economic costs (i.e. annualized and discounted) and which other economic costs, such as donated items and time of volunteers, were included. Eight studies only included financial costs. Some studies quoted the cost net of any cost savings, and one study included the community effect, i.e. the protection gained by people without a mosquito net when sleeping near ITN users. Comparison of cost estimates between studies that had used the same output was therefore not always meaningful.

Sources of cost data

A number of studies were undertaken alongside the early efficacy trials for ITNs, which will have facilitated the collection of cost data. However, the resulting cost estimates may be an overestimate of the costs of a routine intervention because efficacy trials require close monitoring and supervision to provide reliable clinical results; in such circumstances effectiveness may also be overestimated compared to routine conditions [21]. On the opposite end of the spectrum, one study collected cost data for ITN distribution alongside measles vaccination, but did not fully account for the proportional use of campaign resources. Some studies did not collect primary data on expenditure. Instead, investigators either used results of other studies to recalculate cost estimates under different assumptions or a combination of results from other studies extrapolated to a generalized scenario [see [22]].

Issues of time

An ITN programme generally lasts for more than one year and the capital items purchased to implement it mostly have a life expectancy in excess of one year. It is therefore useful to express capital costs as an annual equivalent, which requires judgements on the average life expectancy of each type of item and choice of a discount rate. Capital inputs that were annualised included mosquito nets, vehicles and equipment. Assumptions on the life expectancy and discount rate varied considerably between studies. Mosquito nets were considered to last three to seven years, the need to re-treat them was estimated at once or twice a year and the number of people protected by each net ranged from one to more than three. Discount rates applied to capital items ranged from 3% to 10%.

Choice of output and outcome measure

Costs were generally quoted in US$ and in some cases also in the equivalent local currency. Two studies provided results only in local currency. Overall, five intermediate output measures were encountered: i) Cost per net delivered, ii) Cost per insecticide treatment delivered, iii) Cost per ITN delivered, iv) Cost per treated net year, and v) Cost per person protected. More than half of the studies provided only one of these; the preferred one being cost per ITN delivered. Programmes that did not distribute pre-treated nets could have distinguished the costs of delivering the net and insecticide, but often failed to report these separately. Outputs were thus not easily comparable between studies and the information provided was insufficient to calculate cost for a different output measure from that provided by the authors.

Most of the cost-effectiveness studies determined their own effectiveness data and provided one or more of a selection of outcome measures, such as cost per case averted, cost per death averted or cost per disability-adjusted life year (DALY) averted. As in case of the outputs reported, the selection of outcome measures varied between studies, with most authors reporting results for only one.

Sensitivity analysis

Cost estimates inevitably involve some assumptions and methodological controversy. To account for this, careful analysts need to identify critical assumptions and areas of uncertainty and then re-estimate the results using different assumptions to test the sensitivity of the results and conclusions to such change. Of the studies reviewed, approximately half provided some form of sensitivity analysis. Eight (i.e. 73%) of the cost-effectiveness analysis included this component, as opposed to four (29%) of the studies that provided cost estimates.

Viewpoint

Cost analysis should be undertaken from the broadest perspective, that of society, to incorporate all costs regardless of whom incurs them [18]. Other perspectives, such as that of the provider, patient or Ministry of Health, can be included alongside this perspective.

Costing

General costing methods are outlined elsewhere and should be consulted prior to the study. To aid generalisability and to ensure transparency, the required resources, their quantities and their value should be reported separately. Specifics related to ITN programmes are as follows.

Resource identification

The costs of any activities undertaken to educate people to facilitate behavioural change should be accounted for, including any advertising that is part of the intervention. It should also be made clear whether an intervention provides mosquito nets or treatment only, or whether it delivers both components. For the latter, separate costs should be provided.

In areas where LLINs are being introduced alongside conventional nets the resources associated with this change, such as additional educational campaigns/materials and the amount of insecticide wasted by re-treating LLINs during net treatment campaigns should be included. For example, the malaria control programme in Uganda treats all nets while LLINs are being phased in. In 2004, an estimated 25% of all nets were LLINs, amounting to approximately 500,000 nets (A. Kilian, pers. com). In a recent campaign in 20 districts the treatment of LLINs brought for re-treatment resulted in a cost of approximately US$ 100,000 (cost includes insecticide, equipment and implementation).

For intervention trials, sensitivity analysis should consider the resources that may be required under routine programme conditions. Research costs should not be included in the intervention costs. However, it should be investigated whether the programme would operate as well if research were not carried out alongside it. If research staff are considered to be essential for the intervention, then this cost should be included [e.g. [5]]. Another parameter that should be subjected to sensitivity analysis is the proportion of shared resources in cases where distribution is carried out as a component of another programme/campaign. Here it is particularly important to be explicit about the criterion used to attribute resource use (e.g. % time, % value of resources), as the basis for the breakdown is always arguable. For collaboration between organisations (e.g. local NGO & international NGO), resources including overheads that are relevant to the programme need to be identified. The final cost estimate needs to reflect the fact that, for example, a programme was made feasible because an international NGO facilitated procurement, management, fund raising, etc.

Programme management resources need to be identified and clearly specified, even if the main purpose of the study is to compare two interventions provided by the same programme (e.g. ITN/IRS). To ensure generalisability it needs to be clear what capacity is assumed to exist and whether authors have calculated average costs for delivery of each intervention or the incremental cost of adding an intervention to an existing programme (e.g. ITN delivery alongside vaccination campaigns). In areas where a mix of locally made (cotton) and imported (polyester, polyethylene) nets is being impregnated, the extra resources required to treat the former need to be considered, as these nets take up at least twice the amount of insecticide [e.g. [31]].

Resource measurement

An attempt should be made to measure and report the leakage of resources, to be able to differentiate between measured outputs (e.g. nets delivered) and outputs that reach the intended target. To inform policy, it is essential to determine the degree to which different approaches to targeting achieve their aim and at what cost.

It should be established how long nets last under local conditions. Observations to date indicate that assumed life expectancies of more than three years are unrealistic (except in the case of polyethylene nets such as Olyset^®). In general, a relevant timeframe needs to be used for the costing. As most projects last for more than one year there may be issues around price level adjustments, so that all the costs can be expressed in the same year, and because of lumped expenditure (i.e. high start-up costs, but falling over time). The general recommendation with regards to tracking of costs is to select a follow-up period that does not bias the analysis in favour of one intervention over another [17].

Resource valuation

Mosquito nets need to be treated as a capital item. Annualization based on a 3 years life span is recommended. Should surveys indicate otherwise, costs for observed and assumed 3 year duration should be provided. To all capital items a discount rate of 3% should be applied, to be consistent with the rate used by the World Bank [32]. Sensitivity analysis should also include 0%, 5% and 10%, to presents costs in their undiscounted form, as well as at higher rates chosen by some analysts.

When comparing alternatives, for example ITNs and IRS, the same or equivalent components for their delivery need to be costed. When using cost estimates reported in other studies, these need to have been presented transparently so that they can be adapted (e.g. by applying local prices to standard quantities). Otherwise, costs for a specific study need to be identified, measured and valued and to be assessed for consistency with other authors' estimates. For programmes that recover costs through user contributions, double counting needs to be avoided. For the societal perspective these costs should be included, whereas if the provider perspective is taken they should be excluded [e.g. [5]].

Data analysis

To reflect the uncertainty in measurements, a sensitivity analysis needs to be carried out on (at a minimum): discount rate, frequency of net impregnation, procurement cost of insecticide & net, number of people protected per net, life span of net. Where it is necessary to estimate a share of resources contributed from other programmes or interventions, the assumptions used should be subjected to sensitivity analysis.

Reporting of results

Cost estimates should be provided in US$ and local currency, and the year to which costs were adjusted needs to be specified. Projects producing nets locally for sale need to report on the cost of making/treating nets, not just the price at which these were sold to users, which may not fully measure the opportunity cost.

The cost per ITN delivered should be reported, as cost per person sleeping under a net varies depending on cultural practices and as cost per case of malaria prevented depends on the incidence of malaria. Both are useful for policy decisions in a given context, but the cost per net delivered is best suited for comparing delivery strategies. Cost-effectiveness studies should not just provide net costs (i.e. subtracting resources saved from total implementation costs), as these cannot be compared to results from costing studies.

Discussion of results

To put cost-estimates for a particular method, or combination of methods, into perspective, it is recommended that they are compared to the per capita expenditure on government health services in the country/region. This does not improve on the comparability of costing studies between countries, but will help readers to assess the affordability of the intervention [e.g. [3]]. When reporting results, one should beware of the assumptions required to extrapolate these to calculate costs of scaling-up an intervention. To date, studies that have investigated the cost of scaling-up health interventions are limited. Available data indicate that scaling-up costs are highly specific to both the type of intervention and its particular setting [30]. Without further study of potential additional cost and without considering factors such as human resources, geography and infrastructure, extrapolations are not valid because they assume constant marginal costs.

The recommendations are summarized in Table 2

The reference case