Cluster-randomized study of intermittent preventive treatment for malaria in infants (IPTi) in southern Tanzania: evaluation of impact on survival

Abstract

Background

Intermittent Preventive Treatment for malaria control in infants (IPTi) consists of the administration of a treatment dose of an anti-malarial drug, usually sulphadoxine-pyrimethamine, at scheduled intervals, regardless of the presence of Plasmodium falciparum infection. A pooled analysis of individually randomized trials reported that IPTi reduced clinical episodes by 30%. This study evaluated the effect of IPTi on child survival in the context of a five-district implementation project in southern Tanzania. [Trial registration: clinical trials.gov NCT00152204].

Methods

After baseline household and health facility surveys in 2004, five districts comprising 24 divisions were randomly assigned either to receive IPTi (n = 12) or not (n = 12). Implementation started in March 2005, led by routine health services with support from the research team. In 2007, a large household survey was undertaken to assess the impact of IPTi on survival in infants aged two-11 months through birth history interviews with all women aged 13-49 years. The analysis is based on an "intention-to-treat" ecological design, with survival outcomes analysed according to the cluster in which the mothers lived.

Results

Survival in infants aged two-11 months was comparable in IPTi and comparison areas at baseline. In intervention areas in 2007, 48% of children aged 12-23 months had documented evidence of receiving three doses of IPTi, compared to 2% in comparison areas (P < 0.0001). Over the three years of the study there was a marked improvement in survival in both groups. Between 2001-4 and 2005-7, mortality rates in two-11 month olds fell from 34.1 to 23.6 per 1,000 person-years in intervention areas and from 32.3 to 20.7 in comparison areas. In 2007, divisions implementing IPTi had a 14% (95% CI -12%, 49%) higher mortality rate in two-11 month olds in comparison with non-implementing divisions (P = 0.31).

Conclusion

The lack of evidence of an effect of IPTi on survival could be a false negative result due to a lack of power or imbalance of unmeasured confounders. Alternatively, there could be no mortality impact of IPTi due to low coverage, late administration, drug resistance, decreased malaria transmission or improvements in vector control and case management. This study raises important questions for programme evaluation design.

Background

Malaria control has benefited from substantial financial investment and political commitment in the last decade, and impressive progress is being made in some settings. However, Plasmodium falciparum infection continues to cause hundreds of millions of clinical episodes, and hundreds of thousands of deaths every year. The greatest burden of malaria deaths still falls on very young children living in sub-Saharan Africa. IPTi involves the delivery of sulphadoxine-pyrimethamine (SP) to children when they receive routine vaccinations at about two, three and nine months of age. A pooled analysis of six randomized controlled trials reported that IPTi with SP can reduce clinical malaria episodes by 30%, decrease anaemia by 21% and admissions to hospital by 23% [1].

The study involved development and implementation of a strategy for IPTi in five remote, rural districts in southern Tanzania in order to assess the feasibility, acceptability, financial and economic costs of IPTi-SP when deployed as part of routine, programme-based activities within the given health system. The scale of the operation also afforded an opportunity to consolidate the safety profile of IPTi-SP. The project was conducted under the auspices of the IPTi Consortium, a group of research collaborations intent on generating an evidence base to inform a policy decision on the use of IPTi. The Consortium considered alternative approaches to evaluate the impact on survival of IPTi. However, it would not have been feasible to complete recruitment to an individually randomized controlled survival trial before a policy decision at the end of 2006. The best opportunity to estimate an effect on mortality thus fell on this five-district implementation study in southern Tanzania.

The aim of the project was to develop, implement and evaluate a strategy for the routine deployment of IPTi by the existing health services in Tanzania. Previous studies had evaluated the safety and efficacy of IPTi in relatively small-scale, individually randomized trials, where provision and administration of the intervention were taken care of by the research teams. Whereas the IPTi intervention is the delivery of SP to infants attending vaccination clinics, a number of elements are required in order to make IPTi available as a strategy integrated within routine preventive health services for infants. Mothers must be aware of IPTi and willing for their children to accept the intervention; health workers must accept and understand the intervention; IPTi needs to be available alongside vaccinations in health facilities; district council health management teams, who would be responsible for the IPTi programme, must be equipped with adequate and specific tools to manage the requisition and supply of IPTi-related materials and to train staff and raise community awareness. Going to scale with IPTi would also depend on effective IPTi-specific management systems at regional and national levels, and the longer-term availability and scaling-up would need sound financing strategies. These elements are prerequisites to high and sustainable coverage of IPTi in rural communities within a systems approach [2]. It is within this context that this study evaluated the effect of IPTi on survival.

The development of the strategy for the IPTi intervention, and specifically acceptability, and results on safety, costs and community effectiveness for malaria parasitaemia, anaemia, and clinical episodes are presented elsewhere [3–9]. This paper presents the evidence of the impact of IPTi on survival in children aged two-11 months from a cluster-randomized trial in 243,000 households.

Methods

Overview

After baseline household and health facility surveys in 2004 [7], the research team developed a strategy for IPTi implementation in partnership with the Ministry of Health during 2004-2005 [4, 6]. Twelve of 24 clusters were randomly assigned to receive IPTi, and implementation started in March 2005 led by routine health services with limited support from the research team. The trial profile is shown in Figure 1. In 2006, a follow-up health facility survey documented the availability of IPTi and staff trained in its delivery, together with an interim household survey to assess IPTi coverage and effects on the prevalence of malaria and anaemia [8]. In 2007, a large household survey which included the entire population was undertaken to assess the impact of IPTi on infant survival. The focus of this manuscript is the analysis of survival and includes process and contextual factors from household surveys to support that analysis. An intention-to-treat approach and an ecological design was used, with survival outcomes analysed according to the cluster in which the mothers lived and being unlinked to individual records of IPTi use.

Figure 1

Trial profile.

Study area

The study area covers five districts, split into 24 administrative divisions, in the Lindi and Mtwara regions of southern Tanzania. Lindi Rural, Ruangwa and Nachingwea districts are in Lindi Region and Newala and Tandahimba are in Mtwara region. The area has a total population of over 800,000 people and the highest reported infant mortality rates in the country [10, 11]. Relatively neglected for many years, the infrastructure is weak with poor access to water and electricity. Most people are small-scale subsistence farmers although the area produces most of Tanzania's cashew nuts as a cash crop through small-holder approaches.

At baseline in 2004 a representative sample of 21,482 households was identified using a modified EPI cluster sampling strategy, as reported in detail elsewhere [7]. In brief, 30 clusters of 30 households were identified in each division. Where written informed consent was given to interview, data were collected using Personal Digital Assistants (PDAs), enhancing data quality and timely availability of cleaned data [12] in anticipation of randomization of divisions to intervention or comparison groups. Birth histories were done in 19,008 (94%) of the 20,138 women aged 15-49 years reportedly resident in the households. For eight clusters in each division, additional information was collected on children aged under 2 years old about illnesses in the preceding 2 weeks, health-seeking behaviour and the prevalence of anaemia and parasitaemia [7, 8].

Randomization

The 24 divisions were arranged into three strata on the basis of mortality in children aged two-11 months, as measured in the baseline household surveys. Restricted randomization was used to allocate each of the 24 divisions into the IPTi or comparison group so as to assure adequate balance in terms of baseline mortality, overall population size and district health management team [13]. Details are given elsewhere [8]. Briefly, of 343,000 possible ways of allocating 12 divisions to the intervention arm (four within each of the three strata), 11,014 achieved "balance" with respect to mortality, population size and district, and one of these was chosen at random.

Implementation of the IPTi strategy

A detailed description of the development and implementation of the IPTi strategy is given elsewhere [4]. Following development of the IPTi strategy and piloting in two health facilities for a month, staff working in district and regional health teams trained front-line health workers how to implement IPTi in the first quarter of 2005. Subsequent follow-up by the project was limited to a single IPTi-specific health facility visit after training, followed by occasional visits to health facilities when the district health teams undertook routine supervision visits. All visits to health facilities were made in conjunction with members of the district health team. The IPTi-specific "follow-up after training" visits to each facility were completed before 24 May 2005: therefore, 24 May 2005 is used as the start date of the intervention.

Monitoring of acceptability, financial and economic costs, the impact on morbidity and the safety of IPTi was established and is reported elsewhere. Four in vivo anti-malarial drug efficacy tests were conducted (one in 2005 and three in 2006), following the standard WHO protocol with 42 days of follow-up. These studies showed 28 day PCR uncorrected adequate parasitological and clinical response rates of 62-63% in three sites and 49% in one site (data not shown), compared with 60% previously documented in the context of an earlier clinical trial which showed IPTi to be highly efficacious [14]. Molecular analysis of samples collected in household surveys showed a 44% frequency of the DHFR triple mutation in 2004, rising to 60% in 2007. The frequency of the 540E mutation (a marker of the quintuple mutation, DHFR triple and DHPS double) was 65% in 2004, 80% in 2006 and 68% in 2007 [unpublished observations].

Surveys in 2006, after a year's implementation of the IPTi strategy

Health facility and household surveys in 2006 showed that IPTi was available in 94% of intended health facilities and that there was evidence of a biological effect on malaria and anaemia in the target group of children aged two-11 months [8].

2007 household surveys

The primary aim of this final household survey was to estimate the effect of the IPTi strategy on mortality in children aged two-11 months. Secondary aims included (i) estimation of the coverage of IPTi in a sub-sample of households, and (ii) to assess whether there were any large imbalances between the IPTi and comparison groups in contextual factors including preventive health measures such as mosquito nets and vaccine coverage, nutritional status, case management and care-seeking for recent illness. Such factors were potential confounders of the effect of the IPTi strategy on infant survival.

Sample size

Mortality (main survey, all households)

The aim was to include all households in the area, with an expected total of roughly 250,000 and a range from 2,500 to 26,000 households in any one division. On the basis of the baseline survey in 2004, a two-11 month mortality rate of 36.2 per 1,000 person-years was expected, or approximately 463 deaths, in the comparison group, providing 80% power to detect an impact of IPTi of 30% on survival, equivalent to a reduction from 36 to 25 deaths per 1,000 child years [13].

Coverage (sub-sample of households)

In a clustered subsample of households this sub-study aimed to estimate coverage of IPTi of 50% to within 5% points, assuming a 95% confidence level, that 3.14% of the population is in this age group, an average of 100 households in each cluster, 3.88 people per household, and a design effect of two. Eight sub-villages (vitongoji) were selected from each division with equal probability and more children from larger vitongoji, thus assuring the sample of children was self-weighting within each division.

Field methods-main survey

Overview

Between 25 June and 30 October 2007, the main survey teams visited all households in the five districts and completed a household module. Women aged 13-49 years had a birth history module completed. Mothers of children < 2 years were asked if they were willing to be visited by the child health survey team. A random sample of those who were willing was selected and visited by these teams after the initial survey (see child health survey, below).

Survey tools

The household module included identifiers, household assets (socio-economic status markers), education and occupation of the household head, listing of all members of the household, and location of the household using a GPS. The birth history module, for women age 13-49 years, documented live births in the previous five years, whether the child was still alive, and dates of all events. For all live births in the 1 year before the survey, further questions related to use of antenatal clinics (ANC), mosquito nets, and IPT in the most recent pregnancy, place of childbirth, attendant, and essential newborn care indicators, for the most recent birth.

Community sensitization and preparation

A series of feedback and sensitization meetings was held in Mtwara and Lindi regions in March and April 2007. The aims were (1) to sensitize the ward councillors, ward executive officers and divisional secretaries about the survey and to seek their assistance in identifying candidates for training as field interviewers, and (2) to update the list of all sub-villages (vitongoji) and obtain estimates of the numbers of households in each sub-village. Widespread support for the project team was evidenced by the assistance of local civil administrators in developing a master list of 2,766 sub-villages comprising more than 250,000 households. This list was used to plan the logistics of interviewer training and the survey itself.

Pre-piloting

Two rounds of pre-piloting were done in a district adjacent to the project area in order to prepare a core team for the main survey training and to explore the feasibility of alternative approaches to the main survey, including estimates of the number of people to be interviewed per day, assessing the robustness of survey tools, development of Standard Operating Procedures for household listing, mapping and interviewing, etc. The survey tools built on experience with previous surveys in southern Tanzania which themselves had drawn heavily on standard MICS, DHS and malaria indicator surveys. The survey tools were developed into PDA-based questionnaires that were pilot tested and validated separately.

Training

The piloting activities were followed by a 1 week pre-training session for 14 facilitators. This ensured that a group of experienced interviewers was then available to support the main training, allowed further pre-testing of the PDA-based questionnaires and supported preparations for the organization of the bigger group.

A total of 243 candidate interviewers took part in a three-week training and evaluation course in Mtwara in June 2007. Most of the trainees came from the study districts and had 4 years of secondary education. They were trained to seek written informed consent, list members of the household, document household assets and recent deaths in the family before completing birth histories by direct questioning of women aged 13-49 years. Specific questions were asked about related issues, such as birth preparedness and care of the newborn. These are sensitive issues for most rural women and so training aimed to prepare interviewers to ask the questions with authority, respect and understanding.

Training was led by IPTi project core staff, supported by pre-trained, experienced field coordinators, and invited experts in fields such as maternal health and birth history studies. The training agenda included lectures, group discussions, field practicals and feedback sessions. During the course of training, potential team supervisors and mappers (whose job it would be to list and map all households in a sub-village) were identified and subjected to an additional training course consisting of management and quality control issues as well as field tests. Interviewers, mappers and supervisors were selected based on merit, and various techniques developed and used to assess performance during the course of training.

Logistics and timing

The survey staff were split into 22 teams arranged into four platoons, each led by a Commander (a senior, experienced field supervisor). The Commander was directly in charge of five or six survey teams, each consisting of a supervisor, a mapper and seven interviewers. Each platoon had a bus and two or three four-wheel drive vehicles. The Commanders were responsible for allocating each team to a sub-village and, between them, ensuring that all sub-villages on the master list were allocated to teams according to pre-determined criteria. This ensured a balanced workload between teams and a reasonable balance of teams working in each division to minimize any potential bias.

The main survey started on 25 June and lasted 128 days, finishing on 30 October. A support unit rotated between the platoons to ensure adequate supplies were maintained and to collect completed consent forms and backed-up data on CDs. These were all delivered to the project office in Mtwara where focussed data cleaning was performed before production of call lists for the child health survey team.

Field methods-child health survey

A simple random sub-sample of eight vitongoji per division (192 of the 2,621) was chosen for the child health survey. In selected vitongoji, all households with a child under 24 months, and whose mother had agreed in principle in the main survey to be re-visited, were approached for interview approximately 2 weeks after the main survey. The child health module included questions on IPTi and vaccine coverage, breastfeeding, Vitamin A supplementation, case management and care-seeking for any recent illness.

Twelve trainees were selected from the main survey group and trained from 28 June 2007 to 7 July 2007. The training comprised lectures, practical sessions, field practice on taking blood samples and interviewing, group discussions and feedback sessions. At the conclusion of training two teams of four interviewers, a driver and a supervisor started the survey itself. Call lists produced from the main survey data enabled identification of individuals for interview. Where consent was provided, mothers of all children under 2 years were interviewed. The availability of geo-coordinates from the main survey greatly enhanced the efficiency of this sub-study, which was completed on 16 November 2007.

Quality assurance

In both the main survey and the child health survey, several approaches to data quality assurance were used. Each day of the survey, for every team, these included random repeats and cross-checking of the mapping, accompanying one interview by one of the team, revisiting all households reported to be empty, and re-interviews repeating part of the questionnaire in a random sample of four interviews. Data were compared with the original interview and discrepancies discussed and resolved.

Data management

Data were entered into PDAs at the time of collection, using range, logical and internal consistency checks at the time of data entry. At the end of each day a manual summary of the day's activities was produced and compared with an electronic summary of the same information. When discrepancies arose, for example as the result of incorrect entry of a cluster or household number, the source of the discrepancy was identified and recorded in a data error log sheet and in supervisors' note books. These documents were used to guide cleaning of data in Access http://www.microsoft.com.

Statistical analysis

Statistical analysis was performed using STATA version 9 (College Station, Texas, USA), using an analytical plan which was reviewed by the Data and Safety Monitoring Board and agreed before the end of data collection.

Primary analysis

The primary analysis was based on a t-test of the (log) mortality rates in infants aged two-11 months in IPTi and comparison divisions, over the 2 year period between 24 May 2005 and the date of the survey visit in 2007. This approach was used, rather than individual-level regression modelling, partly because this is the recommended approach for studies with fewer than 15-20 clusters per treatment arm, and partly because there was no individual level data [11].

Secondary analysis

Restricted randomization ensured balance at baseline in infant survival, population size and district. Despite the use of restricted randomization, with only 24 clusters it was thought likely that the two groups would differ with regard to contextual factors related to infant survival, which would be potential confounders. For example, there were mosquito net campaigns in some parts of the area in 2005. As the use of treated nets is likely to be associated with better child survival, imbalances in net use would make this a potential confounder in the analysis of the effect of IPTi. Other potential confounders of the effect of IPTi, either a priori or from previous analysis, include socio-economic status, distance to the nearest health facility, maternal education, maternal age, twinning, ethnic group, and care-seeking for child illness. Baseline infant survival was also considered a potential confounder, despite the relative balance at baseline.

For each of these potential confounding factors direct or proxy measures were taken from study data, using the full survey where possible as this has the largest sample size. For example, treated net use in individuals is closely related to net ownership in households, but will always be lower than household net ownership. Data on net ownership was available from all 243,612 households visited, whereas data on use of treated nets was available only in the subsample of 2,953 children included in the child health survey (between 63 and 250 from each division). Household net ownership was used as a proxy measure of treated net use in children, because of the increased sample size with this approach.

Secondary analyses were planned to adjust for actual confounders from the above list. Confounding was investigated using Poisson regression analysis, and planned to adjust for any factor that changed the impact estimate by more than 15%. P-values for the adjusted impact estimate were calculated using a t-test, comparing the residuals (i.e. the ratio of observed and expected deaths) in intervention and comparison divisions from a model containing all confounders but not the effect of IPTi.

Exploratory dose response analysis

Using simple graphical methods, an exploratory ecological analysis of dose-response at division level was done. The 'dose' of IPTi was estimated using division-by-division coverage of IPTi by (1) the subsample of 1,731 children aged 12-23 months from the 2007 household survey and (2) routine EPI reports from each health facility as a proxy for the dose of IPTi [A Bush et al., manuscript in preparation]. The 'response' to IPTi was estimated through the division-by-division change in mortality in infants aged two-11 months between 2001-4 and 2005-7, using the 2004 and 2007 household surveys respectively.

Ethical issues

The study was undertaken within the framework of the assessment of the community effectiveness of IPTi, part of the IPTi Consortium http://www.ipti-malaria.org. The study had ethical approval from local and national institutional review boards (Ifakara Health Institute, formerly Ifakara Health Research and Development Centre, Ifakara, and the National Tanzania Medical Research Co-coordinating Committee) through the Tanzania Commission for Science and Technology. Ethical and research clearance was also obtained from the institutional review board of the London School of Hygiene and Tropical Medicine, UK, and from the Ethics Commission of the Cantons of Basel-Stadt and Basel-Land, Switzerland. The project team in Tanzania presented the design, aims, objectives and updates of the pilot IPTi implementation project to District Councillors, who are elected representatives of the community, and benefited from their support of the intervention and its evaluation. During field work, information sheets in Swahili about the study were given out, explaining why it was being done, by whom, and what it would involve. In the household survey, written consent of all household heads was sought. In addition, women aged 13-49 years were asked to give verbal consent before the birth history interview. The trial is registered on clinical trials.gov, number NCT00152204.

Results

Completeness of survey data

The 2007 survey identified 243,612 households in 2,621 vitongoji (sub-villages) (Table 1). The household head was present in 227,789 (94%) and of these 99% agreed to take part. A total population of 818,695 was recorded in these households, with 213,233 women aged 13-49 years. Ninety-one percent of these women were interviewed. Completeness of the information was similar to the baseline survey in 2004.

Table 1 Study population and completeness of information, 2004 and 2007 surveys

For the subsample of households selected for the child health survey, a total of 2,907 households identified in the main survey as having children under 2 years were visited. In these households, 3,024 children aged under 2 years were included, of whom 2,953 (98%) had information on use of treated mosquito nets. Information on sickness in the 2 weeks preceding the survey was available in 3,006 children, of whom 1,432 (48%) had been ill. Of these, 40% (576/1432) had sought care from the formal health sector, among whom 96% (547/576) had information on care-seeking.

Comparability of intervention and comparison groups

Completeness and participation were similar in intervention and comparison groups, with 94% of household heads present and 99% agreeing to participate in both of the two study arms (Table 2). The Makonde ethnic group were more common in comparison divisions (62%) than in intervention divisions (46%), but the difference did not reach statistical significance (P = 0.15, comparing mean proportions in 12 intervention and 12 comparison divisions). The two groups were similar with regard to socio-economic status, mosquito net ownership, maternal education, maternal age, net use in pregnancy, antenatal attendance, and home births.

Table 2 Context and comparability of intervention and comparison groups, 2007 survey

Geographic data were incomplete for both groups due to logistic problems when transferring GPS data to laptops, but by chance a smaller proportion were missing for the intervention group (23%) than for the comparison group (28%). For the subgroup with complete data, 28% of the intervention group lived within 5 km of the nearest health facility, and 23% of the comparison group (P = 0.19, comparing mean proportions in 12 intervention and 12 comparison divisions).

For coverage and other child health indicators from the sub-study, the two groups were similar with regard to exclusive breastfeeding in infants aged nought to 5 months and the use of mosquito nets. Care-seeking for a sick infant from a formal health care provider was less common among the intervention group than the comparison group (37% vs 43%, P = 0.04, comparing mean proportions in 12 intervention and 12 comparison divisions).

Coverage of IPTi and corresponding vaccines (and comparison with previous estimate, from 2006 surveys)

IPTi is given alongside DPT and measles vaccines, and coverage of these vaccines was similar in IPTi and comparison areas (Table 3). DPT vaccine coverage was estimated at 86% in both intervention and comparison groups (P = 0.44) and measles vaccine coverage at 78% in the intervention group and 80% in the comparison group (P = 0.79).

Table 3 Coverage of IPTi and corresponding vaccines, 2007 survey

IPTi coverage was estimated in two different ways. Using a conservative definition in children aged 12-23 months of three doses before 12 months of age, as documented on the health card, IPTi coverage was 48% in the intervention divisions and 2% in comparison divisions (P < 0.0001). Using a more lenient definition in children aged 12-23 months of three doses either documented on the health card or reported by the mother, coverage was estimated at 77% in intervention divisions and 6% in comparison divisions (P < 0.0001).

Mortality

There was a dramatic fall in overall mortality in both intervention and comparison divisions between baseline (2001-4) and the final survey (2005-7). The improvement in survival was comparable in IPTi and comparison areas: rates in infants aged two-11 months fell from 34.1 to 23.6 per 1,000 person-years in intervention areas and from 32.3 to 20.7 in comparison areas (Figure 2).

Figure 2

Mortality in infants aged two-11 months.

In 2005-7, divisions implementing IPTi had a 14% (95% CI -12%, 49%) higher mortality rate in two-11 month olds in comparison with non-implementing divisions. The difference was not statistically significant (P = 0.31) (Table 4). There was no evidence of confounding.

Table 4 Survival impact: mortality in infants aged 2-11 months, 2004 and 2007 surveys

Dose-response analysis

The mortality rate change in the 12 implementing divisions was plotted against the coverage of IPTi as estimated by the 2007 survey. There was no evidence of an association between coverage of IPTi at the division level and survival of children aged two-11 months (Figure 3).

Figure 3

Mortality change in infants aged two-11 months by IPTi coverage, intervention divisions.

Discussion

To the best of our knowledge, this is the first study to have attempted to estimate the effect of IPTi on infant survival. Despite the large scale, the findings are compatible both with a reduction and an increase in mortality associated with IPTi. This may reflect a true lack of effect of IPTi on survival, or the findings may be a false negative result.

A number of design issues need to be considered when considering these results. The most dramatic finding was the marked improvement in survival between 2004 and 2007. In this period, mortality dropped by about one third in both intervention and comparison arms, causing a reduction in the power of the study to find a statistically significant difference between the group. However power also depends on the variability in mortality rates between clusters, and this was also lower than anticipated. Based on the 2007 results, the study had over 80% power to find a 30% difference between the groups as statistically significant, but under 20% power to find a difference of 12% as statistically significant at the 5% level, with 12% being the upper end of the 95% confidence interval for the effect of IPTi on survival found in this study.

A second methodological challenge arises from the relatively small number (24) of intervention and comparison units. Despite the overall rapid change in survival, mortality rates may not change at the same pace everywhere, and there is a need to balance the rate of change between intervention and comparison areas. The best way to achieve this would be to increase the number of intervention units. The chosen unit was the division, a sub-district administrative area, which is reflected in the organization of health services. Operating at this level reflected a compromise between the need to work with functional units of the health system with the need for replication of intervention and comparison units. The study was designed with the primary aim of assessing the feasibility of deploying IPTi: could the IPTi strategy be successfully implemented through the existing public health system in an under-resourced part of a poor country? With minimal inputs from the research team, could IPTi coverage reach or even exceed that of the corresponding EPI vaccines? The estimate of mortality impact was a secondary aim and, despite restriction of randomization at baseline to ensure balance in terms of infant mortality, population size and district location, it is possible that unmeasured-and so far unknown to the study team-but important influences on survival were not balanced between arms. Although there was an imbalance between the arms with regard to the completeness of geographic data, this imbalance seems unlikely to affect the conclusions of the study,

Is it possible that IPTi did not improve survival in southern Tanzania? The current study included household surveys in 2006 which suggested a modest impact of IPTi on malaria parasitaemia (from 38% to 31%, p = 0.06) and anaemia (from 88% to 80%, p = 0.02), despite coverage which was lower than anticipated [8]. Morbidity monitoring of malaria in 11 sentinel health facilities suggested that IPTi was associated with a reduction in clinical malaria of between 18% and 54%, depending on the scope of the analysis, although no differences reached statistical significance [9]. A recent pooled analysis of the efficacy of IPTi in six individually randomized studies in infants from Tanzania, Mozambique, Ghana and Gabon concluded that IPTi prevented 30% of clinical malaria episodes (95% CI 20-39) and 23% of all-cause hospital admissions (95%CI 10-34) [1]. In addition, other malaria control tools are associated with a larger impact on survival that can be explained by direct malaria deaths alone [15, 16]. Given this, it seems unlikely with current knowledge that IPTi has no impact on infant survival, despite the null estimate of effect. Short-term efficacy of IPTi-SP has been documented in a seventh randomized controlled trial of IPTi-SP, published at the same time as the pooled analysis, and conducted in a part of northern Tanzania with very high levels of resistance to SP. Although there was no overall efficacy of IPTi in this setting, there was good evidence of protection in the month following IPTi doses [17]. Resistance was rising during the time of our pilot implementation and it is possible that this undermined the effect of IPTi.

The coverage of IPTi was lower than expected and may not have been sufficient to result in a measurable impact on mortality. Based on the results from the phase-III studies [1], the authors consider it likely that at least two timely doses of IPTi are needed in order to have an impact on mortality. Coverage of two doses, using both the information recorded on child health cards and from asking the mother or guardian, was 82% (95% CI 76-88%) in the intervention group and 14% (95% CI 9-19%) in the comparison group. Using a more conservative approach of three doses written on the health card, and assuming children without a health card had not received IPTi, coverage was 48% (95% CI 40-56%) and 2% (95% CI 0-6%) in the comparison group. Leakage--whereby comparison group children received IPTi--therefore appears to have been minimal. However, there were considerable 'missed opportunities' for IPTi, where children received DPT or measles vaccines but not IPTi. This was due to a combination of reasons such as a reluctance of some health staff to offer IPTi, stock outs of SP, DTP or measles vaccines, and recent history of treatment with SP, rather than lack of acceptability to mothers and families [5, 8]. In addition there is evidence from qualitative work that some health cards were falsely completed, both for vaccines and for IPTi [18]. If less than half of eligible children received a sufficient dose of IPTi, and those who did receive the intervention were at least risk of malaria, it is possible that coverage was simply too low for any impact on survival to have been measurable in the given evaluation.

There was a marked reduction in the prevalence of P. falciparum infection in the study area during the project. Parasite prevalence in infants aged two-11 months in the comparison areas (without IPTi) dropped from 57% to 38% between 2004 and 2006, and to 19% in 2007. The predominant reasons are the subject of ongoing analysis but treated net use in infants aged two-11 months in the comparison areas increased from 11% to 38% over the same time frame. Individual analysis of risk factors for death in infants suggests that the single biggest reason for the rapid drop in mortality was this rise in the use of mosquito nets (unpublished observations). Furthermore, the first-line treatment for malaria changed in 2006 from SP to artmether-lumfantrine (ALu). To the best of the study team's knowledge there were no other major malaria prevention or treatment programmes ongoing in the area. The increase in net use and the change of first-line drug may have reduced malaria transmission to such low levels that IPTi had no effect. However, this seems unlikely as the individually randomized studies of IPTi were done in settings where incidence rates of malaria ranges from 0.16 to 1.27 episodes per child per year and net use varied from 0% to 67% yet notable protective efficacy was described in these studies [11].

IPTi has recently been recommended by WHO as a new malaria control tool for use in settings with moderate to high malaria transmission and where the frequency of the DHPS 540, a marker of the quintuple mutation, is less than 50%. The evidence showing a benefit of IPTi in such settings is overwhelming, and much stronger than for many routinely implemented interventions. The results of the current study were considered as part of the WHO policy review process: the methodological limitations of the study were recognized, but the feasibility of IPTi implementation with available resources confirmed.

At the start of this study a model of impact was outlined, showing the steps that were thought would lead to mortality impact in this effectiveness setting (Figure 4). Firstly, trained health workers and IPTi should be available in health facilities; secondly these should be offered to children attending for EPI vaccines; and thirdly a high proportion of mothers should bring their children for EPI vaccines. Data from our 2006 health facility and household surveys suggest that the first and third steps were reasonably well assured, but that roughly one-third of children attending for EPI vaccines were not offered IPTi due to demand and supply side barriers, all of which had potential to be addressed [8, 18]. The fourth step was that IPTi would be well accepted, and the data suggest that this was indeed the case [5]. The fifth step was that there should be low resistance to SP. Although in vivo efficacy was similar at the outset to that seen in settings where IPTi had previously been shown to be efficacious, resistance increased during the project, with the frequency of the 540E mutation at 68% in 2007. There are few data points from other studies in higher resistance settings. In an area with a background frequency of 56% for this mutation [19], IPTi provided 21% protective efficacy against clinical malaria [20], whereas no measurable protection against malaria was conferred by SP in an area where the 540E mutation frequency was 90% [21].

Figure 4

Model of impact for IPTi.

The last step in the model was that there should be some evidence of reduced malaria morbidity, which was indeed the case in the study [8, 9]. Taking all these elements together, it seems likely that a combination of (i) lack of power combined with (ii) suboptimal coverage and (iii) increasing levels of drug resistance undermined this attempt to estimate the impact of IPTi on child survival.

Conclusions

The findings of this large-scale randomized controlled study are compatible both with a reduction and an increase in mortality associated with IPTi. The lack of evidence of an effect of IPTi on survival could be a false negative result due to a lack of power or imbalance of unmeasured confounders. Alternatively, there could be no mortality impact of IPTi due to low coverage, late administration, drug resistance, decreased malaria transmission or improvements in vector control and case management.

A cluster-randomized trial design was chosen in order to estimate the impact of the intervention under conditions that were close to 'real-life' in a large rural area with a population of almost one million people. This choice meant randomizing a relatively small number of clusters, collecting ecological rather than individual data on IPTi use, and evaluating impact with coverage rather lower than had been hoped. The implications of the findings for future large-scale impact evaluations are sobering: despite a 'probability' design it is not possible to draw a firm conclusion about the impact of IPTi on infant survival. Another recent large-scale impact evaluation faced similar issues [22]. Evidence of the impact on mortality of interventions, as deployed in real life, is likely to require a different type of inference from that used in individually randomized controlled trials. Intervention-and context-specific models of impact need to be developed before systematic, prospective collection of quantitative and qualitative data enables a thorough process evaluation as an integral part of the evaluation. The plausibility, rather than the probability, that a change in survival is the result of the intervention can then be assessed.

Financial support

This study received funding from the Bill and Melinda Gates Foundation through the Intermittent Preventive Treatment in infants (IPTi) consortium http://www.ipti-malaria.org. JAS received funding from the Bill and Melinda Gates Foundation through the Gates Malaria Partnership.