Study area
The study was carried out between 2000 and 2005 in Kyenjojo District in west Uganda, a hilly area with altitudes between 1,350 and 1,550 meters. Annual rainfall ranges from 1,200 to 1,600 mm, mean temperature is 21.3°C, and average relative humidity about 70%. Malaria is meso- to hyperendemic with Plasmodium falciparum prevalence rates in asymptomatic children age 2–9 years between 45 % and 68% [15, 16]. Main vectors are Anopheles gambiae sensu strictu and An. funestus with Entomological Inoculation Rates (EIR) estimated around 7 infective bites per person per year [17]. Study site was in Kirongo Parish, Nyantungo Subcounty and involved five villages: Bucuni, Bwendero, Kasunga, Kidomi and Kyakahuli. These had previously participated in a study on insecticide treated curtains showing excellent cooperation. In addition, village health workers with intensive experience in surveys and field work were available.
Study design and sample size
This was a prospective study with mosquito nets as the unit of observation. Two types of nets, LLIN and ITN, i.e. conventionally treated nets, were randomly distributed to households with users as well as field staff initially blinded with respect to the type of net. In intervals of 6 or 12 months a sample of nets was randomly selected out of the pool of study nets for testing with bioassay (mortality and knockdown rate of mosquitoes) and chemical residue as the principle outcome measures. Two distinct phases of the study can be distinguished each testing a different product of LLIN and comparing it to ITN. Time of follow-up for LLIN was 39 and 36 months respectively and that for ITN was 12 months.
The necessary sample size, i.e. number of nets to be sampled at each time point was calculated based on the main outcome variables. It was found that a sample of 40 nets for each type (LLIN and ITN) would be sufficient. Assuming one measurement for each of the nets per time-point, an alpha error of 0.05, power of 80%, a standard deviation of 8.0 (taken from previous deltamethrin studies) this sample size was sufficient to detect a decline of 8 mg/m2 or more between time points as statistically significant. Similarly, it allows the detection of a difference of at least 15%-points between time points or type of nets in vector knockdown or mortality rates as significant.
While this study was designed before the publication of the WHOPES guidelines for phase III field testing of LLIN [18] its approach is in keeping with these recommendations.
Tested products and net treatments
Two versions of the LLIN PermaNet® (Vestergaard-Frandsen) were tested. The first generation is a multifilament polyester net treated with a target dose of 50 mg/m2 of deltamethrin using a coating technology to enable wash resistance and create a reservoir of insecticide. The first generation test nets were part of a shipment of 10,000 nets delivered to the Commercial Market Strategies (CMS) project in Kampala in November 2000 for social marketing and were produced in September 2000. Nets were stored in the CMS warehouse in bundles of 50 nets. Nets for the study were identified at random by first selecting 46 bundles from the shelves in the warehouse and then from each bundle 10 nets giving a total of 460 LLIN.
For the second generation PermaNet® no large scale shipment was available and 270 test nets were directly received from Vestergaard-Frandsen in August 2002. These multi-filament polyester nets were part of routine production with a target dose of 55 mg/m2 of deltamethrin. At the time of production (July 2002) the company did not yet use batch numbers on the labels so these are not available.
ITN for phase one (test of first generation PermaNet®) were nets identical to the LLIN with respect to netting material but were sent from the factory untreated. The treatment of these 150 nets was done in the CMS warehouse in Kampala using a 1% suspension concentrate of deltamethrin (K-Othrine, Bayer Environmental Science), one of the WHOPES recommended insecticides for net treatment [19]. Based on a surface area of the nets of 13.1 m2 and a target dose of 25 mg/m2 33 ml of insecticide solution and 400 ml of water were used per net. A solution was prepared in a bucket for two nets at a time and nets soaked for two to three minutes under continuous kneading. After excess water had dripped off nets were dried flat on plastic sheeting in the shade of the warehouse.
For the second phase of the study (test of second generation PermaNet®) the ITNs comprised one group of the original nets re-treated after 15–18 months of field use and one group of new, multifilament polyester nets (Siamdutch Netting Company, Thailand) which were exchanged for other conventionally treated nets. Net treatments were done in the field using the wetable tablet version of the same WHOPES recommended deltamethrin [19] (KO-Tab, Bayer Environmental Science) containing 360 mg deltamethrin. Nets were treated by experienced and supervised field staff using basins and 400 ml of water and 1 insecticide tablet according to manufacturer's instructions. Nets were dried flat on the grass and as much as possible in the shade.
All nets, LLIN and ITN, were white, rectangular nets of 75 denier and a size of 160 × 150 × 180 cm (width, height, length).
Field procedures
From each type of net (first and second generation LLIN, ITN) 10 randomly selected nets were kept for baseline analysis while the remaining nets, 450 first generation LLIN, 140 ITN and 260 second generation LLIN were prepared for distribution to households. For the first phase of the study all existing labels on the nets were removed in order to allow blinding and an identification number, printed with wash resistant ink on a piece of polyester band, stitched on the net. The numbers had been previously randomly allocated to the two groups (LLIN and ITN) so that no identification of net type was possible purely by the ID number. After the ID number labels had been fixed to the nets they were re-sorted by ID number thereby mixing the nets at random. Based on a household list from the five villages net numbers were randomly allocated to households according to available sleeping places to ensure equitable distribution. No specific instructions regarding use or washing were given to the net users.
The procedure for second generation PermaNet® was similar with the difference that labels were not removed since at the time only LLIN were distributed. Random allocation to households was done based on "vacancies" according to the master list of nets which was kept by the principle investigator and recorded the status of each net (type, ID number, household number, time of distribution, time of sampling, replacement or removal).
Jointly with the initial net distribution a household survey of all 294 participating households was carried out in December 2000 by experienced field staff following one day of training on the specific questionnaire. In addition to the number of persons and children living in the household the pre-coded questionnaire included information on education and occupation of head of household, physical condition of the house (roof, windows, wall materials, and eaves), family assets and ownership of animals and land.
Between 11 and 15 days after the net distribution a random sample of 116 households was re-visited and a short questionnaire designed to capture any adverse effects of the insecticide on the net users, their duration and severity.
Throughout the study net surveys were carried out among households to collect data from all active study nets on net use (who and frequency of use), perception of net effect, washing habits (method of washing, type of water and soap used) and number of washes since last survey. The physical condition of each study net was assessed with respect to where the net was found (hanging or not) and number, size and position of holes. Three categories of hole sizes were used: up to the diameter or length of a coin, hand width or larger than a hand width. To mark the position of a hole the net was divided into numbered areas (upper and lower part of each side) and the code for the hole's location entered into the questionnaire. A detailed list of the time intervals of the surveys and the number of nets seen is given in the annex [see Additional file 1]. A total of 13 surveys were carried out for phase one nets and seven for phase two nets. For logistic and operational reasons the frequency of surveys per year reduced over time from four in the first, three each in the second and third and two each in the fourth and fifth. During the course of the study the questionnaire for net follow-up was simplified as the information obtained did not change over time. The first change was made in April 2002 (survey 5) when assessment of the hole's position and the information on who used the net were ended and the second in October 2004 (survey 11) when questions on the method of washing and net perception were ended.
Sampling and sample preparation
For each time point a random sample of 40 nets per net type were selected from the net master list together with 2 possible replacements in case the selected households could not be reached or the net had been lost to follow-up since the last visit. Nets were then collected by the field team and each household received a LLIN as a replacement to insure continuous protection of the family. These replacements nets, however, did not have ID numbers and were not included in net follow-up surveys. Details of collected samples at various time points are presented in the annex [see Additional file 2].
Using templates of 30 × 30 cm (bioassay) and 10 × 10 cm (chemical residue) samples were cut out of the nets, marked with the date and the net ID number, packed in aluminium foil and stored at room temperature until transport to the respective laboratories for analysis. Samples for bioassay and chemical residue were always taken from the same spot (long side near the ID number mid-way between top and bottom) immediately next to each other. Generally one sample was taken per net per time point with the following exceptions in order to allow assessment of intra- and inter-net variability. At baseline and 6 months of the first phase a second sample was taken from the short side of the net for bioassay and chemical residue. These locations were termed sites one (standard sample) and two respectively. At base line of the first phase two additional samples for chemical residue were taken from each site immediately next to each other and these were termed position one and two respectively. Therefore, there were 2 samples (site 1 and 2) per net for bioassay at baseline and 6 months while for chemical residue there were 4 samples at baseline (position 1 and 2 each for site 1 and 2) and 2 at 6 months (site 1 and 2). For the second phase additional samples were only taken at baseline and only for chemical residue (site 1 and 2).
After the first samples had been taken the ten baseline nets for the first generation LLIN were kept outside the package exposed to air and dust but not sun and were not used or washed. Samples from these nets were taken 11, 27, 39 and 60 months after the unpacking and sent for chemical and bioassay analysis.
Chemical residue
All samples were analysed at the laboratory of the Pesticides Research Department of the Walloon Agricultural Research Centre in Gembloux, Belgium (WHO Collaborating Centre) using the MEREPERMA methodology which has been ISO accredited (ISO 17025). Surface area and weight of each 10 × 10 cm sample was measured and the sample then introduced into a 100 mL Erlenmeyer flask. Deltamethrin was extracted from the sample by heating under reflux for 60 minutes with 40 mL xylene. After cooling to ambient temperature the extract was quantitatively transferred into a 50 mL volumetric flask. The flask was filled up to volume with xylene. A 10 times dilution was achieved in xylene. The final extract was then analysed for determination of deltamethrin by Capillary Gas Chromatography with 63Ni Electron Capture Detection (GC-ECD) using an external standard calibration. For each sample two chromatographic injections were performed and the mean reported as g/kg deltamethrin and then transformed to mg/m2 based on the surface area of that sample. The analytical method was validated for the determination of deltamethrin residues in conventionally and long lasting treated nets. Specificity, repeatability (precision), linearity of the detector response, recoveries (accuracy) and limit of quantification were determined. The accuracy of the method was determined concurrently with the analysis of samples from 2001 to 2005 by spiking untreated mosquito net samples (which had already been extracted) with know amounts of deltamethrin. The mean recovery varied between 95% and 101 % depending on level of deltamethrin concentration (n = 242) with a Relative Standard Deviation (RSD) between 7% and 11% for fortification levels ranging from 0.3 mg/m2 to 100 mg/m2. The acceptable limit is 90–110 % with a RSD < 15 %. Therefore, the accuracy and precision of the analytical method was found to be excellent.
A total of 115 samples from both first and second phase of the study which had been taken immediately next to the sample for GC-ECD analysis were also analysed at the Vestergaard-Frandsen Quality Control Laboratories in Hanoi, Vietnam using a method where insecticide determination is done by normal phase High Performance Liquid Chromatography with UV Diode Array Detection (HPLC-DAD) using an internal standard. The principle of this method was proposed in 2006 for adoption by the Collaborative International Pesticides Analytical Council (CIPAC). In brief, net samples are cut into small pieces of < 2 × 2 cm and deltamethrin is extracted into solution using a mixture of solvents iso-octan plus 1,4 dioxan with 0.15% HPLC grade water (80/20, v/v). Dibutyl phthalate is added as the internal standard. The extraction bottle is sonicated in a water bath set at 80°C and then shaken vigorously for at least 15 minutes. A proper volume of solution is filtered through 0.45 micrometer membrane syringe filter into a vial. A volume of 5 μL of filtered solution is injected into a normal phase isocratic HPLC equipped with PDA/UV detector and deltamethrin is quantified using an internal standard calibration curve. The method was shown to be suitable for deltamethrin with repeatability (same net sample) of 1.8% (RSD, n = 7), reproducibility (multiple samples over time) of 11.6% (RSD) and recovery of 99.7% (95% CI 98.6%–101.6%) from samples in which deltamethrin content was added at an exact amount by weighting method (n = 5).
Bioassays
Bioassays for the first study phase were carried out by the Laboratoire de Lutte contre les Insectes Nuisible, Montpellier, France (WHO Collaborating Centre) using WHO standardized procedures [20]. For the tests 2–4 day old, unfed female Anopheles gambiae s.s.(Kisumu strain) and Culex quinquefasciatus (S-lab strain) were used, the latter were only tested for the first 12 months. Both species have been well established in culture and are known to be pyrethroid sensitive. The tests were conducted using the standard WHO plastic cones and a three minute exposure time. Five mosquitoes were introduced into cones at a time. Tests were carried out at 25°C ± 2 under subdued light. After exposure, females were grouped into batches of 10 or 20 in 200 mL plastic cups and maintained at 28°C ± 2 and 80% ± 10% relative humidity with honey solution provided. For each sample tested, a total of 50 mosquitoes were used (Inter Quartile Range 50–51, range 40–62, ten cones). Proportion of mosquitoes knocked down at 60 minutes (KD60) was calculated. Percentage mortalities were recorded after 24 h.
Bioassays for the second phase of the study were carried out at the entomology laboratories at Centers for Disease Control (CDC), Atlanta, USA (WHO Collaborating Centre). Tests were carried out with Anopheles gambiae s.s. Exposure time and method were identical, however, only 40 mosquitoes were used per test.
For all bioassays unexposed controls were run to validate the tests results and results were discarded if mortality among control was > 5%.
The definitions of effectiveness of nets based on bioassay results followed recommendations by WHO (Pierre Guillet, personal communication) and were as follows:
Minimal effectiveness: KD60 ≥ 75% or functional mortality ≥ 50%
Optimal effectiveness: KD60 ≥ 95% or functional mortality ≥ 80%
Data analysis
Data were entered using EpiInfo 6.04 software (WHO/CDC, 1997) and then transferred to Stata 8.2 (Stata Corporation, Texas, USA, 2005) for further data management and analysis.
From the household data a socio-economic index was calculated using principal component analysis considering education, ratio household members per bed, physical condition of house, assets (radio, vehicles), animals and land possession. Only the first component was used to build the index. Households were then divided into wealth quintiles for further analysis.
After appropriate data preparation cumulative washes per net were calculated until the net was censured, i.e. sampled or otherwise lost to follow-up. Similarly, a hole index was calculated for each net and time point which was constructed by multiplying the number of holes with the hole category (1–3 and increasing with size) and then calculating a mean over all nets in the sample, including those with no holes.
For the analysis of net performance (chemical residue, bioassay) the mean of all samples per net was calculated. For the expression of the central tendency of sample measurements of all outcome variables mean, geometric mean or median was chosen after evaluation of the distribution of values within the sample.
For the assessment of between-net variability of insecticide concentration the standard deviation of deltamethrin residue was expressed as percent of the sample mean (coefficient of variation). Intra-net variability was expressed as the difference of samples of the same net to the mean expressed as percent of the mean and then averaged over the sample. For the analysis of statistical differences of inter-intra net variations of outcomes ANOVA was used.
Statistical analysis was generally done in two steps, first univariate analysis was carried out considering all co-variables of interest and tested using Chi-squared test for categorical and t-test or Kruskal-Wallis test for continuous variables depending on the validity of the assumption of normal distribution of values. In a second step multivariate analysis (linear or logistic regression models as appropriate) was used to verify any associations found in the univariate analysis.
Ethical considerations and approval
This study was conducted according to the principles of the Declaration of Helsinki and the international guidelines of biomedical research involving human subjects. It was reviewed and approved by the Ministry of Health, Uganda, WHO Roll Back Malaria Project, Geneva and Gesellschaft für Technische Zusammenarbeit, GTZ, Germany.