Long-term field performance of a polyester-based long-lasting insecticidal mosquito net in rural Uganda

In recent years the use of insecticide treated mosquito nets (ITN) and curtains has been established as one of the key measures of malaria prevention in sub-Sahara Africa with proven efficacy and effectiveness [1]. Attempts are now under way to go to scale with this intervention by involving all possible stakeholders and creating the maximum synergy effects through public-private partnerships. One of the most important obstacles to wide-spread coverage with ITN has been, however, the need for regular re-treatment with insecticide every 6–12 months. Up to date re-treatment rates range from 2% to 20% in most instances and rarely reach more than 40% unless re-treatment is done by the public health services without cost to the consumer [2, 3]. The solution to this problem and a true breakthrough in the ITN application is the concept of a long-lasting insecticidal net (LLIN), i.e. a net on which the insecticide effect lasts at least as long as the average useful live of that net even when it is used and washed regularly [4]. Early attempts of increasing wash resistance of permethrin by adding polystyrene to the emulsifiable concentrate [5] showed a significantly prolonged effect but it was not long enough to last for the life of a net.

The new technologies for LLIN can be divided into two types [6]. The first, called incorporation technology, uses polyethylene as material where it is possible to directly incorporate a pyrethroid insecticide into the material from which then the netting is made. Facilitated by certain reagents the insecticide will migrate to the surface of the fibre and will be regenerated from the reservoir after the surface insecticide is washed off or otherwise lost. The second type is based on multifilament polyester as the netting material and here a resin based polymer coating is used as the insecticide reservoir for replacement of surface insecticide and this coating is bound to the surface of the filament. This is called the coating technology.

The first long-lasting insecticidal net (LLIN) based on polyethylene incorporation technology is the Olyset Net^® (Sumika Life-Tech Co, Osaka, Japan), pre-treated with permethrin. Effectiveness over at least three years has been shown to be good [7] and based on available evidence this LLIN was recommended for malaria prevention by the WHO Pesticide Evaluation Scheme (WHOPES) in 2001 [8].

Another LLIN based on the coating technology using deltamethrin as insecticide has been developed by Vestergaard Frandsen Disease Control Textiles (VF, Lusanne, Switzerland). This product has been branded PermaNet^®. Initial field studies showed that these nets had a good level of wash resistance [9]. However, with respect to actual duration of insecticide protection in the field initial data from Burkina Faso seemed to indicate that performance was not quite as good as anticipated although sample sizes for this study were rather small [10].

As a reaction to initial results of varying product performance, the manufacturer reviewed its production processes and reports to have significantly stabilized and improved the manufacturing process. The resulting second generation product was finalized in August 2002 and officially launched in April 2003. It is called PermaNet 2.0^® referring to the original product as first generation.

Recent results with later versions of this product [11–13] suggest that indeed the initial problems have been overcome and based on the phase I and II laboratory wash data and experimental hut trials PermaNet 2.0^® received a preliminary WHOPES recommendation in 2004 [14]

This study reports on the long-term field performance of the initial as well as the improved, second generation product of PermaNet^® as part of the phase III WHOPES evaluation process.

Two possible approaches can be taken to study the field performance of a LLIN product under "real life" conditions. First, a cohort design can be used where a number of nets are randomly distributed to households and repeated measurements taken from the same net over time until a defined endpoint is reached. This design, which was applied by Lindblade and colleagues in Kenya [12], has the advantage that it follows the same net over time but the disadvantage that invasive measurements that require cutting of samples are limited to time of failure and bioassays have to be done on-site requiring an insectary within the study area. The second design option uses multiple random samples of nets with only one measurement per net. This approach which needs a larger number of nets but has no restrictions with respect to number of samples cut from each net for chemical and bioassay tests, is currently recommended by WHOPES [18] and has been applied in this study. By selecting the first generation PermaNet^® for the study randomly from a large batch of LLIN in the warehouse and distributing them randomly among households each random, cross-sectional sample of nets is representative of the original shipment of 10,000 nets to Uganda. In case of the second generation nets there was no large shipment to select from so that each time point represents the overall shipment from the factory. With the exception of the baseline nets only one sample per net was taken in a standardized fashion, i.e. from the same location on the net. This implies that the mean for all sampled nets per time point can be interpreted as a valid estimate of chemical residue, vector mortality or knockdown rate, while the result of an individual net sample has to be interpreted with caution as it will depend on the level of within-net variation of the insecticide and can not be taken as the true average of that net. To obtain an reliable estimate for a specific net a composite sample of at least 5 locations on the net has to be taken as described in the WHO specifications for deltamethrin LN [21] or better 12–14 as is practiced by the VF quality control laboratories (Phan, personal communication).

In the first generation PermaNet^® an unexpected drop in performance was observed after only 6 months at which time the rates for the outcome variables were not significantly different from conventionally treated ITN. However, thereafter chemical residue as well as bioassay results remained quite stable over more than 20 months with 15–45% of nets still showing optimal effectiveness. Even after 39 months of field use 5% of the net samples still had more than 4 mg/m² deltamethrin and 18% showed either a mosquito knockdown rate of ≥ 75% or mortality of ≥ 50% (minimal effectiveness). While this is not sufficient to fulfil WHOPES phase III criteria [18], it is clearly more than what can be expected from conventionally treated nets which have been reported to reach good performance up to 15 months [22] but never up to 27 months. The cause for the observed pattern of performance can not be explained by any of the variables in the data set, however, some potential causes can be excluded: excessive washing, use of more aggressive (alkaline) soaps and also the impact of sun light as none of the samples showed the R-isomer of deltamethrin typical for UV exposure (data not shown). The fact that these nets did not show any loss of activity even after 5 years when they were not used or washed (Table 5) and the bimodal distribution of results at the time points between 6 and 27 months suggest that a certain proportion of nets did loose the coating with the insecticide quickly under the stress of every day use and/or washing but the remaining 20% to 30% of nets continued to perform as LLIN at least up to 27 months and some up to 39 months. This explanation would also be consistent with the statement of the manufacturer that until early 2001 some problems in the manufacturing process existed that could have resulted in parts of a production batch having a lower quality and that between June 2000 and August 2002 continuous improvements on the production quality had been implemented (VF personal communication). This also implies that published data on the performance of the first generation PermaNet^® has to be seen in relation to the time of manufacture of tested nets. The first report on field evaluated PermaNet^® comes from Burkina Faso [10] where after 12 months mean delamethrin (n = 11) was 3.7 mg/m² with a mortality of 54% and after 18 months (n = 5) 1.6 mg/m²and 7% respectively. According to Kroeger and colleagues [23] the nets used in that study were an early version of the product manufactured in the first two months of 2000 while the PermaNet^® tested in Columbia by Kroeger and his team [9, 20] were produced in the second part of 2000 at about the same time as the first generation nets used in this study. In the Columbia study six PermaNet^® were washed 20 times by local women during five months and then used by these families for another 2.5 years. After three years mean deltamethrin content (n = 4) was 9.6 mg/m² and mortality 88% [23]. Asidi et al. [24] tested one PermaNet^® in late 2000 after five washes using an experimental hut design and found no difference to conventionally treated nets. Similarly, Graham et al [13] reports on five wash and experimental hut trials of first generation PermaNet^® in three countries with nets being delivered between January 2000 and April 2002 and found mortality on the LLIN after 15 to 21 washes above 90% but median time to knockdown and blood feeding inhibition only significantly better than conventionally treated nets in one trial. In contrast, Gimnig and co workers [25] had good results in a field trial in Malawi using nets produced late 2001 with mortality 42.5% after 24 months of field use (n = 25) and 60% of the first generation PermaNet^® still having minimal effectiveness, very similar to results reported in this study. This varying performance seems to be in keeping with our hypothesis of inconsistent production quality of the nets between early 2000 and late 2001 with some batches or part of batches being better than others which in conjunction with the small sample sizes of the quoted studies would explain the variations in results. In contrast, tests on first generation PermaNet^® from first half of 2002 – just before the finalization of the second generation PermaNet^® (VF personal communication) – tested in a field trial in Kenya showed excellent performance after two years with mean mortality of 72% and 82% of the nets functional with mortality rates at all times greater than 50% [12]. These same nets washed in the laboratory 20 times under standard WHO conditions still showed 55% mortality, 72% knockdown and a chemical residue of 18.5 mg/m², very close to what was found for the second generation PermaNet^® in this study.

To date there are only two studies published on the second generation PermaNet^® which was officially launched as PermaNet 2.0^® in April 2003. Both, however, were not phase III field trials but rather phase I/II washing trials. In the first study in Pakistan [13] PermaNet 2.0^® performed significantly better after 20 washes in the field than a conventional ITN with mortality 81.8%, knockdown 79.5% and a deltamenthrin residue of 24.1 mg/m² when net swatches were washed and 13.1 mg/m²when the whole net was washed. After 30 washes mortality dropped to 43.2% but knockdown rate remained high with 77.3%. Mean deltamethrin content at this time was 5.8 mg/m². In the second study second generation PermaNet^® were used as a control in the laboratory study of another product [26]. Here a mortality rate of 100% was found after 30 washes with 18.7 mg/m² deltamethrin remaining on the net after these washes. Although these results are not directly comparable they indicate already the strong performance of the improved product. This is very similar to results from this study where only the 24 months sample showed slightly reduced figures for mortality (73.9%) and knockdown (92.4%), with a resulting proportion of nets with optimal effectiveness of 71.0 % (WHOPES phase III criteria 80%). However, the confidence interval for this sample did include the 80% mark and three of the nets had been washed immediately before collection which might have resulted in the poorer performance in the bioassay. Also, the deltamethrin content decline showed a continuous, more or less linear decline with no unusual drop at 24 months supporting the hypothesis that this lower value was a statistical outlier rather than a systematic decline in performance. At 36 months 90.0% (95% CI 76.3–97.2) of the second generation PermaNet^® had either mortality ≥ 80% or knockdown ≥ 95% and therefore fulfilled the WHOPES criteria for phase III LLIN evaluation [18]. The nets tested in this study were produced in July 2002, a few weeks before the finalization of the production details for the PermaNet 2.0^® brand but according to the manufacturer did not differ significantly from that product (VF personal communication).

The declared deltamethrin loading dose for the first generation PermaNet^® was 50 mg/m² and a mean of 43.6 mg/m² (95% CI 36.0–51.3) was found which compares well with other reported results of 44.9 [12], 47.1 [10], and 48.3 mg/m² [11]. In contrast, the baseline deltamethrin content for the second generation PermaNet^® was rather high with a mean of 67.1 mg/m² (95% CI 60.0–74.4), above the declared 55 mg/m²but still within the allowed upper limit of 68.7 mg/m²stated in the WHO specifications for deltamethrin on long-lasting (coated) insecticidal nets [21]. When the content was expressed in g/kg, the units of the actual measurement, the baseline value for the second generation PermaNet^® was 2.1 g/kg (95% CI 1.9–2.4) which is closer to the expected value of 1.8 g/kg and clearly within the required limits of 1.35–2.25 g/kg. This difference between mg/m² and g/kg results indicates that the actual, measured mass of net per unit was slightly higher than the 30 g/m² stated in the specifications and, in fact, it was found to be 31.7 g/m² from measurements taken according to the relevant ISO norm 3801 [27]. The range of deltamethrin content on the baseline nets was 52.5–80.4 mg/m² or 1.5–2.5 g/kg but since only two samples were taken per net this is a reflection of within net variability and – as outlined above – can not be interpreted as mean insecticide level for those nets. Other studies have also found rather high baseline insecticide values for the second generation PermaNet^®. Yates et al. [26] using HPLC found a mean baseline deltamethrin concentration of 66.7 mg/m² and Graham et al [13] 86.3 mg/m² when they tested net swatches before washing but 55.3 mg/m² on the samples from nets prepared for washing.

Interestingly, a systematic difference was observed between results from the analytical methods used by the WHO collaboration centre (GC-ECD) and the VF quality control laboratories (HPLC-DAD, CIPAC) with the latter systematically lower than the former (Figure 5) and the difference reaching 0.30 g/kg at the 2.0 g/kg level. This would imply that a mean deltamethrin content of 2.1 g/kg found in the gas chromatography for the second generation PermaNet^® at baseline would correspond to 1.8 g/kg or 55 mg/m² in the methodology used by VF. It is concluded, therefore, that the baseline deltamethrin content of second generation PermaNet^® in this study was within the specified target dose with either method of determination. Furthermore, even if the initial deltamethrin loading had been slightly lower the result in the three year follow-up would not be significantly different as at 36 months on average well above 15 mg/m²deltamethrin would have remained on the nets which was shown to correlate with high bioassay results.

The difference between the analytical procedures was quite significant at higher deltamethrin content and needs further evaluation as both methods are frequently used in the testing of LLIN. It is not likely that the difference was caused by the fact that samples were not exactly identical although taken immediately next to each other on the net. Such a difference by intra-net variability of insecticide would be expected to go in both directions and would average very close to zero. A direct comparison of the two protocols, HPLC-DAD according to #333 CIPAC and GC-ECD according to ISO 17025 at the WHO Collaborating centre in Gembloux, Belgium (n = 11, two measurements per sample) by independent technicians showed almost the same linear relationship and systematic error at higher levels of insecticide as in our reported data with 2.0 g/kg in GC-ECD corresponding to 1.75 g/kg in HPLC-DAD (Pigeon, unpublished data). This means that the difference is also not caused by the execution of the protocol by either laboratory in our study but is a true, systematic difference in the methodologies. However, it is at this point not clear at which part of the protocol the deviation occurs: sample preparation, extraction or determination of insecticide content. Both analytical methods have been validated using samples with known deltamethrin content and shown to have a very high recovery rates of 99.7% (95% CI 98.6–101.6) in the case of the HPLC-DAD (CIPAC) method (Phan, unpublished data) and 98.2% (97.1–99.4) for the GC-ECD method (Pigeon, unpublished data). However, at a deltamethrin loading of more than 30 mg/m² recovery in the GC-ECD method was 101.2% (99.8–102.6). Repeated re-extracting of those samples consistently failed to show any remaining deltamethrin indicating that all insecticide was captured. The higher values measured in GC-ECD at high deltamethrin levels could possibly be explained by slight dilution variability. On the other hand, the sample preparation could also contribute to the systematic difference as in the HPLC-DAD (CIPAC) methodology the 10 × 10 cm sample is cut up into many small pieces before extraction which could lead to a mechanical loss of insecticide. The GC-ECD protocol is the more universally applicable method for insecticide determination in LLIN as it can be used for LLIN with coating as well as incorporation technology and is able to reliably detect even very small amounts of insecticide. The HPLC-DAD (CIPAC) protocol has the limitation that it can only be done with LLIN samples using coating technology but is very suitable for assessment of baseline loading dose of the LLIN and is, therefore, used by manufacturers for quality control purposes. As both methods will continue to play a significant role in the testing of new LLIN products the systematic difference described here clearly needs further study.

The initial batch of ITN also lost insecticide very quickly with a median deltamethrin content of only 0.7 mg/m² after six months of use and a vector mortality and knockdown rate of 8.5% and 31.6% respectively. Although reported results from deltamethrin treated conventional ITN after washing and/or field use vary considerably [11–13, 23, 25, 28], this appears lower than expected. However, when the same nets were re-treated after 15 months they showed a chemical residue of 3.1 mg/m², mortality of 40.8% and knockdown of 76.9% six months later and there was no difference to new polyester nets treated at the same time. These results are very similar to what was found by Gimnig in Malawi [25]. A possible explanation for the initial rapid loss could be the presence of some warping oil on the nets at the time of dipping or a poor quality of polyester that did not allow the insecticide to attach to the fiber. A second group of re-treated conventional nets had even better results with deltemethrin of 1.4 mg/m², mortality 44.8% and knockdown 59.4% after 12 months. One possible explanation is a cumulative effect with insecticide remaining from the previous treatment [29] or a slightly better persistence of the deltamethrin tablets compared to the liquid formulation [28]. Nonetheless, these results are within the reported limits of performance as mortality of > 70% has been described with deltamethrin treated nets even after 15 months of fields use [22] and deltamethrin concentrations as low as 1.5 mg/m² [28].

Nets in our study were used regularly and washed mainly with cold water in a basin and without rubbing on stones, similar as has been described from Tanzania by Erlanger and colleagues [30]. With very few exceptions locally made soap bars were used rather than industrial detergents. The alkalinity of this soap was found to be moderate with ph 9–10 which has also been found in other studies [13, 30] and can be considered favourable with respect to the potential to degrade deltamethrin compared to the industrial detergents. The washing frequency in our study was low with initially 2.2 washes per year and then declining in phase two to 1.8 washes per year. This is significantly lower than the 1.0 wash per month reported in Uganda from a descriptive household interview study [31]. However, as Miller et al have demonstrated in Tanzania reported washing does not correlate well with actually observed washes [32] the latter being significantly lower than the former. The washing frequency in our study areas was not that much different from some of the rates reported from other countries: Gambia 1.9 washes/year [28], Columbia 1.2 [23]. Other studies reported washing rates between 3.6 and 5.6 per year [5, 12, 30] but none in the range of 12 washes per year. It is difficult to say to which extent results on longevity of insecticidal effects observed in this study would have been different if the washing frequency would have been higher. But it might be useful for the WHOPES evaluation process to require studies in settings with differing washing habits.

In the washing study of PermaNet 2.0^® by Graham and colleagues [13] in Pakistan 72.1% of the baseline insecticide had been lost after 20 washes. This is slightly more than the 58.2% loss found in this study for second generation LLIN after three years. On the other hand, the loss was 59% after 20 washes in the laboratory washing study by Yates at al. [26]. This indicates that the loss observed in this study over three years would be equivalent to somewhere between 15 and 20 washes without field use. However, the mean number of washes actually observed for our tested nets was 4.5 with a range of 0–9. This suggests that under every day use in the field washing alone is not the only determinant of insecticide loss. This finding was further explored using a linear regression model with number of washes and time of field use as key variables and found that, indeed, more insecticide (75.6 %) was lost through every day use than by washing (24.4 %). It is not exactly clear whether physical handling of the nets or environmental factors or a mixture of these causes the loss and this issue will require further study. It is, however, quite clear from our results that at least for LLIN using coating technology, maximum number of washes obtained for a LLIN product in the laboratory without loss of functionality can not be translated into the same number of washes under field conditions. This emphasizes the need for phase III trials under field conditions in the process of WHOPES evaluation. The situation may be different for LLIN using incorporation technology [33].

A highly significant correlation was found between the chemical residue and bioassay results for samples taken from the same location on the net. A very similar relationship has been described by Yates and colleagues between chemical residue and median time to knockdown [26]. Adams et al. [34], on the other hand, did not find a statistically significant correlation between insecticide content and bioassay results possibly because none of the insecticide levels were below 3 mg/m² and most corresponding mortality values above 90% and all above 50%. At least for Anopheles gambiae s.s. deltamethrin levels as low as 1–3 mg/m² can produce knockdown rates between 75% and 95% and mortality above 50%, something that several authors have described previously [23, 27, 29, 30, 34]. Culex quinquefasciatus responded less sensitive to deltamethrin with bioassay results rapidly declining at insecticide levels below 15–20 mg/m². This is thought to be due to a generally higher tolerance of this species to tarsal contact insecticides [22].

Based on a total of 578 observations, it could be established that using a cut-off of ≥ 4 mg/m² and ≥ 15 mg/m² deltamethrin on a net sample can distinguish nets with minimal and optimal performance in the bioassay with An. gambiae with a positive predictive value of > 95%. Sensitivity was rather low and this test, therefore, not suitable to identify an indivudual net that fails the effectiveness standards. But it would be possible to use chemical residue as a proxy to identify nets that continue to provide protection in situations where taking of large samples for bioassays is not possible. One must keep in mind, however, that this test would then need to assume that the insecticide found on or in the fibre also is bio-available to the mosquito which may not always be the case, particularly in LLIN that use the incorporation technology [11, 12].

When investigating the longevity of protection one can expect from a LLIN product under programmatic conditions the duration of the insecticidal effect is only one part of the equation. The duration of the physical integrity of the netting material and the resulting useful life is equally important and in contrast to the former, there is very little published literature on the latter. Ritmeijer et al. [35] report 65% of fine-mesh, 75 denier polyester nets with any holes after two years in a leishmaniasis programme in Sudan and Spencer and co-workers [36] found 78% of 75 denier polyester nets with any holes 12–15 months after distribution in a refugee camp in western Uganda. Both these findings are within the results from this study (Figure 2). In a cross-sectional survey in Tanzania [30] 86% of polyester nets were found with any holes but the age range of the nets was 0.5–5 years and the fibre strength ranged from 40 to 100 denier.

Reports on the severity of damage are difficult to compare as each study used a different definition. They ranged from 45% severely damaged (> 7 holes larger 2 cm) in the cross-sectional net survey [30] to 33.2% with > 5 holes after two years in Sudan [35] to 28% with at least one hole of 40 cm² after 12–15 months in Uganda [36]. In order to provide a more standardized and continuous measure of the physical net condition a hole index was used which was constructed in analogy to the classical spleen index in malariology which combines the frequency of any hole with the size of these holes divided into three categories. It was demonstrated that the physical condition of the 75 denier polyester nets linearly deteriorated in the first 3 years but decay then seemed to accelerate somewhat with evidence that severely torn nets with a hole index of 30 or more were significantly more likely to be stored away rather than used.

Interestingly, the rate of hole acquisition (proportion with any hole) over time was slower in the second phase of the study after families had used nets already for two years. This trend continued in the current third phase of LLIN testing in the very same families where after 12 months only 34% of nets had any holes compared to 50% in phase two and 71% in phase one and after 18 months the figures were 44%, 64% and 80% respectively. This strongly suggests that net users learn to handle the net better or more carefully with increasing user experience and emphasizes the importance of a communication campaign to support the development of such a net culture.

While it is reasonably easy to monitor the physical condition of a net, it is very difficult to infer from these findings on the average useful life of a net under "real life" conditions as the continued use of a torn net will depend on many factors such as availability of a replacement, presence of other nets in the household that can be shared or perceived pressure to keep the net in a study or project setting. In this study 83.5% of the originally distributed polyester nets were still present after five years when those nets sampled or replaced at earlier times were not considered [see additional file 2]. This is similar to what has been reported for polyethylene nets in Tanzania where seven years after distribution 97% of households that received one or more nets still had at least one net in spite of the observation that 93% of the nets had any holes and 55% had 6–15 holes of 2 cm or more which would be equivalent to a hole index of 15–45 [33]. Although these polyethylene nets (equivalent to ~180 denier) are likely to be stronger than a 75 denier polyester net neither observation is likely to be equivalent to a realistic average useful life estimate due to the study/project situation. A household survey design may be a better tool to approach the useful life of nets and based on one such study is likely to average around 3 years for polyester nets of medium quality [37].