Skip to main content

Comparing the new Ifakara Ambient Chamber Test with WHO cone and tunnel tests for bioefficacy and non-inferiority testing of insecticide-treated nets

Abstract

Background

Insecticide-treated net (ITN) durability, measured through physical integrity and bioefficacy, must be accurately assessed in order to plan the timely replacement of worn out nets and guide procurement of longer-lasting, cost-effective nets. World Health Organization (WHO) guidance advises that new intervention class ITNs be assessed 3 years after distribution, in experimental huts. In order to obtain information on whole-net efficacy cost-effectively and with adequate replication, a new bioassay, the Ifakara Ambient Chamber Test (I-ACT), a semi-field whole net assay baited with human host, was compared to established WHO durability testing methods.

Methods

Two experiments were conducted using pyrethroid-susceptible female adult Anopheles gambiae sensu stricto comparing bioefficacy of Olyset®, PermaNet® 2.0 and NetProtect® evaluated by I-ACT and WHO cone and tunnel tests. In total, 432 nets (144/brand) were evaluated using I-ACT and cone test. Olyset® nets (132/144) that did not meet the WHO cone test threshold criteria (≥ 80% mortality or ≥ 95% knockdown) were evaluated using tunnel tests with threshold criteria of ≥ 80% mortality or ≥ 90% feeding inhibition for WHO tunnel and I-ACT. Pass rate of nets tested by WHO combined standard WHO bioassays (cone/tunnel tests) was compared to pass in I-ACT only by net brand and time after distribution.

Results

Overall, more nets passed WHO threshold criteria when tested with I-ACT than with standard WHO bioassays 92% vs 69%, (OR: 4.1, 95% CI 3.5–4.7, p < 0.0001). The proportion of Olyset® nets that passed differed if WHO 2005 or WHO 2013 LN testing guidelines were followed: 77% vs 71%, respectively. Based on I-ACT results, PermaNet® 2.0 and NetProtect® demonstrated superior mortality and non-inferior feeding inhibition to Olyset® over 3 years of field use in Tanzania.

Conclusion

Ifakara Ambient Chamber Test may have use for durability studies and non-inferiority testing of new ITN products. It measures composite bioefficacy and physical integrity with both mortality and feeding inhibition endpoints, using fewer mosquitoes than standard WHO bioassays (cone and tunnel tests). The I-ACT is a high-throughput assay to evaluate ITN products that work through either contact toxicity or feeding inhibition. I-ACT allows mosquitoes to interact with a host sleeping underneath a net as encountered in the field, without risk to human participants.

Background

National malaria control programmes (NMCPs) must ensure that all people living in malaria transmission areas are protected through the provision, nightly use and timely replacement of high quality long-lasting insecticidal nets (ITNs) and where appropriate, the additional application of indoor residual spraying (IRS) [1]. While it is assumed that all ITNs that have World Health Organization (WHO) prequalification listing last for 3 years, several ITN products are available that may vary in price as well as performance under local conditions [1,2,3,4,5,6,7]. Because ITNs are the primary means of malaria control, their durability, measured through physical integrity and bioefficacy against anopheline mosquitoes, needs to be accurately assessed in order to inform NMCPs of the most cost effective products and the correct interval for net replenishment campaigns [8].

Any ITN product is expected to retain its insecticidal activity (bioefficacy) for a minimum number of 20 standard washes or 3 years of use under field conditions as defined by the WHO [9]. However, the durability (years of functional life) of both existing and new net products under development is a crucial consideration. Despite mass distribution of ITNs, currently fewer than 50% of people living in malaria endemic areas are covered by one of the core malaria interventions: either ITNs or IRS [10]. Maximizing ITN access through the provision of the most long-lasting, and cost-effective products remains a critical concern, particularly as a number of countries have shown an increase in malaria in the past year (2016/2017) as investments in malaria control have plateaued [10].

For products within new intervention classes e.g. dual active ITNs, an Evidence Review Group (ERG) report to the WHO Malaria Policy Advisory Committee (MPAC) recommended specific guidance on the assessment of non-inferiority of products within a class [11]. A non-inferiority trial of an intervention aims to demonstrate that the test product is not worse than the comparator/reference by more than a pre-specified margin [12], known as the non-inferiority margin. For ITNs this margin relates to mortality or feeding inhibition. In recognition of the importance of ITN durability, the WHO recommended that once sufficient test and active comparator ITNs from large-scale field trials have been collected over 3 years of field use, a second set of two non-inferiority trials should be conducted to ensure that the test product continues to be non-inferior to the comparator/reference product for up to 3 years on both mosquito mortality and blood-feeding inhibition endpoints [13]. While this guidance recommended that non-inferiority trials should be conducted in experimental huts it was acknowledged that alternative methodology for non-inferiority testing including the ambient chamber test or the tunnel test should be explored.

The standard means of ITN bioefficacy evaluation is through cone bioassays, WHO tunnel tests and experimental hut evaluations [14]. The cone test is a contact assay where mosquitoes are held in proximity to the ITN and mosquito knockdown (KD60) and 24-h mortality are recorded after 60 min and 24 h, respectively. The tunnel test uses a live animal as a bait (rabbit or guinea pig), so mosquitoes are able to exercise host-seeking behaviour, and ITN efficacy is assessed by measuring mosquito mortality and blood feeding inhibition [15,16,17]. Experimental huts are small scale field (phase II) testing assays used to evaluate ITNs that meet laboratory (phase I) testing criteria [8, 18]. Huts are built in areas with high densities of target mosquito species and are designed to resemble small local housing but have features to retain mosquitoes that enter huts such as window traps and baffles [19]. Volunteers sleep underneath the ITNs and wild mosquitoes attempt to feed and interact with the ITNs in the same way as they would in local homes. Both mortality and feeding inhibition are key outcome parameters, which translate to personal and community protection from malaria [20].

However, all assays have some limitations, which need to be considered when assessing bioefficacy of ITNs. WHO cone tests may underestimate the induced mortality of irritant insecticides, as mosquitoes do not settle on treated nets [21]. Indeed, comparatively higher mortality is often measured in experimental hut studies of ITNs where mosquitoes make repeated contacts with treated nets as they try to feed on human volunteers sleeping under nets. In the WHO tunnel test, the live host used as bait is not the preferred host for the strongly anthropophilic Afro-tropical vector Anopheles gambiae sensu stricto (s.s.) [22] and may overestimate feeding inhibition. Alternatively, mosquitoes must be reared by feeding them on small mammals to select them for a preference to these non-preferred hosts, which is both expensive and of animal welfare concern. Experimental hut bioassays are the gold standard for ITN and IRS evaluation, but wild mosquito populations are often seasonal and have high temporal heterogeneity requiring substantial replication to ensure adequate power to detect true effect differences between products [23].

Therefore, presented here is the first evaluation of a new standardized semi-field assay: the Ifakara Ambient Chamber Test (I-ACT) assay. The assay was used to evaluate the bioefficacy of whole ITNs that were returned from the field in a longitudinal durability study. This study measured the bioefficacy of used (field-aged) ITNs using the I-ACT assay and standard WHO durability testing bioassays (cone and tunnel tests). The proportion of nets passing WHO criteria by standard methods and I-ACT was compared. The aim was to demonstrate the utility of this new assay for measuring bioefficacy of different ITN products and to explore its applicability for non-inferiority testing of new ITN products [11]. Further work comparing the I-ACT and experimental hut evaluations of ITNs will be reported separately.

Methods

Study design

Bioefficacy tests were conducted as part of a 3-year prospective project (the ABCDR-Attrition, Bioefficacy, Chemical residual, Damage and Resistance project) to assess of the useful life of three brands of ITNs in Tanzania [24]. The main design characteristics of each bioassay performed in this study are presented in Table 1. The study involved two experiments. In the first experiment, ITNs efficacy, measured by cone bioassay and I-ACT were compared. In the second experiment, ITNs bioefficacy measured by WHO tunnel test and the I-ACT was compared. The overall pass/fail rate for each net brand by year were examined following the criteria outlined in both 2005 and the 2013 World Health Organization guidelines for evaluation of long lasting nets [14, 25]. The pass rate of each product by standard WHO methods and I-ACT was compared.

Table 1 Design characteristics of the WHO cone test, WHO tunnel test and I-ACT

Mosquito rearing

Mosquitoes used during testing were laboratory-reared fully pyrethroid susceptible 3–8 days old female adult An. gambiae s.s. (Ifakara strain, Njage 1996) reared following standard methods [26].

Mosquito nets

All mosquito nets used in this study came from a 3-years prospective longitudinal follow-up study between 2013 and 2016 (ABCDR Project) conducted in eight districts of Tanzania. Net samples were randomly selected using net codes from master list from the three surveys conducted between October and December 2014 (year 1), October–December 2015 (year 2) and October–December 2016 (year 3). The detailed description of the ABCDR Project has been reported previously [24]. Three net brands were used for this study: (1) Olyset® net (permethrin incorporated into polyethylene fibres @ 1000 mg/m2), (2) PermaNet® 2.0 net (deltamethrin coated on polyester fibres @ 55 mg/m2) and (3) Netprotect® net (deltamethrin incorporated into polyethylene fibres @ 63 mg/m2). All nets were rectangular, white, double sized (190 cm length × 180 cm width × 150 cm height) and recommended by WHO [27]. All nets were used in the I-ACT as found i.e. with damage due to wear and tear. In the first experiment, to compare between cone test and I-ACT, a total of 432 nets (144 per net brand) were evaluated and results compared. In the second experiment to compare between tunnel test and I-ACT, nets those failed to meet cone test threshold criteria in the first experiment were assessed using WHO tunnel test (132 nets) and their results were compared with those from I-ACT.

Ifakara Ambient Chamber Test (I-ACT)

This is a 50 m long, 3 m wide and 2.1 m high steel tube frame construction (Fig. 1a) covered by durable UV resistant polyurethane coated netting with an overlaid polyurethane sheet to minimize wind so that bioassays are conducted in still air (as would occur in a house). The structure is constructed upon a concrete base surrounded by a water channel to prevent entry by ants and spiders that eat mosquitoes during the conduct of experiments. The tunnel sits beneath a simple beamed wooden frame supporting a corrugated steel roof to allow work to continue in all weather conditions. The netted tunnel is divided into ten individual test chambers with interconnecting doors that are sealed by means of zips and Velcro to prevent mosquitoes moving from one test chamber to another. Each compartment contains a white netted chamber 5 m long, 2 m wide, and 2 m high that seals with a zip, in which the ITN is hung from a frame with a human volunteer sleeping underneath (Fig. 1b). At each end of the tunnel is an additional double door module to ensure no loss of laboratory-reared mosquitoes into the wild. Mosquitoes are released from the holding cages within each netted chamber by means of raising a netted cage from its removable wooden base. This is achieved by the technician in situ underneath his net pulling a nylon line attached to the mosquito release cage to elevate it (Fig. 1c). After the allotted experimental time period all mosquitoes within each of the compartments are recovered by mouth aspiration (for mosquitoes inside the net) and by a battery powered Prokopack aspirator (for mosquitoes outside the net but inside the compartment). This allows whole ITNs to be tested in a controlled ambient chamber test with a human host sleeping beneath (Fig. 1d) to measure the protective efficacy (both personal protection measured by feeding inhibition and community protection measured by mosquito mortality) under user conditions. The design of the chambers allows 100% recovery of released mosquitoes that improves precision of the data, and experiments can be conducted year-round.

Fig. 1
figure1

The I-ACT. Ifakara Tunnel situated at Bagamoyo branch of IHI (a). Net covered tunnel divided into 10 individual compartments each containing netted cage 2 × 2 × 5 m (b). The volunteer releases mosquitoes by opening the lid of the holding boxes while beneath the tested net (c). A human volunteer sleeps underneath the ITN (d)

Each of the ten testing chambers was randomly assigned one whole net (with wear and tear as found after use for 1, 2 or 3 years) from one of the three net brands using a random number generator. Two chambers were used each night as negative control with untreated SAFI Net (A to Z, Tanzania) holed with six holes 4 × 4 cm (Additional file 1) i.e. two holes on each large side and one hole on each small side (hole surface area of 96 cm2) according to WHO guidance [9]. One adult volunteer per chamber slept underneath the nets from 21:00 to 06:30 h and collected mosquitoes in the mornings. Each volunteer was fixed to the same chamber for the duration of the experiment. On each night of experiment, each volunteer hung the tested net on the bednet frame and tucked it underneath the mattress (between 28 and 35 cm of each net was tucked). At 21:00 h, each volunteer released 30 mosquitoes within the chamber but outside of the ITN by opening the mosquito release cage while remaining beneath their test net. The following morning, at 06:30 h, mosquitoes inside the net were collected first using a mouth aspirator and mosquitoes outside the net but within the chamber (floor and walls) were collected using a 6 V battery driven mechanical aspirator (Prokopack). A study supervisor checked the start and finish of the experiment and intermittently spot checked that the volunteers were in position overnight to ensure good conduct of the experiment. All collected mosquitoes were placed in paper cups and scored as dead-fed, alive-fed, dead-unfed, alive-unfed after which mosquitoes were held for 24 h in the laboratory with access to 10% sugar solution at 27 °C ± 2 and 80% ± 10 relative humidity and scored again. After every experimental night, all tested nets were taken out and chambers were aired and bed sheets were washed daily to prevent any carry-over insecticide residue. Each net sample was tested on two consecutive nights (fixed to a chamber and volunteer) to improve the precision of the estimation of performance of each net. Outcome measures were 24 h mortality and blood feeding inhibition. Nets that induced ≥ 90% blood-feeding inhibition and/or ≥ 80% mortality were regarded as meeting WHO efficacy criteria. Data were discarded and test repeated if control mortality exceeded 10% or control blood-feeding success was less than 50%. A Standard Operating Procedure for conducting Ifakara Ambient Chamber Test is provided as an Additional file 2.

Cone tests

Cone tests were conducted following WHO guidelines [8] to determine the bioefficacy of insecticides on sampled netting pieces. For each of the sampled whole nets, after completion of the I-ACT, four 30 cm × 30 cm sub-samples were cut from positions 2, 3, 4 and 5 from each net sample (Additional file 3). Cone bioassays were held at a 60° vertical angle on the netting sub-samples [28]. Anopheles gambiae s.s., aged 3–5 days old, were exposed for 3 min after which they were held for 24 h with access to 10% sugar solution at 27 °C ± 2 and 80% ± 10 relative humidity. The numbers of mosquitoes knocked down after 60 min (KD60) and dead at 24 h post-exposure were recorded. A sub-sampled net that caused ≥ 95% KD60 and/or ≥ 80% 24 h mortality in the cone test was regarded as meeting WHO efficacy criteria. Tests, where control mortality at 24 h exceeded 10%, were discarded and repeated.

WHO tunnel test

WHO tunnel tests were conducted following WHO guidelines [8] to assess the efficacy of netting sub-samples that failed to meet the WHO threshold criteria for cone test (95% KD60 and/or 80% 24 h mortality). The surface area of the sample netting accessible to mosquitoes was approximately 625 cm2 (25 × 25 cm) with nine holes cut in the net, each 1 cm in diameter: one at the centre of the square and the other eight holes were equidistant and located 5 cm from the border. The sampled net piece was inserted on a cardboard frame and positioned across the tunnel, one-third of the length of the tunnel. A total of 100 sugar-starved An. gambiae s.s. aged 5–8 days were released in the long section of the glass tunnel at 18:00 h. A rabbit was used as a bait and positioned on the other side of the net so that mosquitoes must pass through the holed net to access the bait and feed. The following morning, between 06:00 and 09:00 h, mosquitoes were removed (separately from each section of the tunnel) using a mouth aspirator, counted, scored as alive or dead, blood fed or unfed after which they were held for 24 h with access to 10% sugar solution at 27 °C ± 2 and 80% ± 10 relative humidity. The main outcome measures were 12 h mortality measured in the morning after the experiment and blood feeding inhibition [14]. The 24 h mortality was also recorded as a secondary outcome [25]. Nets that caused ≥ 90% blood-feeding inhibition and/or ≥ 80% mortality was regarded as meeting WHO efficacy criteria [25]. Tests were discarded if control mortality at 24 h exceeded 10% or control blood-feeding success was less than 50%.

Data management and analysis

A sample size calculation for generalized linear mixed effects models (GLMMs) through simulation [12] in R statistical software 3.02 http://www.r-project.org [13] was performed for the semi-field experiments to detect a difference between the nets of 5% mortality (half the smallest anticipated effect size). Simulations were performed using an estimated mosquito mortality of 70% with the Olyset® and 80% for the PermaNet® 2.0 and NetProtect®. With 44 replicates tested on two occasions with an inter-observational variance of 0 for the chamber (controlled environment) and 0.1 for individual and 0.1 for the night of observation based on the variance of the random effects observed in a pilot study. Power was estimated at > 90% with a density of 30 mosquitoes per chamber per night using 1000 simulations.

Data were collected on standardized data collection forms and double entered into Microsoft Excel. Data were cleaned and analysed following a predefined analysis plan using STATA 14.1 (Stata Corp., College Station TX, USA) with significance level of ≤ 0.05 for rejecting the null hypothesis. Descriptive statistics were used to present the comparison of proportion of nets passing WHO threshold criteria as measured by each method. Generalized linear mixed models (GLMM) with a binomial error distribution and logit link function were used to analyse the main outcome measures from cone, tunnel and I-ACT (mortality and blood feeding) as well as the proportion of nets passing WHO criteria in order to detect differences between the two evaluation methods. Net brand, age of net and bioassay method were fitted as fixed effects while date was fitted as a random effect to account for repeated testing of individual nets. Several GLMMs were run for each comparison (with interactions) and the final model selected was that with the lowest Akaike’s Information Criterion (AIC). Residuals were plotted using histogram, qnorm plots and comparison with fitted values to ensure appropriateness of the model selection and testing if the residuals are normally distributed. Odds ratio (OR) and 95% confidence interval were calculated for the differences between methods in each comparison.

In addition, non-inferiority between net products (PermaNet® 2.0, Netprotect® with Olyset® as reference/comparator) measured by I-ACT was analysed using a paired t-test with a 90% confidence interval of the observed effect difference in the mortality and bloodfeeding inhibition rates to measure non-inferiority at a margin of 10% and data were presented for comparison using a Forest Plot [29].

Results

Comparison between cone test and I-ACT

The data presented in Table 2 show that a smaller percentage of nets passed WHO threshold criteria using cone test 62% (268/432) than passed using by I-ACT 97% (417/432) irrespective of brand and net age (cone test criteria ≥ 80% 24-h mortality and/or ≥ 95% KD60; I-ACT criteria ≥ 80% 24-h mortality).

Table 2 Percentage and number of nets (by brand and age) meeting the standard WHO 2013 threshold criteria by I-ACT and cone tests

Using cone bioassays, 8% (12/145) of Olyset® nets, 88% (126/144) of PermaNet® 2.0 nets and 91% (130/143) of NetProtect® nets passed the cone test. When tested by the I-ACT, 94% (137/145) Olyset®, 98% (141/144) PermaNet® 2.0 and 97% (139/143) NetProtect® passed (Table 2). Table 3 shows that, overall, I-ACT measured higher 24 h mosquito mortality than cone test regardless of net brand (OR: 7.9, 95% CI 7.4–8.4, p < 0.0001). Disaggregated by brand the same trend was evident and I-ACT measured higher mortality than cone test: Olyset® nets (OR: 17.8, 95% CI 16.3–19.5%; p < 0.0001), PermaNet® 2.0 (OR: 2.1, 95% CI 1.8–2.3%; p < 0.0001) and Netprotect® (OR: 3.6, 95% CI 3.2–4.1, p < 0.0001).

Table 3 Measurements of percentage 24 h mortality compared between WHO cone test and I-ACT by net brand and age

Comparison between WHO tunnel test and I-ACT

A total of 164 nets (132 Olyset®, 19 PermaNet® net and 13 Netprotect®) did not meet the WHO threshold criteria for cone test and therefore went for WHO tunnel tests. Only bio-efficacy results of Olyset® sampled nets were used for comparison with I-ACT to ensure an adequately replicated and paired comparison, because the majority of PermaNet® 2.0 and Netprotect® sub-sampled nets passed the cone test.

The overall proportion of Olyset nets meeting WHO thresholds in tunnel test using methods outlined in WHO 2013 guidance (mortality recorded the morning after bioassay) and I-ACT is shown in Table 4. In addition, further analysis based on WHO 2005 guidance (mortality recorded after 24 h holding) was also performed and included in the results for comparison. Overall, results from Table 4 shows, regardless of the WHO ITN testing guideline used, more Olyset® nets passed when measured in I-ACT than in tunnel test (using 24-h mortality OR: 5.7, 95% CI 2.5–, p < 0.0001).

Table 4 Overall percentage of sampled Olyset® nets that met the standard WHO threshold criteria as measured by I-ACT and tunnel test following both WHOPES 2013 and 2005 guidelines

Using either 12 h mortality or 24-h mortality, I-ACT recorded higher mortality than the tunnel test (Table 5). At 12 h, 64.1%, (95% CI 60.1–68.3%) vs 49.5% (95% CI 44.5–54.6%) (OR 1.7 (1.6–1.8), p < 0.0001) at 12 h and 71.2% (95% CI 67.7–74.9%) vs 64.4% (95% CI 59.8–69.4%) (OR 1.3 (1.2–1.4), p < 0.0001) at 24 h. For Olyset® nets mortality was significantly higher measured after a 24-h holding period compared to the morning of collection in WHO tunnel test but not I-ACT (Fig. 2). Feeding inhibition (Fig. 3) of Olyset® nets was also higher as measured by I-ACT (96.4%, 95% CI 94.7–98.1%) than tunnel test (88.9%, 95% CI 86.2–91.7%) (OR 3.6 (3.1–4.2), p < 0.0001). Similar trends were seen among the deltamethrin nets PermaNet® 2.0 and NetProtect® but data are not shown due to the imprecision of estimates from the low number of nets evaluated (19 and 13, respectively).

Table 5 Measurement of 12-h mortality, 24-h mortality and blood feeding inhibition compared between WHO Tunnel test and I-ACT for sampled Olyset® nets through 3 years of field use
Fig. 2
figure2

Mortality in susceptible Anopheles gambiae s.s. exposed to Olyset® nets by year using the WHO tunnel bioassay (left panel) and the I-ACT (right panel) following WHOPES 2013 [59] and 2005 [25] guidelines for durability monitoring. Error bars indicate 95% confidence interval

Fig. 3
figure3

Mosquito blood feeding inhibition by year in susceptible An. gambiae s.s. exposed to Olyset® nets using the WHO tunnel bioassay and I-ACT. Error bars indicate 95% confidence interval

Proportion of nets meeting the combined WHO methods

Figure 4 shows that even after 3 years of field use PermaNet® 2.0 and NetProtect® killed a greater proportion of mosquitoes than Olyset® resulting in a higher pass rate by WHO (combined cone/tunnel) methods, while there was less contract in the performance of the three products overall when tested using I-ACT. The data (Table 6) show that overall more nets passed WHO threshold criteria using I-ACT than using standard WHO (combined cone/tunnel) methods irrespective of brand and age (OR: 3.5, 95% CI 1.9–6.5, p < 0.0001). The proportion of nets passing using combined WHO methods agreed with I-ACT for NetProtect® and PermaNet® 2.0 but differed different for Olyset® with 94% passing in I-ACT vs 77% by standard bioassays (OR 5.2, 95% CI 2.3–11.8, p < 0.0001). A second notable difference was that holding time was important in determining the pass rate of Olyset® net with a significant increase in the proportion of Olyset® that passed when 24 h mortality vs immediate mortality scoring was used: 77% (95% CI 69–83%) vs 71% (95% CI 62–78%) pass. A small and not significant difference was observed in the proportion of PermaNet® 2.0 (94% vs 98%) and NetProtect® (98% vs 97%) passing by either WHO methods or I-ACT methods, respectively.

Fig. 4
figure4

Percentage of ITNs by brand and age passing bioassay criteria following WHO 2013 and 2005 guidelines as measured by standard bioassays (a, c) vs I-ACT (b, d) against An. gambiae s.s. (Ifakara strain) fully susceptible to all classes of insecticides

Table 6 Difference in the proportion of nets passing WHO 2005 threshold criteria by combined WHO cone and Tunnel test methods compared to I-ACT for sampled Olyset®, PermaNet® 2.0 and NetProtect® nets

Non-inferiority of sampled net product

In order to measure non-inferiority of field aged nets Olyset® was used as the reference net (active comparator for non-inferiority testing) and the other two nets were compared to it. Using I-ACT data it can be seen that overall, PermaNet® 2.0 and Netprotect® killed greater proportion of mosquitoes than Olyset®. Using a t-test of the effect difference between the products using the 24 h mortality endpoint with a margin of 10% of non-inferiority it can be seen that both PermaNet® 2.0 and NetProtect® were superior to Olyset® (Fig. 5). It should be noted that the figure shows a negative value for superior products because the effect difference is calculated by subtracting the induced mortality of the test net from the reference net. However, using the feeding inhibition endpoint, both PermaNet® 2.0 and NetProtect® were non-inferior to Olyset using a 10% margin of non-inferiority. It can, therefore, be concluded that the three products are equivalent based on a combined mortality and feeding inhibition endpoints, as is current WHO practice.

Fig. 5
figure5

Non-inferiority of PermaNet 2.0 net and NetProtect combined 24 h mortality and feeding inhibition for 3 years of data with Olyset® as the reference performed in the I-ACT using a 10% margin of non-inferiority

Discussion

This is the first ITN durability study to compare bioefficacy of ITNs using standard WHO bioassays (cone and tunnel tests) with data collected from whole nets tested using the I-ACT. A large numbers of three brands of nets returned from the field were evaluated in I-ACT to measure their protection to users sleeping underneath them in the presence of natural wear and tear. The I-ACT allowed high throughput (48 nets per brand per year) to give precise estimates of overall product efficacy. Data from the I-ACT for the deltamethrin nets PermaNet® 2.0 and Netprotect® that function through rapid knock down and mortality largely agrees with the standard WHO methods (cone/tunnel tests). However, for Olyset® that functions through the prevention of feeding a greater proportion of nets passed using I-ACT than standard WHO methods (cone/tunnel tests).

It was observed that each of the three net brands showed lower efficacy measured by standard WHO bioassays compared to efficacy measured by I-ACT. This could be due to: (1) duration of exposure (3 min vs 12 h), (2) surface area of treated fabric presented to the mosquitoes (both standard WHO methods use 20 cm2 samples versus a whole net in the I-ACT) and (3) number of tarsal contacts with the ITN resulting in exposure to different dose of insecticide due to the presence of a human host under the net for the I-ACT. In cone test experiments, mosquitoes are exposed to tested ITN for only 3 min which may not allow the tested mosquitoes to exercise natural host-seeking behaviour with multiple contacts over the net surface resulting in a higher cumulative dose of insecticide. This has also been measured by other authors in studies to understand behavioural and physiological changes in mosquitoes in relation to responses to insecticides. A series of studies by Angarita-Jaimes and colleagues using a novel video-tracking system to quantify the behaviour of nocturnal mosquitoes attacking human hosts in the laboratory and in field observed that, both An. gambiae s.s. and Culex quinquefasciatus showed multiple contacts with bed nets when a human host was present [30], and this host seeking activity is lower for treated nets than in untreated nets [31, 32]. However, the I-ACT study demonstrated that these contacts were sufficient to kill or inhibit feeding among the majority of pyrethroid susceptible mosquitoes used in this study.

In addition, the findings add to existing data that shows that the cone test underestimates the bioefficacy of Olyset® that contains Permethrin, a contact irritant pyrethroid [15, 17, 33, 34]. During cone tests, permethrin causes mosquitoes to minimize contact with the netting fibres and they may sometimes rest on the sides of the cone or cotton plug on the cone and avoid the insecticide and demonstrate frequent take offs from the net [28].

The tunnel test was developed as a consequence of the need to measure feeding inhibition of permethrin-treated nets [35] and has also shown some use in evaluating products that fail cone tests including chlorfenapyr products as it allows mosquitoes to exhibit flight and host seeking feeding behaviour in a natural simulated condition [36]. However, as with cone test, the tunnel test has some limitations. The overall pass rate (using 12 or 24-h mortality and blood feeding inhibition), as measured following both WHO 2005 and 2013 criteria, was lower compared to that measured by I-ACT. A possible explanation for this observation is that, the baits used in tunnel tests are rabbits that are not the preferred bait for An. gambiae s.s. that feeds almost exclusively on humans [22, 37]. Therefore, during standard tunnel test experiments, mosquitoes may be less responsive to non-preferred bait and remain in the releasing chamber throughout the exposure time without interacting with the ITN sample resulting in a lower cumulative exposure to insecticide. Additionally, using a whole net in the I-ACT killed more mosquitoes possibly due to the large surface area of insecticide available for mosquitoes to interact with. It is known from repellent testing that use of a non-preferred bait will overestimate repellent efficacy [38]. However, a similar number of PermaNet® 2.0 and NetProtect® passed the combined WHO tests and the I-ACT whereas fewer Olyset® passed combined WHO tests indicating that the WHO tests are conservative and therefore unlikely to pass a product that is of low efficacy. As the I-ACT is a less conservative test it may have use for early screening of new insecticide treated nets including those with irritant insecticides or those that function through a mode of action other than rapid knockdown before more costly experimental hut tests.

The overall percentage of Olyset® nets that passed tunnel test following WHO 2013 guidelines was marginally lower than when following WHO 2005 guidelines [25]. This suggests that reinstating the WHO 2005 pass/fail efficacy criteria may be justified to avoid missing products that are efficacious during early testing. This will also align tunnel test holding times with those of cone bioassays and experimental huts (24 h). It may also be justifiable when testing some products to hold mosquitoes for even longer than 24 h as some authors have done to measure the effects of slow acting insecticides [39, 40]. This simple pairwise test between the two guidelines demonstrated the usefulness of exploring the impact of holding time on the outcome of product tests.

The mode of action of insecticides used on ITNs is an important consideration when selecting bioassays. New products with modes of action different from pyrethroids (which are fast acting and neurotoxic) are coming to market and there is a need for a suitable means to bioassay them. An example is chlorfenapyr, which acts by disrupting metabolic respiratory pathways (oxidative phosphorylation) in the cells of mitochondria and that require the conversion of the active compound through metabolism [41]. The conversion is optimal at night and is maximized when mosquitoes are metabolically active i.e. during the active part of their circadian rhythm and flying during host seeking [42]. Cone tests are usually conducted during the day and take 3 min exposure time with no bait involved. Findings from two well conducted studies by Oxborough et al. and Ngufor et al. observed extremely low levels of mortality caused by chlorfenapyr compared to pyrethroids when assessed by cone test, but excellent effect against resistant mosquitoes when tested in experimental huts [39, 43]. These data again suggest that cone test may be best suited for fast acting non-irritant insecticides [35] and there is a need to be open to exploring new bioassays for new mode-of-action products. The higher pass rate of I-ACT compared to standard WHO tests may be useful when conducting “quick and dirty” tests for new products to avoid early “kill” of promising products because they are failing to pass bioefficacy criteria in phase I laboratory tests when they may prove highly efficacious in gold standard experimental hut tests (Phase II).

Ifakara Ambient Chamber Test may be useful in evaluating new products that function through either mortality or feeding inhibition. Tests are conducted at times when mosquitoes are metabolically active, and using the preferred host of Afro-tropical malaria vectors. The advantage of using the I-ACT is that nets are evaluated using mortality and feeding inhibition using just one test rather than having to perform the cone (for mortality) followed by the tunnel test bioassays (for feeding inhibition or mortality at night). Regarding the issue of precision in outcome measure estimates, the durability study performed here in the I-ACT used 30 mosquitoes per chamber per night of experiment and allowed large numbers of nets to be evaluated without exhausting the insectary which is always a concern when product testing. It is important to assess a large number of nets in durability studies to allow a sufficient sample of nets to be returned from the field to capture the large heterogeneity in product performance i.e. fabric integrity and insecticidal content, and using a random sampling framework that is large enough to avoid sampling bias such as the Hawthorne effect [44].

When the efficacy of ITNs was compared using standard WHO assays and I-ACT, it was seen that most of the tested nets were extremely effective against mosquitoes and passed WHO criteria of feeding inhibition and/or mortality using the pyrethroid susceptible An. gambiae s.s. (Ifakara) strain even after 3 years of use with natural damage and insecticide depletion from the field. This has also been shown by other research in Tanzania [34, 45, 46]. Many of the tested nets were damaged. The median hole surface area was 459 cm2 in Olyset, 295 cm2 in Permanet and 152 cm2 in NetProtect in year 3, which means that most surviving nets were in the “damaged” category, but remained highly protective.

In addition, a simple non-inferiority test was conducted using WHO criteria to evaluate the effect of difference between products for mortality and feeding inhibition. Olyset® was used as the reference product (first in class or active comparator) against which the two other brands (second in class, test product or innovator product) were compared since it is the standard of care in Tanzania. PermaNet® 2.0 and Netprotect® were non-inferior compared to Olyset® on the feeding inhibition endpoint and superior to Olyset® on the 24-h mortality endpoint when measured in the I-ACT. The WHO passes a product based on a combination of mortality and feeding inhibition, and based on these criteria, PermaNet® 2.0 and Netprotect® were non-inferior to Olyset® based on data for three-year durability. This was also seen with WHO bioassays: Olyset® demonstrated lower mortality and similar feeding inhibition to PermaNet® 2.0 and Netprotect® when tested using cone tests and tunnel tests. Estimates of efficacy from the sample of 144 nets per brand were very precise and a 10% effect difference in mortality could be observed. However, it is unlikely that 144 nets per brand could be cost effectively evaluated in experimental huts. A comparison study between Ifakara experimental huts and the I-ACT using 24-h mortality and feeding inhibition outcome measures is in progress (Moore et al., pers. commun.) and will show how I-ACT and gold standard experimental huts compare for non-inferiority evaluation of ITNs. This is important since experimental huts are used to measure entomological correlates of the epidemiological effectiveness i.e. the public health benefit of interventions [47].

Therefore, the I-ACT could prove useful for testing insecticidal materials that can provide a high throughput option for evaluating functional bioefficacy of ITNs i.e. the true protection as a function of damage and bioavailability of insecticide in durability studies. Functional bioefficacy i.e. incorporating insecticidal effectiveness has also been suggested by WHO’s Malaria Policy Advisory committee to be included for net durability assessment [48]. While the methods presented here may not be useful for operational durability monitoring they may be useful for consideration in WHO “Phase 3” community field assessments of ITNs.

In this new assay, recapture of released mosquitoes is 99% so 30 mosquitoes were consistently “captured” every night I every chamber which is unlikely to be the case in standard experimental hut studies [45, 49,50,51,52,53,54,55,56,57]. Experimental hut studies rely on wild mosquitoes entering the hut, and the nightly number of mosquitoes captured is highly variable and consequently substantial replication is required to obtain adequate precision to estimate true effect differences between products [23]. As mosquito densities fluctuate due to seasonality in rainfall it is useful to have a whole net assay that is not dependent on field populations of mosquitoes that may limit the windows of opportunity to conduct tests with adequate mosquito densities to achieve power. Whole net bioassays where the interaction between insecticide and fabric integrity is measured are important for selecting between products or ranking their durability [48]. Bioassays that assess only the insecticidal bioefficacy of a net sample may favour poor quality nets that tear easily reducing user protection and consequently user acceptance, which will eventually lead to the user discarding the net [58].

The experimental hut bioassay that simulates domestic conditions and allows nets to be tested against wild mosquitoes is the definitive test of ITN efficacy [43]. This study had several limitations. Firstly, I-ACT uses laboratory-reared mosquitoes, which means it relies on laboratory strains that may have different resistance mechanisms to those locally or limited genetic diversity. Secondly, the I-ACT test is a more expensive infrastructure to establish compared to small WHO cones and WHO tunnel glass chambers, requires space and it is immovable. The assay must be conducted in climate-controlled chambers or in areas with suitable ambient conditions to conduct the tests. In contrast standard WHO cones and tunnel chambers which can be taken anywhere and tests conducted provided the environment is set to standard conditions for conducting tests. The I-ACT needs to be compared to experimental hut tests, but it did agree well with findings of standard WHO methods using pyrethroid susceptible mosquitoes. Evaluations of ITNs with pyrethroid resistant strains as well as using dual active ITNs will be reported in subsequent publications.

Based on the data here presented, the overnight I-ACT may be a bridge between the lab and the field. Data agreement with standard WHO testing methods was excellent, with high sensitivity and specificity. It allows mosquitoes to host seek during the active phase of the circadian rhythm, and have multiple contacts with treated netting in a more realistic way. It uses the preferred human host but allows laboratory-reared mosquitoes to be used. This improves safety for human volunteers because laboratory-reared mosquitoes are disease free and allows sufficient numbers of mosquitoes to be released to reach the power needed to conduct precise comparisons of product performance.

Conclusion

Findings from this study showed that, I-ACT can be used for high throughput evaluation of whole nets from ITN durability studies. The new assay may provide useful additional information and could act as a link between lab tests and field experiments measuring composite bioefficacy and net physical integrity with both mortality and feeding inhibition endpoints. For the three products evaluated in this study the bioassay agreed with standard WHO tests for deltamethrin products and measured higher pass for the permethrin-treated nets than standard WHO tests. I-ACT allows mosquitoes to interact with a preferred host sleeping under a net as it would be encountered in the field using a standard number of mosquitoes released to improve the precision of efficacy estimates and safety of human participants.

References

  1. 1.

    Dutta SN, Amon J, Iata H, Cooper RD, Russell TL. Long-term insecticidal activity and physical integrity of Olyset nets in Tafea Province, Vanuatu. J Med Entomol. 2014;51:164–9.

    Article  Google Scholar 

  2. 2.

    Tan KR, Coleman J, Smith B, Hamainza B, Katebe-Sakala C, Kean C, et al. A longitudinal study of the durability of long-lasting insecticidal nets in Zambia. Malar J. 2016;15:106.

    Article  Google Scholar 

  3. 3.

    Allan R, O’Reilly L, Gilbos V, Kilian A. An observational study of material durability of three World Health Organization-recommended long-lasting insecticidal nets in Eastern Chad. Am J Trop Med Hyg. 2012;87:407–11.

    Article  Google Scholar 

  4. 4.

    Wills AB, Smith SC, Anshebo GY, Graves PM, Endeshaw T, Shargie EB, et al. Physical durability of PermaNet 2.0 long-lasting insecticidal nets over three to 32 months of use in Ethiopia. Malar J. 2013;12:242.

    Article  Google Scholar 

  5. 5.

    Morgan J, Abilio AP, Rosario PM, Marrenjo D, Luciano J, Fernandes G, et al. Physical durability of two types of long-lasting insecticidal nets (LLINs) three years after a mass LLIN distribution campaign in Mozambique, 2008–2011. Am J Trop Med Hyg. 2015;92:286–93.

    Article  Google Scholar 

  6. 6.

    Kilian A, Byamukama W, Pigeon O, Atieli F, Duchon S, Phan C. Long-term field performance of a polyester-based long-lasting insecticidal mosquito net in rural Uganda. Malar J. 2008;7:49.

    Article  Google Scholar 

  7. 7.

    Randriamaherijaona S, Raharinjatovo J, Boyer S. Durability monitoring of long-lasting insecticidal (mosquito) nets (LLINs) in Madagascar: physical integrity and insecticidal activity. Parasit Vectors. 2017;10:564.

    Article  Google Scholar 

  8. 8.

    World Health Organization. WHOPES guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions. In: WHO/HTM/NTD/WHOPES. Geneva: World Health Organization; 2011. Accessed 30 Aug 2014.

  9. 9.

    WHO. Guidelines for laboratory and field testing of long-lasting insecticidal mosquito nets. Geneva: World Health Organization; 2005. http://whqlibdoc.who.int/hq/2005/WHO_CDS_WHOPES_GCDPP_2005.11.pdf. Accessed 4 Dec 2014.

  10. 10.

    World Health Organization. World malaria report; 2017. http://www.who.int/malaria/publications/world-malaria-report-2017/en/. Accessed 3 May 2018.

  11. 11.

    World Health Organization. MPAC-Proposed ERG on determining non-inferiority of insecticide-treated net and indoor residual spraying products within an established class. Geneva: World Health Organization; 2018. http://www.who.int/malaria/mpac/mpac-april2018-erg-non-inferiority-session3.pdf. Accessed 26 June 2018.

  12. 12.

    EMA Committee for Medicinal Products for Human Use (CHMP). Guideline on the Choice of the non-inferiority margin London; 2005. https://www.ema.europa.eu/documents/scientific-guideline/guideline-choice-non-inferiority-margin_en.pdf. Accessed 7 Mar 2018.

  13. 13.

    World Health Organization. Data requirements and protocol for determining non-inferiority of insecticide-treated net and indoor residual spraying products within an established WHO policy class. In: WHO/CDS/GMP/2018.22. Geneva: World Health Organization; 2018. http://www.who.int/malaria/publications/atoz/non-inferiority-protocol/en/. Accessed 27 Nov 2018.

  14. 14.

    World Health Organization. WHOPES guidelines for laboratory and field-testing of long-lasting insecticidal nets. In: Control of neglected tropical diseases. Geneva: World Health Organization; 2013. http://apps.who.int/iris/bitstream/handle/10665/80270/9789241505277_eng.pdf?sequence=1. Accessed 22 Feb 2018.

  15. 15.

    Rafinejad J, Vatandoost H, Nikpoor F, Abai MR, Shaeghi M, Duchen S, et al. Effect of washing on the bio-efficacy of insecticide-treated nets (ITNs) and long-lasting insecticidal nets (LLINs) against main malaria vector Anopheles stephensi by three bioassay methods. J Vector Borne Dis. 2008;45:143–50.

    CAS  PubMed  Google Scholar 

  16. 16.

    Vatandoost H, Dehakia H, Djavadia E, Abai MR, Duchon S. Comparative study on the efficacy of lambdacyhalothrin and bifenthrin on torn nets against the malaria vector, Anopheles stephensi as assessed by tunnel test method. J Vector Borne Dis. 2006;43:133–5.

    CAS  PubMed  Google Scholar 

  17. 17.

    Vatandoost H, Gholizadeh S, Abai MR, Djavadian E. Laboratory efficacy of protection rate of torn nets treated with pyrethroids, cyfluthrin, deltamethrin and permethrin against Anopheles stephensi (Diptera:Culicidae). J Biol Sci. 2006;6:331–6.

    CAS  Article  Google Scholar 

  18. 18.

    Moiroux N, Chandre F, Hougard JM, Corbel V, Pennetier C. Remote effect of Insecticide-treated nets and the personal protection against malaria mosquito bites. PLoS ONE. 2017;12:e073718.

    Article  Google Scholar 

  19. 19.

    Okumu FO, Moore J, Mbeyela E, Sherlock M, Sangusangu R, Ligamba G, et al. A modified experimental hut design for studying responses of disease-transmitting mosquitoes to indoor interventions: the Ifakara experimental huts. PLoS ONE. 2012;7:e30967.

    CAS  Article  Google Scholar 

  20. 20.

    Briet OJ, Hardy D, Smith TA. Importance of factors determining the effective lifetime of a mass, long-lasting, insecticidal net distribution: a sensitivity analysis. Malar J. 2012;11:20.

    Article  Google Scholar 

  21. 21.

    Lindblade KA, Dotson E, Hawley WA, Bayoh N, Williamson J, Mount D, et al. Evaluation of long-lasting insecticidal nets after 2 years of household use. Trop Med Int Health. 2005;10:1141–50.

    Article  Google Scholar 

  22. 22.

    Gillies MT. Selection for host preference in Anopheles gambiae. Nature. 1964;203:852–4.

    CAS  Article  Google Scholar 

  23. 23.

    Johnson P, Barry S, Furgerson H, Muller P. Power analysis for generalized linear mixed models in ecology and evolution. Methods Ecol Evol. 2014;6:133–42.

    Article  Google Scholar 

  24. 24.

    Lorenz LM, Overgaard HJ, Massue DJ, Mageni ZD, Bradley J, Moore JD, et al. Investigating mosquito net durability for malaria control in Tanzania—attrition, bioefficacy, chemistry, degradation and insecticide resistance (ABCDR): study protocol. BMC Public Health. 2014;14:1266.

    Article  Google Scholar 

  25. 25.

    World Health Organization. WHOPES Guidelines for laboratory and field testing of long-lasting insecticidal mosquito nets. In: WHO/HTM/NTD/WHOPES. Geneva: World Health Organization; 2005. Accessed 17 Oct 2014.

  26. 26.

    Benedict MQ. The MR4 Methods in Anopheles research laboratory manual. Atlanta: CDC; 2007. https://www.beiresources.org/portals/2/MR4. Accessed 30 May 2018.

  27. 27.

    World Health Organization. WHOPES recommended long-lasting insecticidal mosquito nets. Geneva: World Health Organization; 2012. http://www.who.int/whopes/Long_lasting_insecticidal_nets_Jul_2012.pdf. Accessed 15 Nov 2017.

  28. 28.

    Owusu HF, Müller P. How important is the angle of tilt in the WHO cone bioassay? Malar J. 2016;15:243.

    Article  Google Scholar 

  29. 29.

    Schumi J, Wittes JT. Through the looking glass: understanding non-inferiority. Trials. 2011;12:106.

    Article  Google Scholar 

  30. 30.

    Angarita-Jaimes NC, Parker JEA, Abe M, Mashauri F, Martine J, Towers CE, et al. A novel video-tracking system to quantify the behaviour of nocturnal mosquitoes attacking human hosts in the field. J R Soc Interface. 2016;13:20150974.

    Article  Google Scholar 

  31. 31.

    Parker J, Angarita-Jaimes NC, Abe M, Towers CE, Towers D, McCall P. Infrared video tracking of Anopheles gambiae at insecticide-treated bed nets reveals rapid decisive impact after brief localised net contact. Sci Rep. 2015;5:13392.

    Article  Google Scholar 

  32. 32.

    Parker JEA, Angarita Jaimes NC, Gleave K, Mashauri F, Abe M, Martine J, et al. Host-seeking activity of a Tanzanian population of Anopheles arabiensis at an insecticide treated bed net. Malar J. 2017;16:270.

    Article  Google Scholar 

  33. 33.

    Hougard JM, Duchon S, Darriet F, Zaim M, Rogier C, Guillet P. Comparative performances, under laboratory conditions, of seven pyrethroid insecticides used for impregnation of mosquito nets. Bull World Health Org. 2003;81:324–33.

    PubMed  Google Scholar 

  34. 34.

    Malima RC, Magesa SM, Tungu PK, Mwingira V, Magogo FS, Sudi W, et al. An experimental hut evaluation of Olyset® nets against anopheline mosquitoes after seven years use in Tanzanian villages. Malar J. 2008;7:38.

    Article  Google Scholar 

  35. 35.

    Hossain MI, Curtis CF. Permethrin-impregnated bednets: behavioural and killing effects on mosquitoes. Med Vet Entomol. 1989;3:367–76.

    CAS  Article  Google Scholar 

  36. 36.

    N’Guessan R, Odjo A, Ngufor C, Malone D, Rowland M. A Chlorfenapyr mixture Net Interceptor® G2 shows high efficacy and wash durability against resistant mosquitoes in West Africa. PLoS ONE. 2016;11:e0165925.

    Article  Google Scholar 

  37. 37.

    Diatta M, Spiegel A, Lochouarn L, Fontenille D. Similar feeding preferences of Anopheles gambiae and A. arabiensis in Senegal. Trans R Soc Trop Med Hyg. 1998;92:270–2.

    CAS  Article  Google Scholar 

  38. 38.

    Curtis CF, Lines JD, Ijumba J, Callaghan A, Hill N, Karimzad MA. The relative efficacy of repellents against mosquito vectors of disease. Med Vet Entomol. 1987;1:109–19.

    CAS  Article  Google Scholar 

  39. 39.

    Ngufor C, Critchley J, Fagbohoun J, N’Guessan R, Todjinou D, Rowland M. Chlorfenapyr (a pyrrole insecticide) applied alone or as a mixture with alpha-cypermethrin for indoor residual spraying against pyrethroid resistant Anopheles gambiae s.l.: an experimental hut study in Cove, Benin. PLoS ONE. 2016;11:e0162210.

    Article  Google Scholar 

  40. 40.

    N’Guessan R, Boko P, Odjo A, Knols B, Akogbeto M, Rowland M. Control of pyrethroid-resistant Anopheles gambiae and Culex quinquefasciatus mosquitoes with chlorfenapyr in Benin. Trop Med Int Health. 2009;14:389–95.

    Article  Google Scholar 

  41. 41.

    Black BC, Hollingworth RM, Ahammadsahib KI, Kukel CD, Donovan S. Insecticidal action and mitochondrial uncoupling activity of AC-303,630 and related halogenated pyrroles. Pestic Biochem Physiol. 1994;50:115–28.

    CAS  Article  Google Scholar 

  42. 42.

    Balmert NJ, Rund SSC, Ghazi JP, Zhou P, Duffield GE. Time-of-day specific changes in metabolic detoxification and insecticide resistance in the malaria mosquito Anopheles gambiae. J Insect Physiol. 2014;64:30–9.

    CAS  Article  Google Scholar 

  43. 43.

    Oxborough RM, N’Guessan R, Jones R, Kitau J, Ngufor C, Malone D, et al. The activity of the pyrrole insecticide chlorfenapyr in mosquito bioassay: towards a more rational testing and screening of non-neurotoxic insecticides for malaria vector control. Malar J. 2015;14:124.

    Article  Google Scholar 

  44. 44.

    De Amici D, Klersy C, Ramajoli F, Brustia L, Politi P. Impact of the Hawthorne effect in a longitudinal clinical study: the case of anesthesia. Control Clin Trials. 2000;21:103–14.

    Article  Google Scholar 

  45. 45.

    Maxwell CA, Myamba J, Magoma J, Rwegoshora RT, Magesa SM, Curtis CF. Tests of Olyset nets by bioassay and in experimental huts. J Vector Borne Dis. 2006;43:1–6.

    CAS  PubMed  Google Scholar 

  46. 46.

    Tami A, Mubyazi G, Talbert A, Mshinda H, Duchon S, Lengeler C. Evaluation of Olyset™ insecticide-treated nets distributed seven years previously in Tanzania. Malar J. 2004;3:19.

    Article  Google Scholar 

  47. 47.

    Sherrard-Smith E, Griffin JT, Winskill P, Corbel V, Pennetier C, Djénontin A, et al. Systematic review of indoor residual spray efficacy and effectiveness against Plasmodium falciparum in Africa. Nat Commun. 2018;9:4982.

    Article  Google Scholar 

  48. 48.

    World Health Organization. Vector control technical expert group report to MPAC. Estimating functional survival of long-lasting insecticidal nets from field data. Geneva: World Health Organization; 2013. http://www.who.int/malaria/mpac/mpac_sep13_vcteg_llin_survival_report.pdf. Accessed 31 Oct 2014.

  49. 49.

    Oxborough RM, Kitau J, Jones R, Mosha FW, Rowland MW. Experimental hut and bioassay evaluation of the residual activity of a polymer-enhanced suspension concentrate (SC-PE) formulation of deltamethrin for IRS use in the control of Anopheles arabiensis. Parasit Vectors. 2014;7:454.

    Article  Google Scholar 

  50. 50.

    Asidi AN, N’Guessan R, Hutchinson RA, Traoré-Lamizana M, Carnevale P, Curtis CF. Experimental hut comparisons of nets treated with carbamate or pyrethroid insecticides, washed or unwashed, against pyrethroid-resistant mosquitoes. Med Vet Entomol. 2004;18:134–40.

    CAS  Article  Google Scholar 

  51. 51.

    Ngufor C, N’guessan R, Fagbohoun J, Odjo A, Malone D, Akogbeto M, et al. Olyset Duo® (a pyriproxyfen and permethrin mixture net): an experimental hut trial against pyrethroid resistant Anopheles gambiae and Culex quinquefasciatus in Southern Benin. PLoS ONE. 2014;9:e93603.

    Article  Google Scholar 

  52. 52.

    Darriet F, N’Guessan R, Hougard JM, Traoré-Lamizana M, Carnevale P. An experimental tool essential for the evaluation of insecticides: the testing huts. Bull Soc Pathol Exot. 2002;95:299–303 (in French).

    CAS  PubMed  Google Scholar 

  53. 53.

    Gunasekaran K, Sahu SS, Vijayakumar T, Subramanian S, Yadav RS, Pigeon O, et al. An experimental hut evaluation of Olyset Plus, a long-lasting insecticidal net treated with a mixture of permethrin and piperonyl butoxide, against Anopheles fluviatilis in Odisha State, India. Malar J. 2016;15:375.

    Article  Google Scholar 

  54. 54.

    Lines JD, Myamba J, Curtis CF. Experimental hut trials of permethrin-impregnated mosquito nets and eave curtains against malaria vectors in Tanzania. Med Vet Entomol. 1987;1:37–51.

    CAS  Article  Google Scholar 

  55. 55.

    Chareonviriyaphap T, Grieco JP, Suwonkerd W, Prabaripai A, Polsomboon S, Thainchum K, et al. An Improved experimental hut design for the study of Aedes aegypti (Diptera: Culicidae) movement patterns in Thailand. J Vector Ecol. 2010;35:428–31.

    Article  Google Scholar 

  56. 56.

    Silver JB, Service MW. Chapter 16 Experimental hut techniques. In: Silver JB, editor. Mosquito ecology field sampling methods. Berlin: Springer; 2008.

    Google Scholar 

  57. 57.

    Massue DJ, Kisinza WN, Malongo BB, Mgaya CS, Bradley J, Moore JD, et al. Comparative performance of three experimental hut designs for measuring malaria vector responses to insecticides in Tanzania. Malar J. 2016;15:165.

    Article  Google Scholar 

  58. 58.

    Koenker H, Kilian H, Beyl CZ, Onyefunafoa EO, Selbby RA, Abeku T, et al. What happens to lost nets: a multi-country analysis of reasons for LLIN attrition using 14 household surveys in four countries. Malar J. 2014;13:464.

    Article  Google Scholar 

  59. 59.

    World Health Organization. WHOPES guidance note for estimating the longevity of long-lasting insecticidal nets in malaria control. Geneva: World Health Organization; 2013. http://www.who.int/malaria/publications/atoz/who_guidance_longevity_llins.pdf. Accessed 15 Jan 2018.

Download references

Authors’ contributions

Conceived and designed the experiment DJM, SJM, JDM. Performed the experiments DJM, JDM, WSN, ZMM. Analysed the data DJM. Contributed to the analysis SJM, JB. Wrote the manuscript DJM, SJM. Critically revised the final manuscript LML, SM, EM, WNK, ZMM, JB, HJO, SJM. Drew the diagrams JDM, SA. All authors read and approved the final manuscript.

Acknowledgements

Our special thanks are addressed to all technical staff at Ifakara health Institute, at Kingani Insectary for conducting data collection in the laboratory and semi field experiments. We thank the ITN manufacturers Sumitomo Chemical (Olyset), Vestergaard Frandsen (PermaNet) and BestNet (Netprotect) for their donation of the ITNs free of charge. We would like to sincerely thank Professor Steve Lindsay for comments on an earlier draft of this manuscript that resulted in improvements to the text, analysis and data interpretation.

Competing interests

Sarah Moore conducts product evaluation for a number of vector control product manufacturers. The other authors declare that they have no competing interests.

Availability of data and materials

The datasets supporting the conclusions of this article are included within the article and available from the corresponding author on reasonable request.

Consent for publication

This paper is published with the permission of the Director-General of the National Institute for Medical Research (NIMR) in Tanzania.

Ethics approval and consent to participate

Participants involved in I-ACT were all Ifakara Health Institute (IHI) staff members who have received appropriate training and were experienced in conducting semi-field tunnel tests. Written informed consent was obtained from all sleeping volunteers. Ethical approvals was obtained from Ifakara Health Institute in Tanzania (Reference number—IHI/IRB/No: 19-2013) and from the National Institute for Medical Research (Ref: NIMR/HQ/R.8a/Vol. IX/150 and NIMR/HQ/R.8c/Vol. I/285). All persons involved in this project had medical insurance provided by the project in case of any adverse events associated with sleeping under the net. The animals used in WHO tunnel tests were cared for with consultation of registered project veterinary doctor.

Funding

The research was made possible by the Research Council of Norway through the ABCDR Project no. 220757.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Dennis J. Massue.

Additional files

Additional file 1.

Location of holes on the deliberately holed SAFI net.

Additional file 2.

A standardized operating procedure for conducting experiments to measure feeding inhibition and mortality of different net products using the I-ACT.

Additional file 3.

Sampling patterns of the net for cone bioassay and tunnel test.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Massue, D.J., Lorenz, L.M., Moore, J.D. et al. Comparing the new Ifakara Ambient Chamber Test with WHO cone and tunnel tests for bioefficacy and non-inferiority testing of insecticide-treated nets. Malar J 18, 153 (2019). https://doi.org/10.1186/s12936-019-2741-y

Download citation

Keywords

  • Biological efficacy
  • WHO cone test
  • WHO tunnel test
  • I-ACT
  • Ifakara Ambient Chamber Test
  • Long lasting insecticidal nets
  • Durability
  • Non-inferiority