Comparison of diagnostic methods for the detection and quantification of the four sympatric Plasmodium species in field samples from Papua New Guinea
© Rosanas-Urgell et al; licensee BioMed Central Ltd. 2010
Received: 14 September 2010
Accepted: 14 December 2010
Published: 14 December 2010
Accurate diagnosis of Plasmodium infections is essential for malaria morbidity and mortality reduction in tropical areas. Despite great advantages of light microscopy (LM) for malaria diagnosis, its limited sensitivity is a critical shortfall for epidemiological studies. Robust molecular diagnostics tools are thus needed.
The present study describes the development of a duplex quantitative real time PCR (qPCR) assay, which specifically detects and quantifies the four human Plasmodium species. Performance of this method was compared to PCR-ligase detection reaction-fluorescent microsphere assay (PCR_LDR_FMA), nested PCR (nPCR) and LM, using field samples collected from 452 children one to five years of age from the Sepik area in Papua New Guinea. Agreement between diagnostic methods was calcualted using kappa statistics.
The agreement of qPCR with other molecular diagnostic methods was substantial for the detection of P. falciparum, but was moderate for the detection of P. vivax, P. malariae and P. ovale. P. falciparum and P. vivax prevalence by qPCR was 40.9% and 65.7% respectively. This compares to 43.8% and 73.2% by nPCR and 47.1% and 67.5% by PCR_LDR_FMA. P. malariae and P. ovale prevalence was 4.7% and 7.3% by qPCR, 3.3% and 3.8% by nPCR, and 7.7% and 4.4% by PCR_LDR_FMA. Prevalence by LM was lower for all four species, being 25.4% for P. falciparum, 54.9% for P. vivax, 2.4% for P. malariae and 0.0% for P. ovale. The quantification by qPCR closely correlated with microscopic quantification for P. falciparum and P. vivax samples (R2 = 0.825 and R2 = 0.505, respectively). The low prevalence of P. malariae and P. ovale did not permit a solid comparative analysis of quantification for these species.
The qPCR assay developed proved optimal for detection of all four Plasmodium species. Densities by LM were well reflected in quantification results by qPCR, whereby congruence was better for P. falciparum than for P. vivax. This likely is a consequence of the generally lower P. vivax densities. Easy performance of the qPCR assay, a less laborious workflow and reduced risk of contamination, together with reduced costs per sample through reduced reaction volume, opens the possibility to implement qPCR in endemic settings as a suitable diagnostic tool for large epidemiological studies.
Rapid and accurate diagnosis of Plasmodium infections is crucial for morbidity and mortality reduction in tropical areas, specially in regions where mixed infections are prevalent such as Papua New Guinea (PNG), where all four parasites infecting humans coexist and mixed species infections are common [1, 2]. For improving accuracy in large epidemiological studies, molecular diagnostic tools permitting high through put analysis for the identification and quantification of malaria parasites would be of great benefit.
Traditionally, light microscopy (LM) examination of blood smears has been considered the gold standard for the diagnosis of malaria . LM has clear advantages over molecular typing, since it incurs only low costs, does neither need complex sample preparation nor advanced technology and permits species identification and quantification . However, the role of LM as a gold standard has been questioned due to false negative results at low parasitemia and frequent errors in species identification in mixed infections [5, 6].
The availability of commercial rapid diagnostic tests (RDT) has greatly facilitated the in situ diagnosis of malaria infections in the field. The advantages of RDT are that they do not need special preparation of the sample and diagnostic results become immediately available . However, their use is limited due to lack of sensitivity for Plasmodium vivax, Plasmodium malariae and Plasmodium ovale.
Nested PCR (nPCR), first described by Snounou and co-workers , is a widely used method and is considered as a molecular gold standard due to its good performance in detection of mixed species infection. This assay amplifies the multicopy 18 S rRNA genes of the four Plasmodium species infecting humans. Even though this genotyping technique is now performed in many field laboratories in endemic countries, its use for routine clinical diagnosis is limited, because the analysis is time consuming due to the need of multiple reactions per sample, and the risk of contamination through the requirement for nPCR . Moreover, the technique is not quantitative.
In recent years, various real time quantitative PCR (qPCR) assays have been developed for the detection of Plasmodium species, with most assays targeting the 18 S rRNA genes. Two of these methods detect the genus Plasmodium using generic primers, and thus do not distinguish between species [10, 11]. Sybergreen reagent has been used by other groups, to identify the four Plasmodium species infecting humans in a single reaction by melting curve analysis [8, 12]. The use of TaqMan probes contributed an additional level of specificity to qPCR assays . However, using a single pair of primers for Plasmodium genus detection in duplex assays, introduces competition for amplification among species, which likely leads to difficulties in detecting mixed infections . In order to address this issue, various groups had used different strategies [15–18]. The multiplex PCR-ligase detection reaction-fluorescent microsphere assay (PCR_LDR_FMA) has also been used in molecular epidemiological studies for simultaneously detecting all four Plasmodium species .
Despite the variety of molecular tools available for the diagnosis of malaria and their wide use for the diagnosis of imported malaria in travel clinics, implementation of these techniques in endemic areas has remained limited until now. Even though the low sensitivity and limited detection of asymptomatic and mixed infections by LM constitutes a critical shortfall for some epidemiological studies, LM remains to date the most frequently used method for the diagnosis of malaria in endemic areas. The transfer of molecular techniques for diagnosis of malaria to laboratories in endemic settings is essential for overcoming the limitations by LM. Moreover, a molecular technique with quantification capacity contributes to correctly estimate the burden of Plasmodium species often found in concomitant infections and will be a valuable tool to explore competition in mixed infections.
A qPCR assay initially developed for malaria diagnosis in returning travellers at a reference laboratory was implemented and validated at the PNG Institute of Medical Research (IMR). This assay was chosen because it detects with high specificity all four Plasmodium species which jointly occur in our study area in PNG. The performance of this qPCR assay in conditions of a field laboratory and on field samples was compared to light microscopy, nPCR and PCR_LDR_FMA results.
Study site and sample collection
Samples were collected in 10 villages from Ilahita area of East Sepik Province, PNG , in the framework of a longitudinal cohort study conducted during 2008. As part of the baseline, venous blood samples were obtained from 452 children one to five years of age, after written informed consent was obtained from parents or legal guardians of each child. This genotyping study was approved by the PNG IMR Institutional Review Board (IMR IRB 0720) and PNG Medical Research Advisory Committee (MRAC 07.34).
Blood smear examination
Thick/thin blood smears were prepared as described previously [20, 21]. Blood smears were stained with a 5% Giemsa solution and examined independently by two microscopists, with a third microscopist reading for those slides with discrepant results. A minimum of 200 microscopic fields were examined at a magnification of 1000× using oil immersion optics before a slide was declared negative for malaria parasites by LM. Plasmodium species were quantified by counting infected parasites over 200 leukocytes. Conversion of parasite counts into parasites/μL was performed assuming a mean leukocyte count of 8000/μL whole blood. Routinely slides were read twice. Discordant results were evaluated by a third slide reading. Final species diagnosis was based on the majority agreement.
DNA template extraction and amplification
DNA was extracted from 200 μL whole blood (venous blood collected in EDTA anti-coagulant) using QIAamp 96 DNA Blood Mini Kit (QIAGEN, Valencia, CA), and eluted in a final volume of 200 μL dH2O according to the supplier's instructions. nPCR was carried out as described .
PCR_LDR_FMA was carried out as described elsewhere . Mixes of serially diluted plasmids containing inserts of P. falciparum, P. vivax, P. malariae or P. ovale 18 S rDNA were used as positive controls in addition to P. falciparum or P. vivax positive samples obtained from field isolates. The threshold for positivity for each species was determined using the mean value obtained from negative controls for each species, plus three times the standard deviation.
Primers and probes for qPCR
Evaluation of PCR efficiency and reproducibility was performed on standard curves using four positive control plasmids with the respective amplicons inserted. Geometric mean and standard deviation were calculated from triplicates in three independent assays. Standard curve for each Plasmodium species was made from a 10-fold serial dilution of the control plasmids ranging from 106 copies/μL to 10 copies/μL.
Amplification efficiencies for the different primer pairs and probes were calculated with the formula:
Efficiency = 10(-1/Slope) -1. Inter-assay coefficients of variation (CVs) were calculated for each plasmid dilution separately as (SD/mean)*100 using the Ct values from different runs.
Reproducibility of qPCR was further analysed by repeating 10% of all 452 field-samples in the same laboratory, but at different time points. External quality assurance was performed in addition.
The amount of target in an unknown sample was quantified by converting the threshold cycle (Ct) into template copy number by using the four standard curves. Samples yielding Ct values equal or higher than 40 were considered Plasmodium species negative .
Results from qPCR was compared with those from LM, nPCR and PCR-LDR-FMA. Agreement between diagnostic methods was determined by calculating Kappa Statistics with 95% confidence intervals. Values were interpreted with the Landis and Koch classification  as follows: k = 0.41-0.60, moderate agreement, k = 0.61-0.80, good agreement; k = 0.81-1.00, almost perfect agreement beyond chance. Prevalence for each species was calculated by dividing the number of positive samples through by the number of all samples tested and McNemar test was calculated in order to test for significance. P values <0.05 were considered statistically significant. Correlation between LM and qPCR quantification was calculated by pairwise correlation coefficient. All statistical calculations were performed with STATAv.10 statistical software.
Optimization of qPCR assay
The large scale of field work required optimization of molecular testing with the aim to reduce costs. Two strategies were applied: multiplexing the qPCR and reducing the reaction volume. The performance of the two duplex qPCRs were evaluated by analysing dilution series of the two respective control plasmids in various ratios (Additional file 1). These experiments demonstrated very little inhibition when 200 templates and above were present in the reaction. Only the detection of minute template concentrations, as low as 20 templates per reaction, was slightly inhibited by excess amounts of the alternative template, i.e. in a 100 times excess.
qPCR Efficiencies for each Plasmodium species
Reproducibility of our assay was investigated by re-analysing 10% of the 452 samples in the same laboratory. In 8/45 compared samples discordant results were recorded. Four of these discrepancies occurred in cases of low density in the range of 10-100 copies/μL. Ct values were between 37 and 39 in the positive ones of the duplicate tests. Two other discrepancies were observed in mixed species infections, whereby the minority species was detected in only one of the duplicate experiments in the range of 10 to 100 copies/μL.
Comparison of diagnosis of Plasmodium infection by qPCR, nPCR, PCR-LDR and LM
P. falciparum versus P. vivax agreement among different diagnostic techniques
P. malariae versus P. ovale agreement among different diagnostic techniques
Quantification of Plasmodium parasites
In preparation of major molecular epidemiological field studies in PNG essential parasite detection techniques were compared under conditions of a laboratory close to the field site and located in a malaria endemic country. The diagnostic requirements were: (i) good performance in the detection of mixed species infections, as all four species concurrently occur in PNG, (ii) recognition of P. malariae and P. ovale variants present in the study area, (iii) high through put capacity and robustness of assay, (iv) quantitative results and (v) reasonable costs. The qPCR assay described here was implemented and validated at the PNG-IMR site in Madang, demonstrating the feasibility of applying state of the art techniques in this context. In the meantime the qPCR assay is routinely implemented for molecular diagnosis in large scale epidemiologic studies at IMR.
As part of test validation in the field, the performance of this qPCR assay for Plasmodium species discrimination was compared to two other PCR-based assays (nPCR and LDR) and to LM. Traditionally, test outcomes for different assays are compared to an established 'gold-standard' in order to calculate sensitivity and specificity estimates and to evaluate the performance of newly developed tests. The classical 'gold standard' for malaria diagnosis has been LM , however, with the appearance of new molecular diagnostic tools, LM has become less suitable for this purpose due to its lower sensitivity than molecular methods . Even though the nPCR developed by Snounou et al  has been extensively used as 'gold-standard' for molecular diagnosis [25, 26], the concept of using a 'gold-standard' for the evaluation of new assays is being questioned by various authors, which alternatively propose the use of 'non-gold standard' approaches [27, 28].
The agreement between qPCR and the other techniques was substantial for P. falciparum, but only moderate for P. vivax, P. malariae and P. ovale. In particular, the agreement between qPCR and nPCR for P. falciparum detection was almost perfect. The lower agreement between PCR-LDR and nPCR, together with the higher prevalence shown by PCR-LDR (47.1% compared to 40.9% by qPCR and 43.8% by nPCR), may indicate false positive results by LDR. This is supported by our pairwise analysis and the agreement of two independent PCR based assays, namely nPCR and qPCR. However, in absence of a suitable diagnostic 'gold standard', it remains unclear if those 33 samples positive for LDR but negative by the two alternative molecular methods, represent a greater sensitivity of LDR or simply false positives. This issue cannot be easily resolved in a study involving 'unknown' samples from the field, potentially infected by four different Plasmodium species.
P. vivax prevalence was higher by nPCR than by both, qPCR and PCR-LDR (73.2% by nPCR, 65.7% by qPCR and 67.5% by PCR-LDR). This again could reflect false postitives by nPCR or lower sensitivity by both other molecular methods. Our observations in qPCR validation using plasmid template suggested that qPCR of P. vivax is lightly compromised by performing a duplex Pf/Pv reaction. nPCR involves a very high number of cycles (55 cycles by nPCR versus 45 cycles by qPCR and 35 cycles by LDR), and therefore is expected to show maximal sensitivity. Despite measurements taken over 45 cycles in qPCR, we followed the consensus rule for considering a sample positive, i.e. a Ct value < 40 . In our samples this led to the loss of 9 samples with Ct values for P. vivax between 40 and 43.6 cycles, which otherwise would have increased the sensitivity of the assay. Further analysis was performed on samples with discrepant results for P. vivax (P. vivax negative samples by qPCR and positive by nPCR). Most of these samples were mixed infections by nPCR and harboured P. falciparum with more than 10,000 copies/μl. Thus competition for amplification at the beginning of the PCR due to P. falciparum high densities may be precluding P. vivax detection . 14/16 of the remaining samples were also negative by LM. Most likely these very low-grade P. vivax infections were missed. The scarcity of the template in case of a very low parasite density is expected to lead to imperfect detection. Prevalence for P. malariae and P. ovale were low with significant differences between assays, even though the agreement between pairwise compaired methods was moderate. Higher prevalence for P. malariae detection by LDR is likely to occur as a result of false positive results, probably occurring due to high background noise of the P. malariae probe used in the assay. Low detection of P. ovale by nPCR (3.8%) is due to the use of a primer pair with sub-optimal amplification of P. ovale sequences present in the study area. Finally, LM measured the lowest prevalence for all four Plasmodium species.
The major advantage of qPCR over the other compared molecular techniques was the quantification of parasite densities. Parasite densities shown as copies of 18 S rRNA template/μL were quantified by converting the threshold cycle (Ct) into template copy number by using the standard curves. When correlating quantification by qPCR with LM counts in samples where both techniques showed positive results, a high correlation for P. falciparum (R2 = 0.8253) and a lower correlation for P. vivax (R2 = 0.5049) was found. But for P. vivax this correlation of parasite densities by qPCR and LM increased when only single infections were taken into account. Therefore, our results suggest a variation in the detection limit in both methods, due to overlooking P. vivax in case of an overwhelming P. falciparum infection. Difficulties in identifying P. vivax by LM arise when this parasite is found at low densities and in mixed infections. The high P. falciparum densities found in the samples identified as mixed infection by qPCR (> 10000 target copies/μl) further supports this explanation. The correlation for P. malariae and P. ovale could not be analysed due to poor detection of both species by LM.
The qPCR assay was found optimal for both tasks, detection of all four Plasmodium species and quantification. The latter could only be analyzed for P. falciparum and P. vivax. Overall qPCR shows substantial agreement with other molecular techniques for the detecting prevalence of P. falciparum and P. vivax, while moderate agreement was observed for P. malariae and P. ovale. It is clear, that sensitivity of our qPCR assay can be increased by simply performing independent reactions of each Plasmodium species. However, this would substantially increase costs. Limiting factors, such as duplex assays, need to be balanced against costs or work load. The specific research objectives of a particular study should guide the choice of experimental procedures.
Overall, the superior performance of PCR based methodologies over LM has been clearly demonstrated by these results and others. In a recent study conducted in Benin, a high number of children (between 27% and 44%) aged 5 or above, who initially had negative RDT tests (most also with negative blood slides), were later found to be infected with P. falciparum using PCR . These undetected submicroscopic infections have an enormous impact for malaria transmission in endemic areas. In a time where malaria erradication has become the primary goal of malaria agendas, the accurate estimation of the burden of malaria infection is imperative to control transmission.
In conclusion, this qPCR assay was sensitive and specific for the detection of all four Plasmodium species and results agreed well with other molecular techniques tested. Added advantage of the qPCR assay is quantification of parasite densities and a less laborious workflow. Moreover, the assay performed well in field samples and due to its high through put capacity, it is suitable for large scale epidemiological studies. Finally, a quantitative assay is of a greatest value for monitoring of malaria control programmes.
We are particularly thankful to all the children of Ilahita area, their parents and/or guardians who participated in the study. We thank the staff from the Institute of Medical Research in Maprik for their assistance in collecting and processing blood samples, to the staff from the Institute of Medical Research in Madang for malaria microscopy readings and to Nomin-Dora Tenakanai for her help with the Bioplex analysis. This investigation received financial support from the Fundación Privada CELLEX (Catalonia, Spain).
- Mehlotra RK, Kasehagen LJ, Baisor M, Lorry K, Kazura JW, Bockarie MJ, Zimmerman PA: Malaria infections are randomly distributed in diverse holoendemic areas of Papua New Guinea. Am J Trop Med Hyg. 2002, 67: 555-562.PubMed CentralPubMedGoogle Scholar
- Mueller I, Galinski MR, Baird JK, Carlton JM, Kochar DK, Alonso PL, del Portillo HA: Key gaps in the knowledge of Plasmodium vivax, a neglected human malaria parasite. Lancet Infect Dis. 2009, 9: 555-566. 10.1016/S1473-3099(09)70177-X.View ArticlePubMedGoogle Scholar
- Organization WH: World malaria report 2008. 2008, Geneva, Switzerland: WHO PressGoogle Scholar
- Moody AH, Chiodini PL: Non-microscopic method for malaria diagnosis using OptiMAL IT, a second-generation dipstick for malaria pLDH antigen detection. Br J Biomed Sci. 2002, 59: 228-231.PubMedGoogle Scholar
- Coleman RE, Sattabongkot J, Promstaporm S, Maneechai N, Tippayachai B, Kengluecha A, Rachapaew N, Zollner G, Miller RS, Vaughan JA, Thimasarn K, Khuntirat B: Comparison of PCR and microscopy for the detection of asymptomatic malaria in a Plasmodium falciparum/vivax endemic area in Thailand. Malar J. 2006, 5: 121. 10.1186/1475-2875-5-121.PubMed CentralView ArticlePubMedGoogle Scholar
- Ochola LB, Vounatsou P, Smith T, Mabaso ML, Newton CR: The reliability of diagnostic techniques in the diagnosis and management of malaria in the absence of a gold standard. Lancet Infect Dis. 2006, 6: 582-588. 10.1016/S1473-3099(06)70579-5.View ArticlePubMedGoogle Scholar
- WHO: Malaria rapid diagnostic test performance: results of WHO product testing of malaria RDTs: round 1 (2008). 2009, Geneva: World Health OrganizationView ArticleGoogle Scholar
- de Monbrison F, Angei C, Staal A, Kaiser K, Picot S: Simultaneous identification of the four human Plasmodium species and quantification of Plasmodium DNA load in human blood by real-time polymerase chain reaction. Trans R Soc Trop Med Hyg. 2003, 97: 387-390. 10.1016/S0035-9203(03)90065-4.View ArticlePubMedGoogle Scholar
- Rubio JM, Benito A, Roche J, Berzosa PJ, Garcia ML, Mico M, Edu M, Alvar J: Semi-nested, multiplex polymerase chain reaction for detection of human malaria parasites and evidence of Plasmodium vivax infection in Equatorial Guinea. Am J Trop Med Hyg. 1999, 60: 183-187.PubMedGoogle Scholar
- Lee MA, Tan CH, Aw LT, Tang CS, Singh M, Lee SH, Chia HP, Yap EP: Real-time fluorescence-based PCR for detection of malaria parasites. J Clin Microbiol. 2002, 40: 4343-4345. 10.1128/JCM.40.11.4343-4345.2002.PubMed CentralView ArticlePubMedGoogle Scholar
- Farcas GA, Zhong KJ, Mazzulli T, Kain KC: Evaluation of the RealArt Malaria LC real-time PCR assay for malaria diagnosis. J Clin Microbiol. 2004, 42: 636-638. 10.1128/JCM.42.2.636-638.2004.PubMed CentralView ArticlePubMedGoogle Scholar
- Mangold KA, Manson RU, Koay ES, Stephens L, Regner M, Thomson RB, Peterson LR, Kaul KL: Real-time PCR for detection and identification of Plasmodium spp. J Clin Microbiol. 2005, 43: 2435-2440. 10.1128/JCM.43.5.2435-2440.2005.PubMed CentralView ArticlePubMedGoogle Scholar
- Rougemont M, Van Saanen M, Sahli R, Hinrikson HP, Bille J, Jaton K: Detection of four Plasmodium species in blood from humans by 18 S rRNA gene subunit-based and species-specific real-time PCR assays. J Clin Microbiol. 2004, 42: 5636-5643. 10.1128/JCM.42.12.5636-5643.2004.PubMed CentralView ArticlePubMedGoogle Scholar
- Bialasiewicz S, Whiley DM, Nissen MD, Sloots TP: Impact of competitive inhibition and sequence variation upon the sensitivity of malaria PCR. J Clin Microbiol. 2007, 45: 1621-1623. 10.1128/JCM.02145-06.PubMed CentralView ArticlePubMedGoogle Scholar
- Perandin F, Manca N, Calderaro A, Piccolo G, Galati L, Ricci L, Medici MC, Arcangeletti MC, Snounou G, Dettori G: Development of a real-time PCR assay for detection of Plasmodium falciparum, Plasmodium vivax, and Plasmodium ovale for routine clinical diagnosis. J Clin Microbiol. 2004, 42: 1214-1219. 10.1128/JCM.42.3.1214-1219.2004.PubMed CentralView ArticlePubMedGoogle Scholar
- Shokoples SE, Ndao M, Kowalewska-Grochowska K, Yanow SK: Multiplexed real-time PCR assay for discrimination of Plasmodium species with improved sensitivity for mixed infections. J Clin Microbiol. 2009, 47: 975-980. 10.1128/JCM.01858-08.PubMed CentralView ArticlePubMedGoogle Scholar
- Dormond L, Jaton-Ogay K, de Valliere S, Genton B, Bille J, Greub G: Multiplex real-time PCR for the diagnosis of malaria: correlation with microscopy. Clin Microbiol Infect. 2010,Google Scholar
- Vo TK, Bigot P, Gazin P, Sinou V, De Pina JJ, Huynh DC, Fumoux F, Parzy D: Evaluation of a real-time PCR assay for malaria diagnosis in patients from Vietnam and in returned travellers. Trans R Soc Trop Med Hyg. 2007, 101: 422-428. 10.1016/j.trstmh.2006.09.004.View ArticlePubMedGoogle Scholar
- McNamara DT, Kasehagen LJ, Grimberg BT, Cole-Tobian J, Collins WE, Zimmerman PA: Diagnosing infection levels of four human malaria parasite species by a polymerase chain reaction/ligase detection reaction fluorescent microsphere-based assay. Am J Trop Med Hyg. 2006, 74: 413-421.PubMed CentralPubMedGoogle Scholar
- Genton B, al-Yaman F, Beck HP, Hii J, Mellor S, Rare L, Ginny M, Smith T, Alpers MP: The epidemiology of malaria in the Wosera area, East Sepik Province, Papua New Guinea, in preparation for vaccine trials. II. Mortality and morbidity. Ann Trop Med Parasitol. 1995, 89: 377-390.PubMedGoogle Scholar
- al-Yaman F, Genton B, Kramer KJ, Taraika J, Chang SP, Hui GS, Alpers MP: Acquired antibody levels to Plasmodium falciparum merozoite surface antigen 1 in residents of a highly endemic area of Papua New Guinea. Trans R Soc Trop Med Hyg. 1995, 89: 555-559. 10.1016/0035-9203(95)90106-X.View ArticlePubMedGoogle Scholar
- Snounou G, Viriyakosol S, Jarra W, Thaithong S, Brown KN: Identification of the four human malaria parasite species in field samples by the polymerase chain reaction and detection of a high prevalence of mixed infections. Mol Biochem Parasitol. 1993, 58: 283-292. 10.1016/0166-6851(93)90050-8.View ArticlePubMedGoogle Scholar
- Bustin SA: Why the need for qPCR publication guidelines?--The case for MIQE. Methods. 50: 217-226. 10.1016/j.ymeth.2009.12.006.Google Scholar
- Landis JR, Koch GG: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977, 33: 363-374. 10.2307/2529786.View ArticlePubMedGoogle Scholar
- Mens P, Spieker N, Omar S, Heijnen M, Schallig H, Kager PA: Is molecular biology the best alternative for diagnosis of malaria to microscopy? A comparison between microscopy, antigen detection and molecular tests in rural Kenya and urban Tanzania. Trop Med Int Health. 2007, 12: 238-244.PubMedGoogle Scholar
- Mixson-Hayden T, Lucchi NW, Udhayakumar V: Evaluation of three PCR-based diagnostic assays for detecting mixed Plasmodium infection. BMC Res Notes. 2010, 3: 88. 10.1186/1756-0500-3-88.PubMed CentralView ArticlePubMedGoogle Scholar
- Koukounari A, Webster JP, Donnelly CA, Bray BC, Naples J, Bosompem K, Shiff C: Sensitivities and specificities of diagnostic tests and infection prevalence of Schistosoma haematobium estimated from data on adults in villages northwest of Accra, Ghana. Am J Trop Med Hyg. 2009, 80: 435-441.PubMed CentralPubMedGoogle Scholar
- de Clare Bronsvoort BM, von Wissmann B, Fevre EM, Handel IG, Picozzi K, Welburn SC: No gold standard estimation of the sensitivity and specificity of two molecular diagnostic protocols for Trypanosoma brucei spp. in Western Kenya. PLoS One. 2010, 5: e8628. 10.1371/journal.pone.0008628.PubMed CentralView ArticlePubMedGoogle Scholar
- Faucher JF, Aubouy A, Beheton T, Makoutode P, Abiou G, Doritchamou J, Houze P, Ouendo E, Deloron P, Cot M: What would PCR assessment change in the management of fevers in a malaria endemic area? A school-based study in Benin in children with and without fever. Malar J. 2010, 9: 224. 10.1186/1475-2875-9-224.PubMed CentralView ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.