Reader technique as a source of variability in determining malaria parasite density by microscopy
© O'Meara et al; licensee BioMed Central Ltd. 2006
Received: 06 September 2006
Accepted: 12 December 2006
Published: 12 December 2006
Accurate identification and quantification of malaria parasites are critical for measuring clinical trial outcomes. Positive and negative diagnosis is usually sufficient for the assessment of therapeutic outcome, but vaccine or prophylactic drug trials require measuring density of infection as a primary endpoint. Microscopy is the most established and widely-used technique for quantifying parasite densities in the blood.
Results obtained by 24–27 expert malaria microscopists, who had independently read 895 slides from 35 donors, were analysed to understand how reader technique contributes to discrepancy in measurements of parasite density over a wide range of densities.
Among these 35 donations, standard deviations ranged from 30% to 250% of the mean parasite density and the percent discrepancy was inversely correlated with the mean parasite density. The number of white blood cells indexed and whether parasites were counted in the thick film or thin film were shown to significantly contribute to discrepancy amongst microscopists.
Errors in microscopy measurements are not widely appreciated or addressed but have serious consequences for efficacy trials, including possibly abandoning promising vaccine candidates.
Microscopy has been used to detect malaria parasites in the blood of infected patients since Laveran first identified the parasites in 1880 . Microscopic examination of blood is the most affordable, accessible, widely used and reliable technique for diagnosis of malaria infection. Although molecular techniques for quantifying parasites have made significant progress in recent years, microscopy remains the primary technique for quantification of parasites. Microscopy is routinely relied upon as a primary endpoint measurement for epidemiological studies, intervention studies, and clinical trials. Despite the critical importance of microscopy for the study and treatment of malaria, little effort has been made to precisely determine and distinguish sources of error in microscopic diagnosis and quantification of parasitaemia or to evaluate the impact of this error on endpoint measurements.
Like all detection methods, microscopy is an imperfect technique. However, unlike other methods, such as PCR and immunochromatographic assays, it relies heavily upon the judgment and experience of the individual user. This was noted at least as early as 1930, when Knowles and White reported on the 'Training and experience of the observer, the personal factor in the diagnosis of malaria' .
To understand how reader technique contributes to discrepancies in reporting parasite species and densities, results from 895 slides made from 35 blood donations were analysed. Some of the slides were parasite-positive and some were guaranteed parasite-negative donations. One slide from each donation was sent to each of 27 expert malaria microscopists for evaluation of parasite presence, species and density. Each participant was asked to report the number of sexual and asexual stages of each species present, but was permitted to choose the manner in which slides were read. There were considerable differences in the densities and species reported for each sample. Analysis of these results yields important insights into the sources of discrepancies between readers reported elsewhere [3–5] and points to possible unevaluated error in reported microscopy results supporting much of the malaria literature.
Details of the patient selection, sample collection and preparation, and reference reader selection and participation have been described in detail elsewhere  and are summarized below.
Sample collection and preparation
Donors were chosen from among symptomatic patients self-presenting to regional health clinics or involved in Internal Review Board-approved malaria research protocols in Cambodia and Indonesia and consent specific to this study was obtained from each participant prior to drawing blood. Malaria-negative donations were taken from individuals who are natives of non-endemic areas who had not been exposed to risk of malaria in the past two years. Approximately 3 ml of venipuncture blood was collected in an ethylene-diaminetetraacetic acid (EDTA)-filled tube and multiple slides (up to 150), with both a thick and a thin smear, were prepared from each donation within hours of sample collection. Thick films were made by spreading exactly 6 μl of blood in a circle of 12 mm in diameter. Two microliters of blood were spread using the edge of a clean slide to make a thin film. The thin film was fixed with methanol and the entire slide was stained with Giemsa solution using standard procedures. Negative donations were stained in separate containers which had never been used for positive donations. Only slides from the same donor were stained in the same batch. Slides were coverslipped to preserve them.
Density and species determination
Slides from 35 donations were sent to 27 reference readers from 13 countries who were considered skilled malaria microscopists by reputation and who accepted the invitation to participate. Each reader received a slide from each of the 35 donations, but each slide was unique (i.e. no reader observed exactly the same slide). Each reader was asked to record the density of asexual and sexual forms of each species observed on the slide. When it was not possible to distinguish between asexual forms in mixed infections, readers often reported the species observed and recorded a total asexual parasite density. No guidelines were given as to how to count the parasites, whether by co-counting white blood cells (WBC) or co-counting red blood cells (RBC), or how many reference RBCs or WBCs should be reported, although the spreadsheet used by each reader to record data included columns for number of WBCs and number of RBCs. The absence of a rigid counting protocol generated a heterogeneity of counting techniques which permitted us to analyse differences between them.
Results which were reported as percent parasitaemia, percent parasitized RBCs or had no information as to how many RBCs or WBCs were indexed were excluded from this analysis (approximately 2.5% of reads). The number of asexual parasites observed was divided by the number of RBCs or WBCs indexed and multiplied by the standard approximation of 8,000 WBCs per microliter or 4.5 × 106 RBCs per microliter . Parasite density data were analysed by summing the asexual forms across all the species present and creating a total-parasites-per-microliter value. Non-parametric statistical tests were performed using the Analyze-It tool for Microscoft Excel.
Variation between readers
To further dissect the source of reader discrepancy at very low density, one microscopist examined the entire thick film of two slides from the same donation. He identified 57 parasites on one slide and 49 parasites on the other, giving densities of 10 and 8 parasites/μl, respectively. The original results reported for these same two slides by two other independent readers were 80 parasites/μl and 0 parasites/μl, indicating that random chance in the selection of fields to examine may play a considerable role in reader discrepancy, particularly at low densities.
Readers were permitted to use either the thick film or the thin film and to count either parasites per WBC or parasites per RBC. There were no guidelines given as to how many WBCs or RBCs should be indexed when measuring the parasite density. Some readers indexed as many as 8,600 WBCs and 150,000 RBCs or as few as 3 WBCs and 400 RBCs (median WBCs and RBCs counted were 203 and 2,000, respectively). 10.4% of all positive results were reported as parasites per RBC.
Thick film (WBC) method
Thick film (WBC) method vs. thin film (RBC) method
White blood cell distribution
There is a critical and long-standing need for tools to standardize the training and evaluation of malaria microscopists. Slide sets consisting of donations with three species individually or in mixed infections and with a wide range of densities were produced to serve as such a tool . Because microscopy is the only technique available for enumerating parasites in dried blood films, such slides must be validated by the very technique they are designed to improve. The slides intended for these sets were validated by a panel of reference readers with extensive experience in malaria microscopy. Comparing results among such a large panel of experienced microscopists from all over the world was only possible due to the meticulous banking of large numbers of slides prepared simultaneously from each donation. Despite the collective knowledge and experience of the readers, there was considerable discrepancy among them concerning species  and densities. This study analysed the contributions of reader technique and method of enumeration to the variability in the density measurements.
Discrepancy among readers decreased with increasing parasite density. This is in agreement with an analysis of field data from Peru and Thailand  and from Uganda . Counts from the same slide made by two different microscopists reported by Earle and Perez in 1932  also show a very similar trend with density. The data in these three studies compared the differences between two readings performed on individual samples. The data analysed in this study are unique in the number of readings taken for each sample (n = 24–27) by independent experts. It is the first study analysing more than three quantitative measurements per sample. The median standard deviation of parasite density for a donation was 58% of the mean and ranged from 30% of the mean to 250% of the mean. The deviation among reads of a single sample is attributable to differences in reader technique and judgment as well as differences between slides made from a single blood sample, a complication that is not present in the studies mentioned above where reads were made from the same slide. It was not possible to estimate from these data differences between slides prepared from the same blood sample which arise due to sample handling and preparation, imperfect lysis of RBCs in the thick film, and random distribution of parasites in the blood, but the preliminary data reported above indicate that the latter may be quite significant. The analysis of field data from Peru and Thailand demonstrated that differences between slides made from the same blood sample and read by the same microscopist are not trivial and were, on average, 8–17%. This is a critical issue which remains to be addressed systematically in a manner which will allow the magnitude of the differences between reader technique, differences between slides, and random error to be quantified and compared.
The observation that the deviation from the mean decreased as the number of WBCs indexed by the reader increased is logical. In general, as the number of measurements in any experimental system increases, the sample mean approaches the true mean or the expectation of the mean. Since the parasite density was evaluated as a function of the number of parasites per WBC, the estimate of the true number of parasites per WBC improves as the number of measurements (i.e. WBCs) increases. The validity of this technique is limited by the implicit assumption that WBCs are evenly distributed in the film. To test this assumption, the number of parasites and the number of WBCs per field were counted in a single thick film. Over 30 fields, the WBCs were much less uniformly distributed than the parasites. This leads to the conclusion that, even if the readers identified every true parasite without error, the variation in the distribution of WBCs from field to field could significantly contribute to discrepancy among readers and errors in estimating true parasite density.
Comparing parasite densities observed in thick films to those observed in thin films confirmed the finding of Dowling and Shute  and others  who reported that as much as 60% percent of parasites were either obscured in the thick film or lost during staining and lysis of red blood cells. The results presented here show that 0–65% fewer parasites were counted in the thick films when compared to the thin film. In only one instance were more parasites (46%) observed by microscopists reading the thick film. One other study  found acceptable agreement between thin and thick film density measurements when parasite density counted against RBCs in the thin film was compared to parasite density counted per high powered field (HPF) in the thick film.
This analysis of differences in reader technique suggests several important considerations when attempting to reliably estimate parasite density from blood films. To improve precision, or agreement among readers, a uniform counting protocol should be applied. Increasing the number of indexed cells (WBCs or RBCs) also improves agreement between readers. However, to improve both precision and accuracy, the use of WBCs could be eliminated in favour of a grid counting or HPF counting method which allows the parasites per volume to be counted directly and avoids the problem of unevenly distributed WBCs and an inaccurate conversion factor [11, 12]. Using the standard conversion factor of 8,000 WBCs per μl was adequate in this application because reads were only being compared between microscopists examining the same blood sample. For studies where patient parasitaemias are being compared to each other or within a group, the standard WBC approximation is grossly inadequate as WBC counts can vary greatly between individuals and decrease as a result of malaria infection . Applying a volume-counting method or using true WBC counts for each patient as has been done in some clinical trials [14–16] would eliminate this source of error.
Accurate and precise quantification of malaria parasites is a critical endpoint for many types of intervention trials. In a clinical setting, errors could endanger a patient. Recent studies and unpublished personal experience have shown that false positive and negative rates using microscopy in the field can be as high as 30%  and discrepancies in density measurements can be as much as 2–3 fold . This study identified similar discrepancies in ideal laboratory conditions among expert microscopists and demonstrated that at least a portion of the discrepancy is due to differences in reader technique and the use of WBCs as a counting index. It also highlights the importance of reporting slide reading protocols in published studies and the potential danger of comparing density data between studies. More experiments and analyses need to be done to determine how much this problem could be ameliorated by increased training, standardization of techniques and counting protocols, and improved awareness of the errors inherent in microscopic detection and quantification of malaria parasites.
Conflict of interest
The author(s) declare that they have no competing interests.
The views and opinions are those of the authors and do not purport to represent those of the U.S. Navy or Department of Defense.
We thank Dr. S. Duoung of the Cambodian National Center for Parasitology, Entomology and Malaria Control (CNM), and Dr. P. Sismadi of the Indonesian National Center for Health Research and Development for their support. We would like to express our sincerest appreciation to the technical and support staff of Parasitic Diseases Program, NAMRU-2 and the Ministry of Health, Cambodia. Finally, we gratefully recognize the dedicated work ofour reference malaria smear readers. This work was supported in part by U.S. National Institutes of Health Malaria Research and Reference Reagent Resource Center (MR4) Prime Award Number NO1-AI-85355.
- Laveran A: Deuxieme note relative a un nouveau parasite trouve dans le sang des malades atteints de la fievre paludisme. Bulletin de l'Academie medicale. 1880, 2ieme serie: 1346-1347.Google Scholar
- Knowles R, White RS: Studies in the parasitology of malaria. Indian Medical Research Memoirs. 1930, 18:Google Scholar
- McKenzie FE, Sirichaisinthop J, Miller RS, Gasser RA, Wongsrichanalai C: Dependence of malaria detection and species diagnosis by microscopy on parasite density. Am J Trop Med Hyg. 2003, 69: 372-376.PubMed CentralPubMedGoogle Scholar
- Kilian AHD, Metzger WG, Mutschelknauss EJ, Kabagambe G, Langi P, Korte R, von Sonnenburg F: Reliability of malaria microscopy in epidemiological studies: results of quality control. Trop Med Int Health. 2000, 5: 3-8. 10.1046/j.1365-3156.2000.00509.x.View ArticlePubMedGoogle Scholar
- Prudhomme O'Meara WA, McKenzie FE, Magill AJ, Forney JR, Permpanich B, Lucas C, Gasser RA, Wongsrichanalai C: Sources of variability in determining malaria parasite density by microscopy. Am J Trop Med Hyg. 2005, 73: 593-598.Google Scholar
- Maguire J, Lederman E, Barcus M, Prudhomme O'Meara W, Jordon R, Duong S, Muth S, Sismadi P, Bangs M, Prescott W, Baird J, Wongsrichanalai C: Production and Validation of Durable, High Quality Standardized Malaria Microscopy Slides for Teaching, Testing and Quality Assurance during an Era of Declining Diagnostic Proficiency. Malaria Journal. 2006, 5: 92-10.1186/1475-2875-5-92.PubMed CentralView ArticlePubMedGoogle Scholar
- Warhurst DC, Williams JE: Laboratory diagnosis of malaria. J Clin Pathol. 1996, 49: 533-538.PubMed CentralView ArticlePubMedGoogle Scholar
- Earle WC, Perez M: Enumeration of parasites in the blood of malarial patients. J Lab Clin Med. 1932, 17: 1124-1130.Google Scholar
- Dowling MAC, Schute GT: A comparative study of thick and thin films in the diagnosis of scanty malaria parasitemia. Bull World Health Organ. 1966, 34: 249-267.PubMed CentralPubMedGoogle Scholar
- Bejon P, Andrews L, Hunt-Cooke A, Sanderson F, Gilbert S, Hill A: Thick blood film examination for Plasmodium falciparum malaria has reduced sensitivity and underestimates parasite density. Malaria Journal. 2006, 5: 104-10.1186/1475-2875-5-104.PubMed CentralView ArticlePubMedGoogle Scholar
- Planche T, Krishna S, Kombila M, Engel K, Faucher JF, Ngou-Milama E, Kremsner PG: Comparison of methods for the rapid laboratory assessment of children with malaria. Am J Trop Med Hyg. 2001, 65: 599-602.PubMedGoogle Scholar
- Greenwood BM, Armstrong JRM: Comparison of 2 Simple Methods for Determining Malaria Parasite Density. Trans R Soc Trop Med Hyg. 1991, 85: 186-188. 10.1016/0035-9203(91)90015-Q.View ArticlePubMedGoogle Scholar
- McKenzie FE, Prudhomme WA, Magill AJ, Forney JR, Lucas C, Permpanich B, Gasser RA, Wongsrichanalai C: WBC counts and malaria. J Infect Dis. 2005, 192: 323-330. 10.1086/431152.PubMed CentralView ArticlePubMedGoogle Scholar
- Nosten F, Luxemburger C, Kyle DE, Ballou WR, Wittes J, Wah E, Chongsuphajaisiddhi T, Gordon DM, White NJ, Sadoff JC, Heppner DG, Bathe K, Blood J, Brockman A, Cobley UT, Hacking D, Hogg D, U KH, Maelankiri L, Chuanak N, Permpanich B, Price R, Raimond D, Schabenberger O, Singharaj P, Singhasivanon P, Slight T, Tulayon S, Tway KL, Ynint T, VincentiDelmas M, deVries A, Webster HK: Randomised double-blind placebo-controlled trial of SPf66 malaria vaccine in children in northwestern Thailand. Lancet. 1996, 348: 701-707. 10.1016/S0140-6736(96)04465-0.View ArticlePubMedGoogle Scholar
- Genton B, Betuela I, Felger I, Al-Yaman F, Anders RF, Saul A, Rare L, Baisor M, Lorry K, Brown GV, Pye D, Irving DO, Smith TA, Beck HP, Alpers MP: A recombinant blood-stage malaria vaccine reduces Plasmodium falciparum density and exerts selective pressure on parasite populations in a phase 1-2b trial in Papua New Guinea. J Infect Dis. 2002, 185: 820-827. 10.1086/339342.View ArticlePubMedGoogle Scholar
- Forney JR, Magill AJ, Wongsrichanalai C, Sirichaisinthop J, Bautista CT, Heppner DG, Miller RS, Ockenhouse CF, Gubanov A, Shafer R, DeWitt CC, Quino-Ascurra HA, Kester KE, Kain KC, Walsh DS, Ballou WR, Gasser RA: Malaria rapid diagnostic devices: Performance characteristics of the ParaSight F device determined in a multisite field study. J Clin Microbiol. 2001, 39: 2884-2890. 10.1128/JCM.39.8.2884-2890.2001.PubMed CentralView ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.