- Open Access
Analysis on external competency assessment for malaria microscopists in China
Malaria Journal volume 18, Article number: 366 (2019)
In order to meet the requirement of malaria elimination (ME), three courses of the External Competency Assessment of Malaria Microscopists (ECAMM) were conducted during 2017–2018 in China by facilitators designated by the World Health Organization (WHO-ECAMM). A training course with a model copied from the WHO-ECAMM course was also held a week ahead of ECAMM in March 2018. Thirty-six participants completed these courses and obtained different results.
The slide structures, agendas, score calculations, and the levels of certifications of the four courses strictly adhered to the WHO guidelines. All the data were collected in Excel 2016 and analysed in Graphpad Prism5 or SPSS 23. Significant differences were evaluated in Graphpad Prism5 by two-tailed paired t tests between the pre-assessment and final-assessment for each of the four courses, as well as one-way ANOVAs with Kruskal–Wallis tests and Dunn’s post hoc tests among the final assessments of the four courses. Correlations between participants’ competency results and their ages, years working on malaria, and numbers of malaria cases reported in their provinces were evaluated by bivariate correlations (two-tailed) and linear regression (excluding cases pairwise) in SPSS 23. The Pearson correlation coefficients (r values), P values (two tailed), adjusted R square (Adjusted R2), standardized coefficients (β) and Sig. P values were recorded. The percentages of participants who gave the right answer to each slide (PPS) in the final assessments of the three WHO-ECAMM courses were calculated. Correlation analysis between PPS and parasitaemia (100–2000 parasites/μL) of Plasmodium falciparum slides used in species identification and parasite counting, were also evaluated via bivariate correlations (two-tailed) tests.
Among the 36 participants, 16 participants were certificated as Level 1 (two from NRL), 10 were certified as Level 2 (one from NRL). Within the same course, participants had improved their average scores from pre-assessments to final assessments. The numbers of malaria cases reported in participants’ provinces were strongly correlated to their species identification (SI) scores; r = 0.45, P = 0.040, n = 21; r = 0.57, P = 0.001, n = 32; r = 0.56, P = 0.007). The parasitaemia of P. falciparum within 100–2000 parasites/μL was correlated significantly (r = 0.44, P = 0.008, n = 36) with the PPS of all counting slides but not with slides for identification (r = − 0.018, P = 0.93, n = 30).
The analysis and comparison of participants’ competency results not only verified that the model of the WHO-ECAMM course had strong power in improving and assessing microscopists’ competencies but also reflected the correlation between decreased numbers of indigenous malaria cases and microscopists’ competencies in certain areas in China.
China is planning to declare malaria elimination (ME) in 2020 . To support the declaration and sustain the achievements in ME, a sensitive surveillance system based on precise and timely diagnosis of malaria cases in current and future China is required [2,3,4,5,6,7]. Hence, certificated microscopists with high levels of competency in malaria diagnosis were essential. In the past dozen years, the model for the External Competency Assessment of Malaria Microscopists (ECAMM) courses—developed by the World Health Organization (WHO)—has been conducted in more than 100 selected countries in the West Pacific Region, Southeast Asia Region, Africa Region and Eastern Mediterranean Region. This model had been verified to be an effective model to evaluate competencies of microscopists. In order to meet the requirement of ME, three WHO-ECAMM courses were held during 2017–2018, which were attended by 36 Chinese microscopists from either the National Reference Laboratory for Malaria Diagnosis (NRL) or 31 provincial laboratories. To obtain preliminary knowledge prior to the WHO-ECAMM course, 12 microscopists from the course in March 2018 received a training course with a model copied from the WHO-ECAMM course. The 36 participants had different backgrounds and obtained different results from the WHO-ECAMM courses. In this study, the competency results from these four courses were reported and analysed to determin ways to best sustain and enhance these competencies in China.
Administration of WHO-ECAMM and the training courses
WHO-ECAMM courses were held at the National Institute of Parasitic Diseases (NIPD) at the Chinese Center for Disease Control and Prevention (China CDC) in Shanghai by the facilitator, Ken Lilley (from WHO), in September 2017, March 2018 and November 2018. The slide structure and agenda strictly followed WHO guidelines [8, 9]. The training course was conducted a week ahead of the WHO-ECAMM course that was held in March 2018. The slide structure (Table 1), agenda, score calculations and level of certification of the training course were copied from WHO-ECAMM [8, 9], in which all participants were required to read 74 slides in 4 days, of which 56 slides were used for competency assessment. The slides for the four courses were all from the slide banks of the Research Institute for Tropical Medicine (RITM). Participants’ scores or accuracy (%) for parasite detection (PD), species identification (SI), and parasite counting (PC) were calculated by the following three formulas:
Pp denotes the number of slides with malaria parasites (positives slides) noted by the participant. FNp denotes the number of false negatives noted by the participant. Np denotes the number of negatives noted by the participant. FPp denotes the number of false positives noted by the participant. CCs denotes the number of slides with PC within 25% of the true count noted by the participant. T denotes the total number used for assessment. A participant was certificated as having a high level (1 or 2 levels) of competency only when a participant’s score of PD and SI were both equal to or higher than 80% and their PC score was equal to or higher than 40% (Table 2).
Background of participants from the three different WHO-ECAMM courses
For the 12 participants in the WHO-ECAMM course in March 2018, seven of whom came from non-malaria-epidemic provinces, one participant had attended the WHO-ECAMM course in the past and all received the training a week ahead of the main course. Twenty-four participants in the other two WHO-ECAMM courses were from NRL and malaria-epidemic provinces. Nine of the 12 participants attending the course in November 2018 had taken part in the WHO-ECAMM course at least once in the past three to 6 years. However, most of the participants (11/12) from the course in September 2017 had neither of the above experiences.
All the data were collected in Excel 2016. Detailed basic information of the 36 participants (e.g., ages and years working on malaria) from the three WHO-ECAMM courses and their competency results—including PD, SI, PC and certification levels (CLs)—were collected and consulted from mission reports of the WHO-ECAMM facilitator. The malaria situations (reported cases) in 2014–2017 in the 31 provinces from which the 32 participants worked were gathered (two participants from same province) [10,11,12,13]. Also, the answer of each participant given to each slide in the final assessment of the three WHO-ECAMM courses was recorded.
All the data were analysed in Graphpad Prism5 or SPSS 23. To compare the competency results (mean values with 95% confidence interval, CI) between the pre-assessments and final-assessments from each of the four courses, significant differences (P < 0.05, significant; P ≥ 0.05, not significant) were evaluated by two-tailed paired t tests in Graphpad Prism5. Additionally, the final-assessments of the four courses were evaluated by one-way ANOVAs with Kruskal–Wallis tests and Dunn’s post hoc tests in Graphpad Prism5. The Pearson correlation coefficients (r values) and P values (two tailed) among the 36 participants’ competency results of PD, SI and PC scores were evaluated by bivariate correlations (two-tailed) in SPSS 23. To verify the factors influencing participants’ competency results, the correlations between participants’ competency results and their ages, years working on malaria, and numbers of malaria cases reported in their provinces were also evaluated. Firstly, the 36 participants’ ages, years working on malaria, and the numbers of malaria cases reported in the 31 provinces in the different courses were counted and compared. The correlation analysis was carried out within each of the four combinations that were assembled by data in the final assessments of the four courses by bivariate correlations and linear regression in SPSS 23. The correlation coefficients (r values), P values (two tailed), adjusted R square (Adjusted R2), standardized coefficients (β) and Sig. P values were recorded. Combination 1 included data in the WHO-ECAMM course in September 2017 and in the training course in which participants attended prior to the main course. Combination 2 included data of Combination 1 and those in the WHO-ECAMM course in November 2018. Data for Combinations 3 and 4 were all obtained from the WHO-ECAMM courses. Combination 3 included data from September 2017 and November 2018 in which participants had not received similar training over the past 3 years. Combination 4 included data from the three WHO-ECAMM courses.
The percentages of participants who gave the right answer to each slide (PPS) in the final assessment of the three WHO-ECAMM courses were calculated and recorded.
Pearson correlation coefficients (r values) and P values (two tailed) between the PPS and parasitaemia (100–2000 parasites/μL) of the Plasmodium falciparum slides used in species identification and parasite counting were evaluated by bivariate correlations (two-tailed) test separately.
Competency results in the training course
The competency results in the pre-assessment and final-assessment were shown in Table 3. One participant from NRL was certified at Level 1, four participants from provincial laboratories were certificated as Level 2, and the other participants were certified as either Level 3 or Level 4. All participants improved their scores. The average scores improved as follows: PD increased from 92% (CI 84–100%) to 93% (CI 88–97%); PC increased from 28% (CI 16–41%) to 42% (CI 31–54%); and SI increased significantly from 58% (CI 40–76%) to 77% (CI 69–86%; P = 0.007, n = 12, paired t test).
Competency results of the three WHO-ECAMM courses
The competency results of all participants in the three WHO-ECAMM courses were shown in Table 4. In total, 16 participants were certificated as Level 1 (two from NRL), 10 participants were certified as Level 2 (one from NRL), and the others were certified as Level 3 or Level 4. Separately, in the course from September 2017, two participants were certificated as Level 1, four participants were certified as Level 2, three participants were certified as Level 3, and three participants were certified as Level 4. In the course from March 2018, six participants were certificated as Level 1, five participants were certified as Level 2, and one participant was certified as Level 4. In the course from November 2018, eight participants were certificated as Level 1, one participant was certified as Level 2, and three participants were certified as Level 3.
Correlation analysis showed that the 36 participants’ competencies of SI were significantly correlated with PC (r = 0.48, P = 0.003; n = 36). All average scores of the 36 participants in the final assessments had improved compared with those in pre-assessments, except for the average PD score in September 2017, which was 98% (CI 95–101%) and 97% (CI 95–99%; P = 0.63, n = 12), respectively. Also, participants from September 2017 had improved their SI and PC scores significantly, which increased from 71% (CI 59–84%) to 85% (CI 80–90%; P = 0.014, n = 12), and from 25% (CI 13–37%) to 42% (CI 31–54%; P = 0.036; n = 12), respectively. The participants from March 2018 improved their SI score significantly from 87% (CI 81–93%) to 94% (CI 91–97%; P = 0.021, n = 12). Participants from November 2018 improved both their PD and SI scores significantly from 92% (CI 87–97%) to 98% (CI 96–101%; P = 0.044, n = 12), and from 80% (CI 69–91%) to 94% (CI 91–97%) (P = 0.0065), respectively.
Among the three courses, participants’ average PD scores in the final assessments were 97% (CI 95–99%), 100% (CI 99–101%) and 98% (CI 96–101%). Following a one-way ANOVA with a Kruskal–Wallis test, there was no significant difference among these scores (P = 0.059; n = 12, 12, 12). For the average scores of SI in the three final assessments, participants from the three courses obtained SIs of 85%, 94% and 94%. Participants’ average SI scores in the two courses from 2018 were significantly higher than that in September 2017 (P = 0.0034, Kruskal–Wallis test; n = 12, 12, 12), but there was no significant difference between the courses during 2018. For the average score of PC in the final assessments, participants from the three courses obtained PC scores of 42% (CI 31–54%), 56% (CI 46–66%) and 50% (CI 43–57%). There were no significant differences among these scores (Kruskal–Wallis test, n = 12, 12, 12).
Among the four courses, the competency results in the final assessments of the training course were the lowest. However, analysis showed that 12 participants in this course improved their average scores of PD, SI or PC (P = 0.0084, P = 0.0003, P = 0.0043; Kruskal–Wallis test; n = 12, 12, 12, 12) from pre-assessment in the training course to the final-assessment in the WHO-ECAMM course in March 2018.
Influence of parasitaemia on participants’ competency results of P. falciparum identification and counting
The correlation analysis showed that there was no correlation between the PPS and parasitaemia of P. falciparum slides (r = − 0.018, P = 0.93, n = 30) that were used in species identification. However, these two datasets correlated significantly (r = 0.44, P = 0.008, n = 36) when the slides were used for parasite counting and their parasitaemia were within 100–2000 parasites/μL.
Analysis of participants’ backgrounds and basic information
The backgrounds and basic information of the 36 participants from 31 provincial laboratories and NRL in the different courses were analysed and compared (Table 5, 6). The average age of participants in the three WHO-ECAMM courses was 38 years old (CI 36–40 years old, n = 36). Among them, the youngest and oldest participants were 28 and 54 years old. There was no significant difference among the ages of participants from the three different WHO-ECAMM courses (P = 0.61). All of the participants had experience working on malaria. They had worked on malaria for at least 1 year and for as long as 32 years. On average, they had worked in malaria diagnosis for 12 years (CI 9–14 years, n = 36). The numbers of malaria cases in the 31 provinces ranged from 1 to 469 (Mean number = 101 cases, CI 59–144 cases, n = 31). Participants from provinces where more cases were reported tended to work on malaria longer (r = 0.37, P = 0.036). There were no variations in their ages in terms of the differences in the numbers of malaria cases among their different provinces (r = 0.21, P = 0.25).
Participants from the WHO-ECAMM course of March 2018 had worked in malaria diagnosis significantly less time (mean = 6 years, CI 3–8 years, n = 12) than those from the other two courses (mean = 16 years, CI 10–21 years, n = 12; mean = 14 years, CI 9–19 years, n = 12; P = 0.0040, Kruskal–Wallis test). The average number of malaria cases (Mean = 199 cases, CI 111–289 cases, n = 11) in provinces from participants in the course of November 2018 was significantly greater than those in provinces from participants in the other two courses (mean = 68 cases, CI 19–116 cases, n = 11; mean = 54 cases, CI − 20 to 116 cases, n = 10; P = 0.005, Kruskal–Wallis test).
Evaluation of correlation between competency results and factors
The correlation analysis results of data in the four combinations were shown in Table 7. Nearly none of the combinations showed a correlation of participants’ competency results with their ages and years that participants worked on malaria.
For the numbers of malaria cases in provinces, the correlation analysis between them and participants’ SI scores presented a significant positive correlation in Combinations 1, 2 and 3 (r = 0.45, P = 0.040, n = 21; r = 0.57, P = 0.001, n = 32; r = 0.56, P = 0.007, n = 22). The regression results were also well supported these correlations in Combination 2 and 3 (Adjusted R2 = 0.28, 0.20; β = 0.50, 0.57; Sig. P = 0.006, 0.011; n = 32, 22). However, the correlation disappeared or degraded in Combination 4 (r = 0.30, P = 0.092, Adjusted R2 = 0.13; β = 0.45; Sig. P = 0.021; n = 32). In all correlation analysis, no factors were shown to correlate with participants’ PD scores (|r| ≤ 0.38, P ≥ 0.11).
Discussion and conclusion
In 2017, no indigenous cases of malaria were reported in China, which gave a boost to all Chinese people [10, 14]. This represented a crucial milestone for China in terms of reaching the ME goal. However, it should be noted that imported cases were still reported in all 31 provinces in China, including seven non-epidemic provinces [5, 6, 10,11,12,13]. Additionally, the malaria patients were not restricted to known areas or designated by natural environments any longer. If the cases could not be found in time, especially in some areas where malaria cases were seldom or had not been reported for a long time, severe cases (even potentially fatal cases) [5, 6, 10,11,12,13] and/or retransmission could occur [15,16,17]. Hence, continued efforts were needed to strengthen the malaria monitoring system, which depends on a strong and widely distributed laboratory network that can ensure accurate and timely diagnoses of malaria cases.
From September 2017 to the end of November 2018, three WHO-ECAMM courses were held to determine microscopists’ competency levels and to strengthen the laboratory network for malaria diagnosis in China. Thirty-six participants were certificated in these courses and their certifications were valid until 2020. Thirty-two of them covered all provinces in China. Most of them were young and experienced. Sixteen participants (including two from NRL) were certificated as Level 1 and 10 participants (including one from NRL) were certified as Level 2. All of them were either trainers, facilitators, or inspectors in the laboratory network of malaria diagnosis for achieving the ME goal in China. However, because malaria activities were organized in provincial units under a national level in China, at least one microscopist with a high level (1 or 2 level) in each province was essential for the laboratory network. Thus, 26 microscopists with high levels is not sufficient for such a large area of China. Hence, more training courses and WHO-ECAMM courses were required to strengthen and certify microscopists’ competencies.
Analysis on the competency results of the participants showed that there were differences among the three WHO-ECAMM courses, especially in terms of SI and PD scores. PD and SI were the key requirements in the accurate diagnosis of malaria. These two competencies not only influenced each other, but also influenced PC competency greatly. As such, it was necessary to find the factors that might influence these two competencies. Thus participants’ basic information and backgrounds were investigated. The competency results in the training course were also applied in this investigation because the results reflected their original competencies in their routine work. The results showed that (data in Combination 1, 2 and 3) if there were no training the numbers of malaria cases reported in participants’ provinces significantly correlated to participants’ SI scores. In contrast, competencies of even experienced microscopists in provinces where malaria gradually became a rare disease concomitantly decreased because they did not have enough malaria slides to review in their routine work over a long period of time [5, 6, 10,11,12,13, 18]. Additionally, microscopists in provinces where malaria cases were less reported usually would not remain at their posts for a long time, and new staff in these provinces would not be able to improve their competencies because they would have less of a chance of reading slides [4, 18,19,20,21]. These outcomes were also roughly in accordance with the trend of participants’ competency results in different courses and their backgrounds and/or basic information.
Surprisingly, although their previous competency results in the training course were worse, participants’ results in the WHO-ECAMM course of March 2018 were distinctly better than those in 2017 and were not inferior to those in November 2018. This significant change was ascribed to the training course held ahead of the WHO-ECAMM course. The training course had a model that was copied from the WHO-ECAMM and it improved these participants’ competencies greatly and led to the disappearance or degradation of the influence of participants’ previous working experiences (Combination 4). This power was consistent with the significant improvement of participants’ assessment results from pre-assessment to final assessment in each course.
Because of the strong power of the model of the WHO-ECAMM course in improving and assessing microscopists’ competencies, this model provides a new perspective for simultaneously training and assessing microscopists in China, especially in areas where malaria cases have been occasional or seldom reported in recent years. In these areas, microscopists usually had worked for a period but their competency could not sustain or improve due to the nature of their routine work. Even slides might be inadequate for conducting training courses in some provinces. As such, this kind of model (WHO-ECAMM) might not be applicable in this case.
Besides participants’ experiences, the slides themselves also influenced their competency results. Extensive misidentification of P. falciparum was not regarded as a difficult point for all microscopists but only for those who had less of a chance to examine the slides. However, parasite counting was difficult for all microscopists, especially when counting slides with low parasitaemia (one less or more parasite counted might influence the results greatly). Additionally, according to the analysis, participants’ PC competencies were significantly influenced by their SI competencies. Hence, microscopists’ capabilities both on SI should be strengthened before making improvements in PC competency. Additionally, counting skills that strictly followed the rules recommended by WHO were also practical for accurate counting. All participants in the course of March 2018 and nine participants in the course of November 2018 had experienced formal counting practice repeatedly. Consequently, they mastered their counting skills more skillfully than those in the course of September 2017, in which no participants had this experience. However, the low score or PPS in counting also indicated that most of the participants in China seldom counted parasites in their routine work.
Availability of data and materials
The data and material in this file were available by contacting Dr. Mei Li, email@example.com.
Ministry of Health of the People’s Republic of China. [Action Plan of China Malaria Elimination (2010–2020)] (in Chinese). http://www.moh.gov.cn/mohbgt/s10788//201005/47529.shtml.webcite.
Lu GY, Liu YB, Wang JS, Li XM, Liu X, Beiersmann C, et al. Malaria training for community health workers in the setting of elimination: a qualitative study from China. Malar J. 2018;17:95.
Yin JH, Yan H, Li M, Ruan Y, Zhang XQ, Wang LY, et al. Competency and challenges in malaria microscopy in China. BioSci Trends. 2017;11:702–5.
Ding GS, Zhu GD, Cao CQ, Miao P, Cao YY, Wang WM, et al. The challenge of maintaining microscopist capacity at basic levels for malaria elimination in Jiangsu Province, China. BMC Public Health. 2018;18:489.
Zhou S, Li ZJ, Yu HJ, Yang WZ. Trends of imported malaria in China 2010–2014: analysis of surveillance data. Malar J. 2016;15:39.
Cao J, Liu YB, Cao YY, Zhu GD, Zhou SS. Sustained challenge to malaria elimination in China: imported malaria (in Chinese). Chin J Parasitol Parasit Dis. 2018;36:93–6.
WHO. A framework for malaria elimination. Geneva: World Health Organization; 2017.
WHO. Malaria microscopy quality assurance manual. Version 2. Geneva: World Health Organization; 2016.
Ashraf S, Kao A, Hugo C, Christophel EM, Fatunmbi B, Luchavez J, Ken Lilley K, et al. Developing standards for malaria microscopy: external competency assessment for malaria microscopists in the Asia-Pacific. Malar J. 2012;11:352.
Zhang L, Feng J, Zhang SS, Xia ZG, Zhou SS. The progress of national malaria elimination and epidemiological characteristics of malaria in China in 2017 (in Chinese). Chin J Parasitol Parasit Dis. 2018;36:201–9.
Zhang L, Zhou SS, Feng J. Malaria situation in the people’s Republic of China in 2014 (in Chinese). Chin J Parasitol Parasit Dis. 2015;33:319–26.
Zhang L, Feng J, Zhang SS, Xia ZG, Zhou SS. Malaria situation in the people’s Republic of China in 2015 (in Chinese). Chin J Parasitol Parasit Dis. 2016;34:477–81.
Zhang L, Feng L, Zhang SS, Jiang S, Xia ZG, Zhou SS. Malaria situation in the people’s Republic of China in 2016 (in Chinese). Chin J Parasitol Parasit Dis. 2017;35:1–5.
Feng J, Zhang L, Huang F, Yin JH, Tu H, Xia ZG, et al. Ready for malaria elimination: zero indigenous case reported in the People’s Republic of China. Malar J. 2018;7:315.
Krisher LK, Krisher J, Ambuludi M, Arichabala A, Beltrán-Ayala E, Navarrete P, et al. Successful malaria elimination in the Ecuador-Peru border region: epidemiology and lessons learned. Malar J. 2016;15:573.
Whittaker M, Smith C. Reimagining malaria: five reasons to strengthen community engagement in the lead up to malaria elimination. Malar J. 2015;14:410.
Fernando SD, Dharmawardana P, Semege PS, Epasinghe G, Senanayake N, Rodrigo C, et al. The risk of imported malaria in security forces personnel returning from overseas missions in the context of prevention of re-introduction of malaria to Sri Lanka. Malar J. 2016;15:144.
Yitbarek T, Nega D, Tasew G, Taye B, Desta K. Performance evaluation of malaria microscopists at defense health facilities in Addis Ababa and its surrounding areas, Ethiopia. PLoS ONE. 2016;11:e0166170.
Zhou RM, Liu Y, Qian D, Su YP, He LJ, Yan QY, et al. Assessment of malaria knowledge and capability on Plasmodium detection among professionals from class III malaria endemic counties of Henan province in 2012 (in Chinese). Chin J Parasitol Parasit Dis. 2014;32:46–9.
Zhang SS, Xia ZG, Yin JH, Yan H, Zhou SS, Li SZ, et al. Analysis report of the national technique competition for diagnosis of parasitic diseases in 2012. I. Capability analysis of Plasmodium detection (in Chinese). Chin J Parasitol Parasit Dis. 2011;31:131–4.
Ekawati LL, Herdiana H, Sumiwi ME, Barussanah C, Ainun C, Sabri S, et al. A comprehensive assessment of the malaria microscopy system of Aceh, Indonesia, in preparation for malaria elimination. Malar J. 2015;14:240.
We gratefully acknowledge National Health commission of the People’s Republic of China, WHO Regional Office for the Western Pacific, WHO Representative in the People’s Republic of China for the supports on conduction of the WHO-ECAMM courses.
This study was supported by National Key S&T Special Project of China (Nos: 2018ZX10101002-002-001, 2018ZX10101-002-002-005) and National Health Commission Special project for Lancang-Mekong coorperation in 2017 (No. 2020399).
Ethics approval and consent to participate
The programme was reviewed and approved by the Ethical Committee of National Institute of Parasitic Diseases, China CDC. All participants provided written informed consent for the study.
Consent for publication
All authors have read and agree to publish this article.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Li, M., Zhou, H., Yan, H. et al. Analysis on external competency assessment for malaria microscopists in China. Malar J 18, 366 (2019) doi:10.1186/s12936-019-2996-3
- Training course