Research | Open | Published:
An exploratory study of factors that affect the performance and usage of rapid diagnostic tests for malaria in the Limpopo Province, South Africa
Malaria Journalvolume 6, Article number: 74 (2007)
Malaria rapid diagnostic tests (RDTs) are relatively simple to perform and provide results quickly for making treatment decisions. However, the accuracy and application of RDT results depends on several factors such as quality of the RDT, storage, transport and end user performance. A cross sectional survey to explore factors that affect the performance and use of RDTs was conducted in the primary care facilities in South Africa.
This study was conducted in three malaria risk sub-districts of the Limpopo Province, in South Africa. Twenty nurses were randomly selected from 17 primary health care facilities, three nurses from hospitals serving the study area and 10 other key informants, representing the managers of the malaria control programmes, routine and research laboratories, were interviewed, using semi-structured questionnaires.
There was a high degree of efficiency in ordering and distribution of RDTs, however only 13/20 (65%) of the health facilities had appropriate air-conditioning and monitoring of room temperatures. Sixty percent (12/20) of the nurses did not receive any external training on conducting and interpreting RDT. Fifty percent of nurses (10/20) reported RDT stock-outs. Only 3/20 nurses mentioned that they periodically checked quality of RDT. Fifteen percent of nurses reported giving antimalarial drugs even if the RDT was negative.
Storage, quality assurance, end user training and use of RDT results for clinical decision making in primary care facilities in South Africa need to be improved. Further studies of the factors influencing the quality control of RDTs, their performance of RDTs and the ways to improve their use of RDTs are needed.
The South African National Malaria treatment guidelines stipulate that malaria treatment (using artemesinin-based combination therapy) should be based on definitive diagnosis using microscopy or malaria rapid diagnostic tests (RDTs) . South Africa has been implementing RDTs to diagnose malaria within malaria endemic areas since 2001. In a primary health care setting, RDTs are most appropriate: they are easy to use, do not require sophisticated technology and give rapid results . The functioning and accuracy of RDTs can be affected by several factors, including manufacturing defects, storage, transport, and end-user performance . Malaria diagnostic tests need to be highly accurate because false negative and false positive diagnoses have medical, social, and economic consequences such as prolongation of illness, increase in morbidity and mortality and loss in credibility of health services [5, 6].
The Limpopo Province, is one of three malaria endemic provinces in South Africa and has the highest malaria incidence . Figure 1 provides a map showing magisterial areas. Although RDTs was introduced for malaria diagnosis in Limpopo in 2003, operational issues relating to its performance and use have not been rigorously investigated. A study was therefore undertaken in the Limpopo Province to determine which factors affected quality and usage of RDTs.
Purposefully selected key informants and randomly selected nurses at the primary health facility level were interviewed using a semi-structured questionnaire. Key informants included three hospital pharmacy staff, one regional pharmacy manager, three district malaria managers, one provincial malaria control manager and two researchers. Among the nurses interviewed, 17 were from primary health care (PHC) clinics and three were from hospitals.
A two-stage sampling procedure was used to select PHC clinic staff . All clinics and health centers within the three malaria-affected districts were listed. Clinics and health centers with fewer than 10 malaria cases per annum were excluded from the sampling frame. In the first stage of sampling, 10% of the clinics (10/100) and 20% of health centers (7/35) within each selected district were randomly selected. In the second stage, one nursing sister was randomly selected from each selected health facility. Interviews were transcribed, ordered and coded in matrixes, using the key categories of procurement & stock monitoring, storage, transport, quality control and end user experiences .
The University of Limpopo Research Ethics Committee, the Limpopo Department of Health and Social Development and the London School of Hygiene and Tropical Medicine granted ethical permission for this study. Informed consent was acquired from all interviewees.
Procurement and stock monitoring
In all health facilities, there was evidence that stocks and expiry dates were monitored regularly. Stock monitoring methods ranged from stock cards (paper based) to electronic systems. Nursing staff were aware of the seasonal increase in malaria cases and stated that they ordered more RDT stock before and during the season; however, only 20% of the nurses interviewed were able to accurately give the limits of the malaria season (September-May). More than half (55%) of the nurses indicated that stock-outs of RDTs occurred. However, they reported that this was rare (one or two times in a season) and contingency plans existed to replace stock from either the nearest clinic or hospital pharmacies. Replacement of stock took place within 24 hours. District malaria managers reported RDT stock-outs in clinics in the 2005/2006-malaria season. They responded by alerting the necessary authorities (district hospital or regional pharmacy depots) or transporting malaria kits to facilities.
Pharmacy staff reported deploying pharmacy assistants to some health facilities to assist with ordering of pharmaceuticals including RDTs. One hospital had a bar code system for ordering pharmaceutical supplies. At the regional pharmacy level an electronic system is in place to increase stock as demand increased, and in most cases this is proportional to the seasonal increase in malaria cases.
Storage of RDTs
Sixty five percent (13/20) of the nurses reported that RDTs were stored correctly i.e. in an air-conditioned room with regular temperature monitoring. Among the nurses from the seven facilities that did not implement correct storage, three (42%) were very concerned with temperature fluctuation as thermometers were unavailable for temperature monitoring. The remaining 4/7 (53%) indicated that although their clinics lacked air conditioning, room temperature was monitored, and that it rarely rose above 30°C. This however was not corroborated with any recorded data.
Malaria managers (district and provincial) accepted that some clinics did not keep RDTs in a cool environment; however they were not concerned about this, commenting that "the kits did not stay in the clinics for too long", due to their frequent use.
Pharmacy managers in hospitals and regional depots stored RDTs in an air-conditioned environment (temperature range 15–25°C). Monitoring charts were produced when requested by the interviewer.
Quality control of using RDTs
Only 4/20 nursing staff said that they checked the quality of RDTs, this they reported to have been done, by comparing the agreement between a diagnosis based on clinical signs and symptoms and the RDT result. Two of 20 nurses reported that they used blood smear results to confirm the RDT results occasionally. Three (15%) nurses reported that they gave antimalarial drugs to RDT negative patients if the clinical presentation was suggestive of malaria.
Figure 3 highlights the key challenges identified by nurses, researchers and malaria managers relating to quality.
In summary, managers were very concerned that RDTs quality were not being monitored at health facility level. One informant stated, "RDT quality control, both at the manufacturing side and at the testing stage, was lacking. The key is the end user's ability to distinguish between positive and negative results."
End User experiences in using RDTs
Nursing staff had huge praise for the use of the RDTs, Figure 4 highlights nurses' responses on positive aspects of using the RDTs.
More than half the nurses (12/20) however reported that they did not receive external training on RDT use; however, 10 nurses reported receiving in-house training. Almost all (19/20) nurses said that they were confident in using the RDT.
The accuracy of RDT result was the key concern for 35% of the respondents, false positive and negatives were stated as major challenges.
Managers and laboratory technologists' views
Malaria managers commented that RDT readings were problematic – "the fault could have been due to the clinic staff not reading the test in time." One laboratory technologist said that the nurses were not doing the test properly – "When we receive the test back we see that they are putting too much blood."
Although this study involved a small representative sample of users of RDT at the primary care facilities in South Africa, it has revealed that (1) there are sporadic problems of stock-out of RDT during peak transmission seasons; (2) the storage facilities and monitoring is inadequate in many primary care facilities; (3) the accuracy of the RDT was tested on an adhoc basis leading to nurses sometimes offering antimalarials to RDT-negative cases on the basis of their clinical judgment; (4) there is a possibility that RDTs under-diagnose malaria in the current context and (5) there is very limited training on the use of RDT at the PHC level.
According to the WHO, temperatures above 30°C are considered inappropriate for storing RDTs . As temperatures above 30°C can affect overall performance of the RDTs, temperature-monitoring needs to take place in all clinics and environments. Air-conditioning or similar cooling equipment should be considered in those clinics that exceed the WHO recommended threshold . RDT should be stored in a centralized store as long as possible and care should be taken during transport and storage at the health facilities to minimise degradation. Use of positive control wells and temperature monitors should be considered in South Africa to assure the quality of the RDTs and to build confidence of the users on RDT [9–11].
The quality of the RDT can be established at three levels: post-manufacture level, end user level and through the use of positive control wells [9, 12]. Due to the uncertainty of the quality of the test and lack of confidence in some cases of interpreting the results, patients were getting inappropriate treatment. For example some cases were given antimalarial treatment on clinical diagnosis even if the RDT was negative. It is possible to obtain false negative RDT results . However there is no system of evaluating the performance of RDT in the routine health services and to build confidence among the users of RDT. Very few users at the primary health care clinics were formally trained in performing and interpreting RDTs. Thus an in-service training and quality control system is needed urgently to ensure appropriate use of RDTs and effective treatment of malaria in South Africa. Although package inserts are useful it would be easier for the end-user to have posters or job aids so that the test procedure can be easily visible and read especially during busy periods and late in the night [13–15]. End user proficiency testing such as those described in other studies may be considered[13, 14, 16].
This study in South Africa may not be applicable to other countries in Africa. However, the lessons on storage, quality assurance and training observed in this study would be applicable to most settings where RDTs are introduced. Further studies of the factors influencing the appropriate use of RDTs and the ways to improve the use of RDTs are needed in the study setting.
All the authors conceptualized the study, participated in the analysis, drafting of the manuscript and writing the final version of the paper.
Ameena Ebrahim Goga, John Frean, Philip Kruger and Daniel Chandramohan contributed equally to this work.
Health ND: Guidlelines for the Treatment of Malaria in South Africa. 2002
Moonasar D, Johnson CL, Maloba MRB, al. : Malaria. South African Health Systems Review. 2004, South Africa , The Press Gang, 243-256.
Moody A: Rapid diagnostic tests for malaria parasites. Clinical microbiology reviews. 2002, 15 (1): 66-78. 10.1128/CMR.15.1.66-78.2002.
WHO: Malaria Rapid Diagnosis, Making it Work. RS/2003/GE/05(PHL)
Annexo M, Tolhurst R, et.al.: Malaria Misdiagnosis: Effects on the Poor. The Lancet. 2004, 364 (20): 1896-1898. 10.1016/S0140-6736(04)17446-1.
Reyburn H, Mbatia R, Drakeley C, Carneiro I, Mwakasungula E, Mwerinde O, Saganda K, Shao J, Kitua A, Olomi R, Greenwood BM, Whitty CJ: Overdiagnosis of malaria in patients with severe febrile illness in Tanzania: a prospective study. BMJ (Clinical research ed. 2004, 329 (7476): 1212-10.1136/bmj.38251.658229.55.
Hennekens CH, Buring JE: Epidemiology in Medicine. 1987, Boston , Little Brown & Company
Silverman D: Doing Qualitative Research. 2005, London , Sage, 2
Bell D: Is there a Role for malaria rapid diagnostic tests in Africa? . WHO, September 2004
Lon CT, Alcantara S, Luchavez J, Tsuyuoka R, Bell D: Positive control wells: a potential answer to remote-area quality assurance of malaria rapid diagnostic tests. Transactions of the Royal Society of Tropical Medicine and Hygiene. 2005, 99 (7): 493-498. 10.1016/j.trstmh.2005.01.001.
Jorgensen P, Chanthap L, Rebueno A, Tsuyuoka R, Bell D: Malaria rapid diagnostic tests in tropical climates: the need for a cool chain. The American journal of tropical medicine and hygiene. 2006, 74 (5): 750-754.
WHO: The use of Malaria Rapid diagnostic tests. 2004
Tavrow P: Using Quality Design to improve Malaria Rapid Diagnostic tests in Malawi. Quality Assurance Project (QAP) for the United states Agency for international development. (online), (Bethesda MD, USA). [http://www.qaproject.org/pubs/pdfs/malariaforweb.pdf]
Rennie W, Phetsouvanh R, Lupisan S, Vanisaveth V, Hongvanthong B, Phompida S, Alday P, Fulache M, Lumagui R, Jorgensen P, Bell D, Harvey S: Minimising human error in malaria rapid diagnosis: clarity of written instructions and health worker performance. Transactions of the Royal Society of Tropical Medicine and Hygiene. 2007, 101 (1): 9-18. 10.1016/j.trstmh.2006.03.011.
Knebel F, Lundahl S, Edward-Raj A, Abdulla H: The Use of Manual Job Aids by Health Care Providers: What do We Know? . Quality Assurance Project Bethesda. 2000. accessed: 27 February 2007, [http://www.qaproject.org/pubs/PDFs/ISSUESJA.PDF]
Mayxay M, Newton PN, Yeung S, Pongvongsa T, Phompida S, Phetsouvanh R, White NJ: Short communication: An assessment of the use of malaria rapid tests by village health volunteers in rural Laos. Trop Med Int Health. 2004, 9 (3): 325-329. 10.1111/j.1365-3156.2004.01199.x.
The authors would like to acknowledge the assistance of the district malaria managers of three districts in the Limpopo Province and the Environmental Health Practitioners for coordinating the study visits. The authors would also like to acknowledge the support of the Ernest Openheimer Trust and the South African National Department of Health for financial assistance for conducting this study.