Skip to main content

Advertisement

Measuring malaria diagnosis and treatment coverage in population-based surveys: a recall validation study in Mali among caregivers of febrile children under 5 years

Article metrics

Abstract

Background

Nationally-representative household surveys are the standard approach to monitor access to and treatment with artemisinin-based combination therapy (ACT) among children under 5 years (U5), however these indicators are dependent on caregivers’ recall of the treatment received.

Methods

A prospective case–control study was performed in Mali to validate caregivers’ recall of treatment received by U5s when seeking care for fever from rural and urban public health facilities, community health workers and urban private facilities. Clinician-recorded consultation details were the gold standard. Consenting caregivers were followed-up for interview at home within 2 weeks using standard questions from Demographic and Health Surveys and Malaria Indicator Surveys.

Results

Among 1602 caregivers, sensitivity of recalling that the child received a finger/heel prick was 91.5%, with specificity 85.7%. Caregivers’ recall of a positive malaria test result had sensitivity 96.2% with specificity 59.7%. Irrespective of diagnostic test result, the sensitivity and specificity of caregivers’ recalling a malaria diagnosis made by the health worker were 74.3% and 74.9%, respectively. Caregivers’ recall of ACT being given had sensitivity of 43.2% and specificity 90.2%, while recall that any anti-malarial was given had sensitivity 59.0% and specificity 82.7%. Correcting caregivers’ response of treatment received using a combination of a visual aid with photographs of common drugs for fever, prescription documents and retained packaging changed ACT recall sensitivity and specificity to 91.5% and 71.1%, respectively.

Conclusions

These findings indicate that caregivers’ responses during household surveys are valid when assessing if a child received a finger/heel prick during a consultation in the previous 2 weeks, and if the malaria test result was positive. Recall of ACT treatment received by U5s was poor when based on interview response only, but was substantially improved when incorporating visual aids, prescriptions and drug packaging review.

Background

To assess coverage of recommended first-line anti-malarial drugs, malaria control programs often draw on data from large-scale population-based household surveys, such as the Demographic and Health Survey (DHS) and Malaria Indicator Survey (MIS) [1,2,3]. The standard indicator used to assess if treatment guidelines are being adhered to is the proportion of children receiving ACT (or other recommended treatment) among children under 5 years (U5) with fever in the last 2 weeks who received any anti-malarial drugs [3]. This estimate of ACT coverage is used by National Malaria Control Programmes to understand whether children are receiving appropriate treatment for uncomplicated malaria when seeking care from public health facilities, community health workers and private health services. In addition, this indicator contributes to analysis estimating global estimates of diagnosis and treatment services, performed by the World Health Organization (WHO) and other international organizations [4, 5].

The proportion of U5s with fever in the last 2 weeks who had a finger or heel prick is commonly collected in population-based surveys to provide information about access to confirmatory diagnostics for malaria at the locations where caregivers seek care for the child. However, further questions such as whether caregivers were told the result of the test, if they remember the result of the test, and if the caregivers were told the final diagnosis made by the health worker are not routinely included in population surveys, but have the potential to further contribute to monitoring of malaria diagnosis and treatment practices if found to be valid.

The potential for bias in caregiver recall of diagnosis and treatment of febrile children was among multiple challenges identified in the use of population-based surveys to measure key maternal and child health indicators [6]. A validation study at public health facilities in Zambia found that caregiver recall of ACT received by a febrile child under 5 years of age had high sensitivity and specificity [7]. Zambia has successfully limited the use of monotherapies for malaria treatment [8], and has relatively low levels of treatment-seeking from the private sector [9], therefore, there is a need to replicate this study in settings where a wider range of drugs could potentially be prescribed for malaria, and among caregivers attending private sector facilities.

Mali was identified as an appropriate location to further explore the validity of caregiver recall of ACT received by children under five. Findings in recent household surveys in Mali indicate that caregivers reported non-ACT drugs such as amodiaquine, sulfadoxine-pyrimethamine (SP) or chloroquine were given to their sick children for uncomplicated malaria during visits to health facilities [10, 11]. Recent introduction and expansion of seasonal malaria chemoprevention (SMC) among U5 s [12] has the potential to influence caregiver recall of anti-malarial drugs used for curative purposes. Seasonal malaria chemoprevention aims to reduce malaria morbidity and mortality in children under five in high transmission settings by administration of SP and amodiaquine to children aged 3–59 months at 1-month intervals for up to 4 months during the peak transmission season [13]. Patients in Mali often seek treatment from a combination of public and private sector locations [10], and while policies are in place to provide free malaria diagnosis and treatment in the public sector, public health facilities operate on a cost-recovery basis and are dependent on user fees and pharmaceutical sales [14]. Literacy rates among Malian women are low [10], and a previous study identified inconsistencies in local language terms for specific anti-malarial medicines [15].

This study aimed to assess the validity of malaria diagnosis and treatment coverage indicators collected during household surveys. Specifically, the validity of caregiver-reported receipt of anti-malarials for febrile children seeking treatment from community health workers (CHWs), public facilities in rural and urban areas, and private facilities in urban areas both prior to and during the SMC campaign period were ascertained.

Methods

Study site

The study was conducted in urban areas of Bamako and Sikasso, and in rural areas of Niena District, Sikasso region. Bamako has a population of approximately 2.3 million, while the population of Sikasso town is approximately 600,000 and Niena district 160,000. Niena district and Sikasso town are in the Sudano-Guinean zone of high malaria transmission, while Bamako is in the Sahelian zone and experiences lower levels of malaria transmission due to the urban environment [16]. The Sudano-Guinean zone experiences perennial malaria transmission with a seasonal peak during June–November, while the Sahelian zone has a shorter transmission season and similar seasonal peak.

Mali adopted ACT as treatment for uncomplicated malaria in 2005, replacing chloroquine [17]. At present, artemether–lumefantrine is the recommended first-line treatment for uncomplicated malaria, with artesunate–amodiaquine as the second-line. National malaria treatment guidance recommends that children with severe malaria receive an injection of artesunate, artemether or quinine (according to availability), followed by a full dose of ACT as soon as they are able to ingest the drug without vomiting [17]. Laboratory diagnosis (by microscopy or RDT) and treatment of malaria is free at all levels of the health system for U5s and pregnant women.

Mali introduced a policy of SMC in 2012, with implementation commencing in 2014. Nationwide SMC was conducted for the first time in 2016, implemented door-to-door using a dispersible format of co-blistered SP and amodiaquine. During the SMC campaign, any child eligible for SMC who has a current or reported fever was tested by RDT. Children with a positive RDT and no signs of severe disease received ACT in accordance with standard treatment guidelines, and did not receive SMC drugs. Coverage of SMC was high across all study areas in 2017: 88–109% by round in urban Sikasso, 96–103% by round in Niena district, and 112–124% by round in Bamako (S. Fomba, personal communication). Coverage exceeding 100% is a result of the number of children receiving SMC exceeding the estimated eligible population.

Four different types of health care services were included in the study: urban private health facilities, urban public health facilities, rural public health facilities and CHWs. Public health facilities were purposively selected for inclusion in the study based on their weekly patient flow and mean weekly number of confirmed malaria cases in children U5 during the 2016 malaria transmission season. No database capturing the number of malaria cases recorded in private health facilities exists, therefore, private facilities were identified using local knowledge about facilities with high attendance by children under 5 years. Initially, two public urban facilities [one district referral health center (CSRef) in Sikasso, one community health centre (CSCom) in Bamako], two public rural facilities (CSComs) in Niena district, eight CHWs working in the catchment area of the two rural CSComs, and two private facilities in Bamako were selected for participation. Four additional private urban facilities were added due to slow recruitment of participants in the first weeks of the study, with only the two highest-recruiting private facilities retained for the second half of the study. One additional CHW in the same Niena catchment area was added halfway through the study (Fig. 1).

Fig. 1
figure1

Map of participating public urban facilities, public rural facilities, and private urban facilities. Participating community health workers were close to the two public rural facilities

Study design, participants and data collection procedures

A prospective unmatched case control study design was used to assess the validity of primary coverage indicators for diagnosis and treatment of malaria in children U5, as estimated in household surveys. Caregivers aged at least 18 years attending study health facilities or CHWs with a current/recently febrile child U5 were eligible to be recruited to the study. Data collection was divided into two periods: July 2017, prior to the start of the SMC campaign; and September–November 2017, concurrent with the SMC campaign.

For the ACT coverage indicator, ‘cases’ were defined as caregivers of children aged one to 59 months (U5) who were prescribed ACT at the health facility or CHW site, while ‘controls’ were caregivers of children U5 who attended the health facility or CHW site, but were not prescribed ACT. Children with signs of severe malaria or other serious illness were excluded.

Details of the child’s consultation, including diagnostic used (if any), test result, final diagnostic decision of health worker, and drugs prescribed were recorded by the health worker during or immediately after the consultation using simple data collection forms. These served as the ‘gold standard’ of procedures and drugs during the consultation [18]. Health workers were told the general aim of the study, but advised to continue their consultation and prescribing practices as normal and not to divulge the specific aim of the study to participants.

Eligible caregivers were asked by the health worker if they would like to be involved in a study related to child health, and interested caregivers were consented and registered by the study team at the health facility (or by the CHW at CHW sites). Caregivers provided basic information to allow a follow-up interview at their home. Follow-up interviews were randomised to be completed either 1–7 or 8–14 days following the consultation.

Informed consent was obtained from caregivers when visited at their home for the follow-up interview. The interview was directed to the caregiver who attended the facility or CHW site with the child, and used a standardised questionnaire based on the DHS relating to recent fever and treatment-seeking for children U5 years. The questionnaire also included standard DHS questions relating to caregiver’s sex, age, literacy, and education level, as well as ownership of household assets. At the end of the questionnaire, further detail on treatment received was collected using a visual aid with photographs of common drugs prescribed for fever in Mali (Additional file 1), and requesting to review documents or packaging retained by the caregiver from the consultation.

All data were collected on paper-based forms, then double-entered into a CSPro template and discrepancies resolved by reviewing original questionnaires. Drug names and brands reported by caregivers that were not pre-coded in the questionnaire were classified by first searching the ACTwatch database of anti-malarial drugs [19], then using broader internet searches if not found in the anti-malarial database.

Sample size

A sample size of 200 cases (received ACT) and 200 controls (did not receive ACT) were required for each of the four types of recruitment site (public urban, public rural, private urban and CHW). The sample size calculation assumed a 5% probability of committing a type-1 error (2-tailed, sensitivity of 60–70%, specificity of 70–80%, precision of ± 7%, and interview refusal rate of 10%. The total required sample size was 1600, split evenly between the periods before and during the SMC campaign.

Statistical analysis

Data management and cleaning were performed with Stata version 14. Primary outcomes for this study were the sensitivity, specificity and accuracy of caregivers’ recall of the child having received a finger/heel prick test for malaria, result of the malaria test, final diagnosis made by the health worker, and treatment received.

Additional outcomes of corrected ACT recall were created, where the caregivers’ recall of ACT was defined as reporting during the interview that ACT was received for the child, identifying ACT on the visual aid, having a prescription document listing ACT received by the child, or having retained ACT packaging from the child’s recent illness. These “corrected ACT recall” variables were generated for each correction method separately (visual aid, prescription, or retained packaging) and where ACT was reported by interview or any of the correction methods.

Household socio-economic status was derived using principal components analysis of household assets, then divided into wealth quintiles [20]. Both binary assets (e.g. radio ownership) and continuous variables (number of cows, sheep, etc.) were included in the principal components analysis. Assets with ownership frequency < 5% were excluded.

Sensitivity, specificity and accuracy and their respective 95% confidence intervals (CIs) were estimated using the Huber–White Sandwich estimator to account for correlated data within each health facility or CHW site [21]. Sensitivity represents the proportion of caregivers who correctly recalled that the child received ACT; specificity is the proportion of caregivers who correctly recalled that the child did not receive ACT; and accuracy represents the proportion of caregivers whose recall of whether ACT was received by the child agreed with the gold standard records made by the clinician during the consultation. Differences in sensitivity, specificity and accuracy by caregiver and child characteristics were assessed using a Chi squared test, or by Fishers’ exact test where any cell value was below 10.

Logistic regression models were used to estimate adjusted odds ratios for facility, caregiver and child characteristics associated with sensitivity, specificity and accuracy of ACT recall. Sensitivity, specificity or accuracy of ACT recall were included as primary binary outcomes in three separate models, with a further set of models generated for corrected ACT recall (recall including visual aids, review of prescription and retained packaging). The sensitivity model binary outcome variable was 1 when caregivers correctly reported that the child received ACT, and 0 when caregivers incorrectly reported that the child did not receive ACT when they did. The specificity model binary outcome variable was 1 when caregivers correctly reported that the child did not received ACT, and 0 when the caregiver incorrectly reported that the child received ACT when they did not. The accuracy binary outcome variable was 1 when caregivers correctly reported if the child did or did not received ACT, and 0 when the caregiver incorrectly reported if the child did or did not receive ACT. The models developed included site of enrolment as a random intercept, with all other variables of interest included as fixed effects.

Individual-level validity (of recall of test, test result, diagnosis, treatment) was estimated by calculating the area under the receiver operator curve (AUC) for plots of sensitivity against 1-specificity for all sites combined, and for each type of site individually [18]. Specific treatment recall variables were recall of any anti-malarial treatment, recall of ACT, and corrected recall of ACT.

Population-level validity of recall was assessed by estimating an inflation factor (IF), which gives an estimate of the extent to which the survey-based estimates provide an unbiased estimate of the true coverage [18]. Briefly, using measured sensitivity and specificity of test, test result, diagnosis and treatment recall, the coverage that would be measured by a household survey is modelled across a range of ‘gold standard’ coverage levels. Stata code produced by Munos et al. was used to prepare IF plots [18].

Results

A total of 1789 caregivers were approached to participate in the study, with 1665 (93%) accepting. Follow-up interviews were successfully completed with 1602 caregivers, with 49% being caregivers of children who received ACT during the consultation (Fig. 2). Participant enrolment over time is described further in Additional file 2. Only 38.0% of participants were recruited in July, prior to the start of the SMC campaign, lower than the 50% target.

Fig. 2
figure2

Participant enrolment flowchart

The majority (82%) of enrolled children were taken to the health facility by a female caregiver, but the proportion varied by type of facility (p < 0.001, Table 1). Overall, 46.9% of caregivers had not attended formal education, and 58.1% could not read. Both these characteristics showed considerable variation by type of health facility (p < 0.001). Slightly more male children were recruited than female children, with no evidence for difference by type of facility.

Table 1 Demographics of enrolled children and attending caregivers who completed both registration at health facility and follow-up interview

Across all sites, half (50%) of children received a confirmatory diagnostic test for malaria, and 73% of those tested had a positive test result (Table 2). Irrespective of test results, 76% of all participating children received a diagnosis of malaria from the health worker, 49% received ACT, and 79% received an anti-malarial drug of any type. Non-ACT anti-malarials prescribed were primarily injectable artesunate, but also included artemether, amodiaquine and quinine. Health worker diagnostic and prescription practices reported during this study will be presented in detail in a separate publication.

Table 2 Summary of malaria diagnostic procedures and treatment prescribed at participating health facilities and community health worker (CHW) sites, as recorded by the health worker during or immediately after consultation

Table 3 presents the sensitivity, specificity and accuracy of caregiver’s recall of diagnosis and treatment the child received during the consultation, against the gold standard recorded by the health worker. Caregivers’ recall of a finger/heel prick and a malaria test result were good; there was a high sensitivity and specificity of recall of a finger/heel prick (91.5% and 85.7%, respectively), and high sensitivity but moderate specificity for recall of a positive test result (96.2% and 59.7%, respectively). Caregivers’ recall of a malaria diagnosis being made by the health worker had moderate sensitivity and specificity (74.3% and 74.9%, respectively). Caregivers’ recall of treatment had low sensitivity for both any anti-malarial (59.0%) and ACT (43.2%), however specificity was high for both (82.7% and 90.2, respectively). Documents describing drugs provided to the child had been retained by 87.6% of caregivers, and were available for review by the interviewer among 82.2% of these individuals. Drug packaging had been retained by 87.4% of caregivers, with 98.8% of these caregivers able to show the drug packet to the interviewer. Sensitivity of ACT recall was dramatically increased by incorporating identification of ACT from visual aids, prescriptions or retained packaging (91.5%), but with a slight reduction in specificity to 71.7%. Sensitivity, specificity and accuracy of recall of diagnosis and treatment are presented by facility type in Additional file 3.

Table 3 Sensitivity, specificity and accuracy of caregivers’ recall of diagnosis procedures at facilities and treatment received during consultation, summarized for all facility types

In multivariate models, sensitivity of ACT recall was higher at CHW sites than in the reference group of public urban facilities (adjusted odds ratio [AOR] of 12.2), with no significant differences between other site types (Table 4). Sensitivity of recall was also significantly lower among caregivers from the two poorest quintiles than caregivers from the richest quintile (AOR = 0.20 and AOR = 0.37, respectively). Literacy was associated with sensitivity of ACT recall, with those who were illiterate or could read only part of a test sentence having lower sensitivity than literate caregivers (AOR 0.27 and AOR 0.38, respectively). Specificity of ACT recall differed by study period, with lower specificity during the period of SMC implementation compared with the period before SMC campaigns began (AOR 0.33). When reviewing the multivariate model for “corrected ACT recall” which included caregiver identification of ACTs from visual aids, prescriptions or retained packaging (Additional file 4), attending a CHW site was no longer significantly associated with recall sensitivity (p = 0.077), however the associations between recall sensitivity and caregiver literacy (AOR = 0.29, p = 0.006 comparing those who can read only a little against those who were literate) and between specificity and SMC were retained (AOR 0.53, p = 0.003). Summaries of sensitivity, specificity and accuracy of ACT recall by various facility, caregiver and child characteristics, together with Chi squared test p-values are presented in Additional file 5, with summaries for ACT recall corrected by visual aid, prescription or packaging in Additional file 6.

Table 4 Random effects multivariate logistic regression models of sensitivity, specificity and accuracy of caregivers’ recall of treatment with artemisinin-based combination therapy (ACT): associations with follow-up and socio-demographic characteristics

Individual-level validity of diagnosis and treatment recall (defined by those that correctly recalled the event in question) is estimated using AUC (Table 5, Fig. 3). Recall of a finger/heel stick, a positive malaria test, and a malaria diagnosis by the health worker all had AUC > 0.7, indicating good individual-level validity, with some variation by facility type. Caregivers’ recall of ACT being provided to the child had only moderate individual-level validity at all site types except at CHW sites, where AUC was 0.75. Including visual aids, prescription and packaging review in ACT recall increased AUC > 0.7 at all site types (Additional files 7, 8).

Table 5 Area under the curve (AUC) of the receiver operating characteristic (ROC) including 95% confidence intervals for each of the main recall indicators, across all sites and by type of site
Fig. 3
figure3

Receiver operating characteristic (ROC) curve by site for recall of artemisinin-based combination therapy (ACT), and ROC by site type for corrected recall of ACT

Table 6 shows modelled estimates of the inflation factor for recall of each of the diagnosis and treatment indicators. Population-level bias is low (0.75–1.25) for recall of a finger/heel prick and a positive malaria test at all site types. Population-level bias was moderate to high (0.47 to 0.62 by site type) for recall of ACT, however bias was reduced when modelling corrected recall (including ACT identified by visual aid, retained packaging or prescription). Modelled estimates of coverage of each recall indicator for a range of coverage levels are shown in Fig. 4. Sensitivity and specificity of caregivers’ recall of a positive malaria test indicates that this indicator would likely be over-estimated at low levels of parasitological testing, but is accurate when a parasitological testing rates are high. Coverage of ACT is likely to be increasingly underestimated at higher levels of ACT coverage, but is estimated to be more accurate where fewer children receive ACT when seeking treatment for fever. When ACT recall is corrected using visual aids, prescription and packaging review, modelled estimates of ACT coverage are a closer match to true coverage across a wider range of ACT coverage levels.

Table 6 Inflation factor (IF) including 95% confidence intervals for each of the main recall indicators, across all sites and by type of site
Fig. 4
figure4

Inflation factor scatterplots for (i) blood test; (ii) positive blood test; (iii) malaria diagnosis (clinical or confirmed); (iv) receipt of any anti-malarial; (v) receipt of artemisinin-based combination therapy (ACT); (vi) receipt of ACT, corrected by visual aid, prescription review or retained packaging review

Discussion

This study investigated the validity of caregivers’ recall of the diagnosis and treatment received by children U5 years when seeking care for a fever (without severe disease), as measured by a household survey questionnaire in Mali. Recall of the child receiving a finger or heel prick, assumed to mean that the child was tested for malaria by microscopy or a rapid diagnostic test, had high sensitivity and specificity. However caregivers’ recall of treatment received by the child had low sensitivity, meaning that caregivers would falsely report during the interview that the child did not receive anti-malarials, when the gold standard measure indicates that they did receive an anti-malarial. Sensitivity was lower for receipt of ACT than for receipt of any type of anti-malarial, indicating that some caregivers knew their child was given an anti-malarial, but few correctly reported that the child received ACT. This finding was also clear from questionnaires, where some caregivers reported that their child received anti-malarial pills or an injection, but did not know the name of the drug.

The result of the finger or heel prick test (assumed to be a malaria test) and the health worker’s diagnosis for the child are not commonly included in DHS or MIS surveys. Our data suggest caregivers’ recall of a confirmed malaria diagnosis had high individual accuracy (based on AUC) and low population-level bias (based on an inflation factor) in this context, although specificity was only moderate due to caregivers incorrectly reporting the child had a positive malaria test result when the result was negative. These false positive responses may be related to incomplete adherence of health workers to negative malaria test results; instances of health workers prescribing anti-malarials to children regardless of a negative microscopy or RDT result were recorded during the study.

Indicators of individual- and population-level bias were worse for caregivers’ recall of a malaria diagnosis made by the health worker than for caregivers’ recall of a positive malaria test result, particularly among those caregivers who attended the public rural health facilities. It should be noted, however, that while only 49% of children recruited from public rural facilities were classified as cases (received ACT), 94% of children recruited at these facilities were diagnosed as malaria cases by health workers (presumptive and confirmed); this resulted in wide confidence intervals for estimates of specificity of caregivers’ recall of malaria diagnosis at public rural facilities.

Currently, standard household surveys do not measure the proportion of children receiving appropriate treatment for malaria because the denominator (children with confirmed malaria) has been considered too difficult to accurately measure in a survey setting. Alternative methods have been used to estimate appropriate treatment of malaria by restricting ACT coverage indicators to those children who had a positive Plasmodium falciparum histidine-rich protein 2 (HRP2) RDT result during the survey [5], since HRP2 antigen persists for several weeks after clearance of the infection [22, 23]. The results of this study suggest that caregivers’ recall of a positive malaria test result, in this context of high test positivity rates, may be sufficiently reliable to incorporate in standard population-based survey questionnaires. However, further validation of this indicator is recommended in settings with lower test positivity rates among febrile children, and in settings with low testing rates. Including an indicator of ACT coverage among children with confirmed malaria in standard population-based malaria surveys is increasingly feasible as countries continue to expand access to confirmatory diagnostics at public health facilities, at community-level, and in the private sector [4, 24].

Caregivers’ recall was substantially improved by correcting the initial response to the interview question asking what drugs the child received by including a visual aid, or requesting to see any retained prescription document or drug packaging retained from the facility or CHW visit. When based on the standard question response only, ACT recall had moderate individual-level accuracy and moderate population-level bias. However, by incorporating caregivers’ identification of ACT medicines from the photographs of common drugs prescribed for fever, or from prescriptions and packaging retained by the caregiver, AUC and IF estimates improved, indicating reduced individual- and population-level bias. Including all three additional methods to identify ACT medicines received by the child resulted in low to moderate population-level bias, and high individual-level accuracy. Inclusion of visual aids has been demonstrated to improve caregiver recall of pneumonia treatment in a survey in Pakistan and Bangladesh [25], and challenges in correctly identifying and differentiating anti-malarial drugs have previously been reported in Mali [15, 26]. Use of visual aids has been previously recommended to improve reporting validity of maternal and child health indicators in settings with multiple sources of medicines [6], and is included in DHS and MIS interviewer guidelines [27, 28], but has not been consistently included in all DHS and MIS surveys.

Improved ACT recall sensitivity (lower proportion of false negatives) was found to be associated with the treatment seeking event occurring with a CHW, having higher household wealth and being able to read. We hypothesize that improved recall among caregivers taking their child to a CHW could be a result of CHWs spending more time discussing the diagnosis and treatment with caregivers, or simply the smaller range of drugs available. Studies in Uganda reported communication gaps between patient and health workers, with limited explanation of diagnostic procedures and treatment provided [29], and higher perceived quality of care from CHWs than from public health facilities [30]. Literacy and education levels have been found to be associated with adherence to ACT [31], however, the previous recall validation study in Zambia found no association between sensitivity of ACT recall and caregivers’ education [7].

This study found some potential information bias in reporting of ACT received by the child during the period of SMC implementation, indicated by a significant decline in specificity of ACT recall during the SMC period compared with the pre-SMC period. During the SMC campaign in Mali, any child with fever is tested by RDT, and if the RDT is positive the child receives ACT instead of the SMC drug (sulfadoxine–pyrimethamine and amodiaquine). It is likely that some of these apparent false positive reports of ACT were the result of febrile and RDT positive children receiving ACT as a part of the SMC campaign, however it is not possible to confirm this without review of individual-level SMC campaign records. It should also be noted that not all countries implementing SMC have adopted this approach of testing febrile children with RDTs on the SMC campaign day.

This study specified that any child with signs of severe malaria or other severe disease should be excluded. However, signs and symptoms exhibited by participating children were not systematically recorded during the consultation or enrolment process. While both the study team members at health facilities and the clinicians conducting consultations at study locations were trained on the exclusion criteria, it is possible that some children with severe malaria were included in the study. Nevertheless, the rate of injectable anti-malarial use was high, and supports the conclusion that injectable anti-malarials were being provided to children with uncomplicated malaria.

Limitations of this study included the potential for caregivers’ recall to be biased as a result of contact with the study team during registration at the facility. While the specific aim of the study was not disclosed at the facility, interaction with the study team at the facility to enroll in a “child health study” could have biased recall. Similarly, the consultation record forms that provided the gold standard were completed by the health workers, without independent observation of the consultation to validate the information recorded. It was not possible to match child-level data from the study to SMC registers to confirm whether children received ACT during the SMC campaign, which may have biased ACT recall specificity downwards during the SMC period. Finally, this study includes a relatively small number of facilities, although from a variety of settings, so findings may not be directly applicable to different settings. However the results differ from those collected during a similar study in Zambia: a setting with high parasitological testing rates, high use of the public sector and no stock-outs of ACT medicines during the study period. While the Zambia study found that caregivers’ recall of ACT was good, our study indicates that in a high transmission setting where multiple anti-malarial drugs are commonly available and private sector use is higher, ACT recall has low sensitivity, unless additional tools such as visual aids are used. While the Zambia study directed the follow-up questionnaire to the child’s caregiver and asked separately who took the child to the health facility, the current study differed in that the questionnaire was directed to the individual who took the child for treatment. This could potentially have biased results from this study in Mali towards improved recall, compared to the study in Zambia.

This study found unexpectedly high use of non-ACT anti-malarials. The National Malaria Treatment Guidelines in Mali state that artemether–lumefantrine is the first line drug, with artesunate–amodiaquine as second-line. For severe malaria an injection of artesunate, artemether or quinine (depending on availability) is recommended, followed by a full course of ACT after the child is able to safely ingest the drugs. A stock-out of ACT was identified at the rural public health facilities during a supervision visit, which may have contributed to the use of injectable anti-malarials, however non-ACT anti-malarials were prescribed at all facility types. A subsequent publication will present detailed discussion of diagnosis and prescription practices recorded during this study.

Conclusions

Indicators of parasitological testing for malaria and the result of that test were found to have high individual validity and low population bias in the examined sites in Mali, indicating that caregiver response to these questions in household surveys is comparable to the gold standard recorded by health workers. The slightly low specificity of positive test recall predicts that surveys would over-estimate this indicator in areas where a low proportion of tests are positive, however this finding should be confirmed in low transmission settings. Caregivers’ recall of ACT received by the child when assessed through an interview response only had low sensitivity, largely as a result of recall error (not knowing drug names) rather than information error (caregiver providing an incorrect response when they do not know). Sensitivity of ACT recall was substantially improved by including visual aids, review of prescriptions and any retained packaging. We recommend that future household surveys collecting standard malaria treatment indicators include at least one of these additional methods (visual aids, prescription or retained packaging review) to assist the caregiver in identifying if the child received ACT. These additional tools may be particularly beneficial in settings with low literacy among caregivers, and where a variety of anti-malarial medicines may be available from the public and/or private sector.

References

  1. 1.

    Measure DHS. Demographic and health surveys. http://www.measuredhs.com/aboutsurveys/dhs/start.cfm. Accessed 2 July 2018.

  2. 2.

    Roll Back Malaria Partnership: Monitoring and Evaluation Reference Group (MERG). Malaria indicator survey. http://www.malariasurveys.org/toolkit.cfm. Accessed 19 Feb 2015.

  3. 3.

    MEASURE Evaluation, MEASURE DHS, President’s Malaria Initiative, Roll Back Malaria Partnership, UNICEF, World Health Organization. Household survey indicators for malaria control; 2013. http://www.rollbackmalaria.org/toolbox/docs/rbmtoolbox/tool_HouseholdSurveyIndicatorsForMalariaControl.pdf. Accessed 6 Mar 2014.

  4. 4.

    WHO. World malaria report 2017. Geneva: World Health Organization; 2017. http://www.who.int/malaria/publications/world-malaria-report-2017/report/en/. Accessed 29 Nov 2017.

  5. 5.

    Bennett A, Bisanzio D, Yukich JO, Mappin B, Fergus CA, Lynch M, et al. Population coverage of artemisinin-based combination treatment in children younger than 5 years with fever and Plasmodium falciparum infection in Africa, 2003–2015: a modelling study using data from national surveys. Lancet Glob Health. 2017;5:e418–27.

  6. 6.

    Bryce J, Arnold F, Blanc A, Hancioglu A, Newby H, Requejo J, et al. Measuring coverage in MNCH: new findings, new strategies, and recommendations for action. PLoS Med. 2013;10:e1001423.

  7. 7.

    Eisele TP, Silumbe K, Yukich J, Hamainza B, Keating J, Bennett A, et al. Measuring coverage in MNCH: accuracy of measuring diagnosis and treatment of childhood malaria from household surveys in Zambia. PLoS Med. 2013;10:e1001417.

  8. 8.

    Sipilanyambe N, Simon JL, Chanda P, Olumese P, Snow RW, Hamer DH. From chloroquine to artemether–lumefantrine: the process of drug policy change in Zambia. Malar J. 2008;7:25.

  9. 9.

    Population Services International, ACTwatch. Malaria Market Trends in Sub-Saharan Africa: 2009–2014; 2015. http://www.actwatch.info/sites/default/files/content/publications/attachments/SSA%20Brief_FINAL_NO_BLEEDS.pdf. Accessed 28 Jan 2016.

  10. 10.

    Programme National de Lutte contre Le Paludisme Ministère de la Santé Publique et de l’Hygiène Publique, Institut National de la Statistique Ministère de l’Aménagement du Territoire et de la Population, INFO-STAT, Institut National de la Recherché en Santé Publique Ministère de la Santé Publique et de l’Hygiène Publique Bamako, ICF International. Enquête sur les Indicateurs du Paludisme au Mali 2015; 2016. http://dhsprogram.com/pubs/pdf/MIS24/MIS24.pdf. Accessed 11 Oct 2016.

  11. 11.

    Cellule de Planification et de Statistique (CPS/SSDSPF), Institut National de la Statistique (INSTAT/MPATP), INFO-STAT, ICF International. Enquête Démographique et de Santé au Mali 2012-2013; 2014. http://www.dhsprogram.com/pubs/pdf/FR286/FR286.pdf. Accessed 10 Oct 2016.

  12. 12.

    Diawara F, Steinhardt LC, Mahamar A, Traore T, Kone DT, Diawara H, et al. Measuring the impact of seasonal malaria chemoprevention as part of routine malaria control in Kita, Mali. Malar J. 2017;16:325.

  13. 13.

    WHO. Seasonal malaria chemoprevention with sulfadoxine-pyrimethamine plus amodiaquine in children: A field guide. Geneva: World Health Organization; 2013. http://www.who.int/malaria/publications/atoz/9789241504737/en/. Accessed 19 July 2016.

  14. 14.

    Klein MC, Harvey SA, Diarra H, Hurley EA, Rao N, Diop S, et al. “There is no free here, you have to pay”: actual and perceived costs as barriers to intermittent preventive treatment of malaria in pregnancy in Mali. Malar J. 2016;15:158.

  15. 15.

    Patterson AE, Winch PJ, Gilroy KE, Doumbia S. Local terminology for medicines to treat fever in Bougouni District, Mali: implications for the introduction and evaluation of malaria treatment policies. Trop Med Int Health. 2006;11:1613–24.

  16. 16.

    Ministère de la Santé et de l’Hygiène Publique, Programme National de Lutte Contre le Paludisme: Plan Strategique de Lutte contre le Paludisme 2016–2018. Mali: Malian Ministry of Health and The National Malaria Control Program; 2017.

  17. 17.

    Castle S, Scott R, Mariko S. Malaria prevention and treatment for children under five in Mali: further analysis of the 2012–13 Demographic and Health Survey; 2014. http://dhsprogram.com/pubs/pdf/FA93/FA93.pdf. Accessed 20 Mar 2017.

  18. 18.

    Munos MK, Blanc AK, Carter ED, Eisele TP, Gesuale S, Katz J, Marchant T, Stanton CK, Campbell H, Improving Coverage Measurement G. Validation studies for population-based intervention coverage indicators: design, analysis, and interpretation. J Glob Health. 2018;8:020804.

  19. 19.

    ACTwatch. ACTwatch: antimalarial survey database; 2017. http://www.actwatch.info/databases/antimalarial_survey_data. Accessed 4 Dec 2017.

  20. 20.

    Vyas S, Kumaranayake L. Constructing socio-economic status indices: how to use principal components analysis. Health Policy Plan. 2006;21:459–68.

  21. 21.

    Williams RL. A note on robust variance estimation for cluster-correlated data. Biometrics. 2000;56:645–6.

  22. 22.

    Kyabayinze DJ, Tibenderana JK, Odong GW, Rwakimari JB, Counihan H. Operational accuracy and comparative persistent antigenicity of HRP2 rapid diagnostic tests for Plasmodium falciparum malaria in a hyperendemic region of Uganda. Malar J. 2008;7:221.

  23. 23.

    Swarthout TD, Counihan H, Senga RKK, van den Broek I. Paracheck-Pf accuracy and recently treated Plasmodium falciparum infections: is there a risk of over-diagnosis? Malar J. 2007;6:58.

  24. 24.

    ACTwatch Group, Hanson K, Goodman C. Testing times: trends in availability, price, and market share of malaria diagnostics in the public and private healthcare sector across eight sub-Saharan African countries from 2009 to 2015. Malar J. 2017;16:205.

  25. 25.

    Campbell H, El Arifeen S, Hazir T, O’Kelly J, Bryce J, Rudan I, Qazi SA. Measuring coverage in MNCH: challenges in monitoring the proportion of young children with pneumonia who receive antibiotic treatment. PLoS Med. 2013;10:e1001421.

  26. 26.

    Hurley EA, Harvey SA, Rao N, Diarra NH, Klein MC, Diop SI, Doumbia SO. Underreporting and missed opportunities for uptake of intermittent preventative treatment of malaria in pregnancy (IPTp) in Mali. PLoS ONE. 2016;11:e0160008.

  27. 27.

    The DHS Program. Demographic and Health Survey Interviewer’s Manual; 2017. https://dhsprogram.com/pubs/pdf/DHSM1/DHS7-Interviewer’s-Manual-EN-12Jun2017-DHSM1.pdf. Accessed 2 July 2018.

  28. 28.

    The DHS Program. MIS Interviewer’s Manual; 2016. http://www.malariasurveys.org/toolkitfiles/06%20MIS%20Interviewers%20Manual.docx. Accessed 2 July 2018.

  29. 29.

    Altaras R, Nuwa A, Agaba B, Streat E, Tibenderana JK, Martin S, Strachan CE. How do patients and health workers interact around malaria rapid diagnostic testing, and how are the tests experienced by patients in practice? a qualitative study in Western Uganda. PLoS ONE. 2016;11:e0159525.

  30. 30.

    Nanyonjo A, Makumbi F, Etou P, Tomson G, Kallander K, inSCALE Study Group. Perceived quality of care for common childhood illnesses: facility versus community based providers in Uganda. PLoS ONE. 2013;8:e79943.

  31. 31.

    Banek K, Lalani M, Staedke SG, Chandramohan D. Adherence to artemisinin-based combination therapy for the treatment of malaria: a systematic review of the evidence. Malar J. 2014;13:7.

Download references

Authors’ contributions

Conceived and designed the study: RAA, TD, LF, CT, FA, JM, EE, TPE. Performed the experiments: RAA, BD, DD, DK, SF. Analyzed the data: RAA. Wrote the first draft of the manuscript: RAA. Contributed to writing of the manuscript: BD DD, TD, LF, CT, FA, JM, DK, SF, EE, TPE. Agree with manuscript results and conclusions: RAA, BD, DD, TD, LF, CT, FA, JM, DK, SF, EE, TPE. All authors read and approved the final manuscript.

Acknowledgements

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

The data generated and analyzed during this study are available from the corresponding author on reasonable request.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Approval for this study was obtained from the Ethics Committee of the Malian National Institute for Public Health Research (L’Institut National de Recherche en Santé Publique, INRSP) and from the Tulane University Biomedical Institutional Review Board.

Funding

This work was supported by the President’s Malaria Initiative (PMI) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AIDOAA-L-14-00004, and by USAID through the DHS Program (#AIDOAA-C-13-00095). MEASURE Evaluation is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, in partnership with ICF; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. The findings and conclusions presented in this manuscript are those of the authors and do not necessarily reflect the official position of the PMI, USAID, or the United States Government.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Correspondence to Ruth A. Ashton.

Additional files

12936_2018_2636_MOESM1_ESM.pdf

Additional file 1. Visual aid of drugs that may be prescribed for malaria in Mali.

12936_2018_2636_MOESM2_ESM.docx

Additional file 2. Participant enrolment over time bar chart.

12936_2018_2636_MOESM3_ESM.docx

Additional file 3. Sensitivity, specificity and accuracy of caregivers’ recall of diagnosis procedures and treatment received during consultation, by facility type.

12936_2018_2636_MOESM4_ESM.docx

Additional file 4. Random effects multivariate logistic regression models of sensitivity, specificity and accuracy of caregiver recall of treatment with ACT, corrected by visual aids, prescriptions and packaging.

12936_2018_2636_MOESM5_ESM.docx

Additional file 5. Summaries of sensitivity, specificity and accuracy of caregiver recall of treatment with ACT (assessed by questionnaire response only) by various facility, caregiver and child characteristics.

12936_2018_2636_MOESM6_ESM.docx

Additional file 6. Summaries of sensitivity, specificity and accuracy of caregiver recall of treatment with ACT, corrected by visual aid, prescription documents or retained packaging by various facility, caregiver and child characteristics.

12936_2018_2636_MOESM7_ESM.docx

Additional file 7. ROC curves for recall of blood test, recall of positive blood test, recall of malaria diagnosis by clinician, and recall of any anti-malarial being prescribed.

12936_2018_2636_MOESM8_ESM.docx

Additional file 8. ROC for corrected ACT recall by each adjustment question/method.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Population survey
  • Malaria
  • Artemisinin-based combination therapy
  • Recall validation

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.