Completeness of malaria indicator data reporting via the District Health Information Software 2 in Kenya, 2011–2015

Background Health facility-based data reported through routine health information systems form the primary data source for programmatic monitoring and evaluation in most developing countries. The adoption of District Health Information Software (DHIS2) has contributed to improved availability of routine health facility-based data in many low-income countries. An assessment of malaria indicators data reported by health facilities in Kenya during the first 5 years of implementation of DHIS2, from January 2011 to December 2015, was conducted. Methods Data on 19 malaria indicators reported monthly by health facilities were extracted from the online Kenya DHIS2 database. Completeness of reporting was analysed for each of the 19 malaria indicators and expressed as the percentage of data values actually reported over the expected number; all health facilities were expected to report data for each indicator for all 12 months in a year. Results Malaria indicators data were analysed for 6235 public and 3143 private health facilities. Between 2011 and 2015, completeness of reporting in the public sector increased significantly for confirmed malaria cases across all age categories (26.5–41.9%, p < 0.0001, in children aged <5 years; 30.6–51.4%, p < 0.0001, in persons aged ≥5 years). Completeness of reporting of new antenatal care (ANC) clients increased from 53.7 to 70.5%, p < 0.0001). Completeness of reporting of intermittent preventive treatment in pregnancy (IPTp) decreased from 64.8 to 53.7%, p < 0.0001 for dose 1 and from 64.6 to 53.4%, p < 0.0001 for dose 2. Data on malaria tests performed and test results were not available in DHIS2 from 2011 to 2014. In 2015, sparse data on microscopy (11.5% for children aged <5 years; 11.8% for persons aged ≥5 years) and malaria rapid diagnostic tests (RDTs) (8.1% for all ages) were reported. In the private sector, completeness of reporting increased significantly for confirmed malaria cases across all age categories (16.7–23.1%, p < 0.0001, in children aged <5 years; 19.4–28.6%, p < 0.0001, in persons aged ≥5 years). Completeness of reporting also improved for new ANC clients (16.2–23.6%, p < 0.0001), and for IPTp doses 1 and 2 (16.6–20.2%, p < 0.0001 and 15.5–20.5%, p < 0.0001, respectively). In 2015, less than 3% of data values for malaria tests performed were reported in DHIS2 from the private sector. Conclusions There have been sustained improvements in the completeness of data reported for most key malaria indicators since the adoption of DHIS2 in Kenya in 2011. However, major data gaps were identified for the malaria-test indicator and overall low reporting across all indicators from private health facilities. A package of proven DHIS2 implementation interventions and performance-based incentives should be considered to improve private-sector data reporting. Electronic supplementary material The online version of this article (doi:10.1186/s12936-017-1973-y) contains supplementary material, which is available to authorized users.


Background
Reliable health data are essential for health planners and programme implementers to make informed decisions about resource allocations to prevent morbidity and mortality [1]. Historically, population-based health surveys have been conducted periodically to collect retrospective data to inform planning and policy needs [2]. However, nationally representative health surveys are expensive, infrequent and might not reflect the current situation, particularly in a dynamic health environment. By contrast, health facility-based data are collected on a routine basis and have the potential to present more realtime data for decision making, allowing disease control programmes to direct interventions in a more timely manner [2]. Health facility-based data reported through routine health information systems form the primary data source for national health planning, health surveillance and programmatic monitoring and evaluation in most developing countries [3,4]. Escalation of malaria control activities in Africa has led to increased demand for more robust data to track changes in transmission patterns and to develop interventions targeted to local heterogeneous conditions [5]. Routine data from health systems is critical for the design and implementation of malaria control programmes so that resources can be targeted to the populations and areas that are most in need [6]. However, the quality of routine data has often been poor with a general lack of timeliness, completeness and accuracy [7][8][9][10][11]. The lack of quality data has resulted in challenges in using routine data to monitor and evaluate health interventions [12][13][14]. The World Health Organization (WHO) relies on modelling to estimate malaria morbidity and mortality trends in most countries in the African region due to incomplete routine data [15,16]. However, the situation is now changing in many countries due to increased recognition of the importance of robust routine data to accurately track health progress and rapid developments in information technology [5,17].
The adoption of innovative, free and open source webbased reporting software such as the District Health Information Software 2 (DHIS2) has contributed to improved availability of routine health facility-based data in many developing countries [4,[18][19][20][21]. Following successful customization, testing and piloting, Kenya adopted DHIS2 as the routine health data reporting platform in 2011 [22]. The current online DHIS2 database is structured by administrative units with individual health facilities assigned to 299 sub-counties and 47 counties. Health facilities report monthly data on diseases, commodities and service delivery through DHIS2. Data are collected through a paper-based system of registers, tally sheets, and monthly data reporting forms at each health facility. Due to poor internet connectivity and lack of infrastructure in most rural health facilities, the collated monthly data are sent to the sub-county level where they are entered into the web-based DHIS2 [21]. The software captures and remotely stores facility-level data by month and can generate quarterly or yearly reports by different administrative levels. The data entered and stored into the system are available to registered users and other stakeholders through a password protected log-in system. Reporting rates for outpatient data have increased substantially since the introduction of DHIS2, and the system has high acceptance and utilization by stakeholders [21][22][23].
In Kenya, malaria accounts for approximately 18% of outpatient visits at health facilities [24]. Clinical and confirmed malaria cases encountered in outpatient clinics are reported via DHIS2. The DHIS2 data repository contains substantial numbers of malaria cases reported across all health facilities in Kenya. However, routine surveillance reports published by the National Malaria Control Programme (NMCP) have consistently shown anomalies in data reported via DHIS2 [25][26][27][28][29][30][31]. A nationally-representative, health facility-based survey that tested outpatients with suspected malaria in October and November 2014 found no confirmed malaria cases at 47% of the surveyed facilities [32]. However, confirmed malaria cases were reported in routine DHIS2 at the same facilities during the same time period [32]. In order to fully utilize routine malaria surveillance data for policy and programming decisions, the NMCP and stakeholders must have confidence in the data. The quality of data reported via DHIS2 during the first 5 years of implementation, from January 2011 to December 2015, was evaluated through an analysis of completeness of 19 malaria indicators reported monthly by health facilities.

Methods
Nineteen malaria data indicators reported monthly by health facilities were extracted from the DHIS2 online database [33] in February 2016. These included total number of clinical and confirmed malaria cases, artemether-lumefantrine (AL) treatments, intermittent preventive treatment in pregnancy (IPTp) with sulfadoxine-pyrimethamine doses, long-lasting insecticidal bed nets (LLINs) distributed via antenatal care (ANC) and children health clinics and malaria tests performed (Table 1). Monthly data were extracted separately for each sub-county, disaggregated by health facility and arranged chronologically. The assembled sub-county data sets were merged into one database for the country.
Completeness of indicator data reporting was expressed as the percentage of data values actually reported over the expected number given that all health facilities were expected to report each indicator for all 12 months in a year. Completeness of reporting was calculated by summing up the number of months with a data value reported for a given indicator across all health facilities in a year and dividing by the expected number of values (i.e., 12 months × total number of health facilities). All blank data fields were considered to be unreported data. Analysis was performed separately for public and private facilities and by epidemiological zones representing malaria transmission risk (i.e., the endemic zones of stable malaria around Lake Victoria and coast regions; the seasonal-transmission zone in the arid and semi-arid areas of northern and south-eastern regions; the highland epidemic-prone zone of the western region; and low-risk zone in the central highlands including Nairobi) [23].
Analysis of completeness of reporting of IPTp doses given to pregnant women was restricted to 1490 public and 709 private health facilities located in 14 counties in the lake-and coast-endemic zones. The analysis was restricted because the intervention is targeted only to HIV-negative women living in areas of moderate-to-high malaria transmission [23,34]. Similarly, analysis of completeness of reporting of LLINs distributed was restricted to 4793 public and 2336 private health facilities located in 36 counties targeted for routine distribution of nets through ANC and child health clinics. Analysis of completeness of malaria testing indicators was only reported for 2015 because data for previous years were not available. The change in indicator data reporting completeness over time is presented as a percentage-point increase or decrease in data values reported in 2015 compared to 2011 except for artemether-lumefantrine (AL) treatment indicators for which 2012 was used as the baseline year due to limited data reported in 2011. The percentage of health facilities in the public sector that reported data for each indicator for all 12 months in a year was also determined. All data were cleaned and analysed in Excel 2013 (Microsoft Corporation, Seattle, WA) and Stata version 12 (StataCorp, College Station, TX).

Description of health facilities analysed
Data were extracted from all 10,090 health facilities registered in the DHIS2 online database at the time of the study. During data cleaning, a total of 712 health facilities were excluded from analysis; 544 were health facilities that offered specialized services, 93 were duplicated facilities due to multiple spellings for the same health facility, 39 were unclassified facilities (i.e., facility type and managing authority not specified), and 36 were reporting units representing aggregated data from several facilities already included. Data were therefore analysed for 9378 (92.9%) of the health facilities registered in DHIS2 at the time of the study. Of these facilities, 6235 (66.5%) were public and 3143 (33.5%) were private ( Table 2).

Completeness of data reported in public health facilities
Within the public sector, the percentage of data values reported for confirmed malaria cases increased in children aged <5 years (26.5-41.9%, p < 0.0001) and in persons aged ≥5 years (30.6-51.4%, p < 0.0001) ( Table 3).   Table 3 (Additional file 1). Analysis by malaria-risk zones showed significant increases in the percentage of data values reported for confirmed malaria cases in all except the low-risk zone (Fig. 1). The increase was highest in the lake-endemic zone (38-84% in children aged <5 years, p < 0.0001 and 41-86% in persons aged ≥5 years, p < 0.0001). The percentage of data values reported for clinical malaria cases declined significantly across all zones. The decline was  Reporting of LLINs distributed through ANC and child health clinics increased across all zones. The largest increase was in the lake-endemic zone with 21 and 36% increase for LLINs distributed via ANC and child health clinics, respectively (p < 0.0001 for both).

Completeness of data reported in private health facilities
Overall, analysis of data reporting completeness from private health facilities showed trends that were similar to those seen in the public sector. However, the completeness of reporting in 2015 compared to 2011 was much lower for all indicators in private health facilities compared to public health facilities (  (Fig. 2).

Discussion
Routine malaria data for 19 indicators reported via the Kenya DHIS2 were assessed for completeness from 2011 to 2015. The assessment showed improvements in both national and sub-national reporting completeness for malaria indicator data over the first 5 years of DHIS2 implementation in Kenya. Completeness of data reporting in the public sector was much higher than in the private sector. Improvements in reporting completeness in the public sector may be attributed to substantial infrastructure and human resource investments for DHIS2 implementation including longitudinal training and support [22]. In addition, the public sector has higher demands for data to meet the reporting requirements of external donors and forecast commodity needs. For example, DHIS2 data fulfils the Global Fund to Fight AIDS, Tuberculosis and Malaria grant reporting requirements and informs forecasting for commodity procurements including RDTs, AL and LLINs [35]. The higher demand for and use of public sector health data has also led to additional disease-specific investments to improve reporting. Private-sector facilities, in contrast, do not rely on donor-funded commodities, and the demand for data from external users is low. Kenya, however, has a large and robust private health sector comprising half of all health facilities [36]. In 2015, 25% of children who had a fever in the 2 weeks preceding a large household malaria survey sought treatment from private health facilities [37]. The private sector would be a rich source of malaria data and increase the generalizability of national and subnational trends if reported. The Kenya Malaria Strategy recognizes the contribution of the private sector in providing care for malaria patients [23]. The NMCP provides support to the private sector primarily through the Affordable Medicines Facility-malaria to increase the affordability and availability of quality-assured AL via subsidies but also through malaria case management trainings [23]. However, data reporting from private health facilities remains poor because there is little incentive to report and no enforcement of the data reporting policy [38]. Providing a DHIS2 implementation package to private sector health facilities that includes training, standardized reporting tools and software support might have a similar positive impact as realized in the public sector. Performance-based incentives might also be considered; private health facilities are likely to be motivated by facility-based incentive programmes that give them an opportunity to earn additional resources [39].
A major limitation of the malaria indicator data was the inability to distinguish true missing values (i.e., no data reported) from 'zero' values (i.e., no events captured) because both scenarios appeared as blank data entries in DHIS2. Health facilities in Kenya do not usually report 'zero' values when no events are captured. All blank data entries were, therefore, classified as missing (i.e., no data reported). As a result, the completeness of reporting observed likely under-represented the true situation. For example, in low-risk and seasonal-transmission zones, the majority of health facilities did not report confirmed malaria cases. Based on the very low prevalence of malaria in these zones, it is probable that the majority of health facilities diagnosed zero confirmed malaria cases for at least some months of the year, which is consistent with findings from a recent health-facility survey [32]. Currently in DHIS2, facilities with no confirmed malaria (See figure on previous page.) Fig. 2 Percentage of malaria data values submitted by private health facilities to District Health Information Software 2 in Kenya by transmission zones, 2011-2015. AL artemether-lumefantrine, ANC antenatal care, CHC child health clinic, IPTp intermittent preventive treatment in pregnancy with sulfadoxine-pyrimethamine (IPTp intervention is only targeted in the lake-and coast-endemic zones), LLINs long-lasting insecticidal bed nets cases that do not report a 'zero' value and facilities that diagnose malaria cases but do not report both show blank fields. DHIS2 should require data fields for key malaria indicators to be filled in using conditional rules requiring input of an integer value including 'zero' and unfilled fields defaulting to 'NDR' (i.e., no data reported) or another coded value prior to data submission.
The relationship between program implementation and reporting is also critical to the interpretation of surveillance data and related to zero-reporting. Without zero-reporting, it is impossible to determine from malaria-indicator data alone if a program service or commodity is not delivered, not reported or both. For example, despite over 90% of public sector health facilities having diagnostic capacity and approximately 40% reporting confirmed malaria cases in 2013, none reported diagnostic testing data via DHIS2 in 2013 [40]. This is an example of service delivery but no reporting via DHIS2. Although completeness of reporting does not necessarily reflect programme implementation, facilities have incentive to report when the data drives actions such as malaria commodity procurements and deliveries.
Analysis by malaria-risk zones showed that the lakeendemic zone had the greatest reporting completeness improvements for confirmed malaria cases, AL treatments, and LLINs distributed via ANC and child health clinics. Because the majority of malaria cases are in the lake-endemic zone, the NMCP and key partners have focused malaria prevention and control interventions in the eight counties located in this zone since 2013 [41]. Improvements in malaria indicator data reporting completeness are likely a result of substantial malaria and other disease programming investments, including strengthening supply chain management for commodities and surveillance, in the zone [41].
Overall, there were significant increases in completeness of reporting of confirmed malaria cases, new ANC clients, and LLIN distribution via ANC and child health clinics in both the public and private sectors while data values reported for clinical malaria cases declined. The increase in reporting completeness of confirmed malaria cases was probably related to the national expansion of malaria RDTs. In late 2012, the NMCP implemented policy to distribute malaria RDTs to all public sector health facilities in Kenya. CareStart Malaria histidinerich protein 2 (HRP2) Plasmodium falciparum RDTs were supplied to all health facilities through the Kenya Medical Supplies Agency [25]. Consequently, the percentage of public sector health facilities with functional capacity to diagnose malaria increased from 59% in 2011 to 97% by the end of 2015 [40]. However, despite significant improvements, the assessment found that the highland epidemic-prone zone continued to report a relatively high percentage of data values for clinical malaria cases. The inability of health workers to establish differential diagnosis for persons with malaria-test negative results might explain continued reporting of clinical malaria cases despite availability of malaria diagnostics [35]. Sustained availability of malaria diagnostics in all health facilities, parasitological diagnosis of malaria for all persons before treatment and capacity to appropriately manage malaria test-negative patients are all critical components required to achieve the national target of 100% parasitological diagnosis of all persons with suspected malaria presenting to a health provider in Kenya [23].
Although 90% of pregnant women make at least one ANC visit, only 70% of public sector facilities reported new ANC visit [37,42]. The indicator, attendance at first ANC visit, is more frequently reported compared to other malaria indicators due, in part, to its widespread use as a tracer indicator in the WHO data quality assessment framework [4,43]. Attendance at first ANC visit, a national indicator, and IPTp 1 and IPTp 2, sub-national indicators, both showed increasing trends for reporting completeness from 2011 to 2014. However, in 2015, the percentage of public-sector facilities reporting IPTp 1 and 2 declined by 19 percentage points compared to 2014 despite the continued positive trend in attendance at first ANC visit reporting. It seems unlikely that facilities would continue reporting attendance at first ANC visit but stop reporting IPTp 1 and 2 doses. Kenya experienced country-wide stock-outs of sulfadoxine-pyrimethamine in 2015, which were reported in national malaria surveillance bulletins and biennial quality-of-care survey [41]. Thus, the decline in reporting completeness of IPTp 1 and 2 doses seems more likely a reflection of interrupted service delivery due to lack of sulfadoxinepyrimethamine at the facility-level in 2015; data interpretation would be more direct if facilities had reported 'zeros' for IPTp doses.
The overall gains in reporting completeness of malaria data indicators in Kenya could be attributed, in part, to technical expertise from the Ministry of Health and strong partnerships with development, technical and implementing partners, who have contributed financial, technical and operational support [23]. Similar expertise and partnerships are being developed at the county level, which have advanced rapidly since 2013. The strong national malaria strategic plan framework has led to robust monitoring and evaluation of malaria indicators such as quarterly national malaria surveillance bulletins since 2012, biennial quality-of-care surveys since 2010, and regular national household surveys [23]. For example, quarterly national malaria surveillance bulletins highlight counties with reporting rates below 60% [25][26][27][28][29][30][31][44][45][46]. Regular data reviews and technical working groups at both the national and county levels monitor the progress of key malaria indicators and support is provided to counties to increase reporting rates to >80% to ensure that the data reported nationally is both generalizable and representative.
There were important gaps in the reporting system as a result of programmatic limitations, which were reflected in the analysis. The lack of data for malaria tests performed and test-positive results was a major gap identified in DHIS2. Prior to the national introduction of malaria RDTs, malaria testing was predominantly performed by microscopy and reported on a quarterly basis through a parallel laboratory management information system. Following the national introduction of malaria RDTs, new data forms to capture persons with suspected malaria tested by RDTs and microscopy and results by type of test were developed and integrated into DHIS2 in 2014. However, the transition to reporting malaria-test data through DHIS2 has been slow as shown by the paucity of data reported in 2015.
The overarching goal of the Kenya national malaria strategy is elimination [23,47]. Zero-reporting is a key surveillance strategy successfully used to increase sensitivity in the polio eradication programme, which was later extended to other disease elimination and eradication programmes [48]. Kenya should consider adopting a zero-reporting policy and instituting DHIS2 systembased solutions along with revised training and support modules to improve surveillance. High reporting completeness for malaria indicators, including zero-reporting, could provide sufficient data to determine when subnational regions and the country meet the required WHO threshold (i.e., slide positivity rate of <5% during the peak malaria season) to move to the malaria pre-elimination phase [49]. Finally, due to the very large number of observations included in the analysis (i.e., all health facilities), very small changes over time, which were statistically significant, may not necessarily indicate sustained systematic improvement. Broader contextual changes were therefore considered in the interpretation of the results.

Conclusions
There have been sustained improvements in the completeness of data reporting for most key malaria indicators since the adoption of DHIS2 in Kenya in 2011. However, major gaps were identified including the lack of data reported for malaria tests performed, no zeroreporting and overall low reporting from private health facilities. Ongoing efforts to integrate malaria-test data into DHIS2 should improve indicator availability and help determine Kenya's progress towards meeting national and international targets. A package of proven DHIS2 implementation interventions and performancebased incentives should be considered to improve private-sector data reporting.