Study site
The study was undertaken in the former Nyando District, Nyanza Province, in western Kenya between February and May 2010. This area encompass the Nyando, Upper and Lower Nyakach, Miwani and Muhoroni Divisions and has been fully incorporated into Kisumu County. The former Nyando District had a population of 355,800 (1999 census), with the majority of the population from the Luo ethnic group. Malaria is perennial and holo-endemic with a parasite prevalence of 18 % among pregnant women presenting for their first ANC visit [10]. HIV prevalence among women in Nyanza Province is the highest across all provinces in the country (14 %, DHS 2008–9 [7]). The former Nyando District has a total of 40 health facilities of which 24 are owned by government, five by missions, seven privately owned and four are community run.
Study design and data collection
A cross-sectional study was conducted in nine out of ten selected health facilities using structured observation, exit interviews of an individual ANC visit with ANC clients and a health facility audit. This study was part of a larger programme under the Malaria in Pregnancy Consortium using multi-disciplinary approaches to assess bottlenecks in the scale-up of malaria in pregnancy interventions. As part of this programme, healthcare provider’s perspective was examined through in-depth interviews following completion of observations in 2011, which will be presented separately, and the community perspective through both focus group discussions [11] and a household survey [12].
Sampling procedure
A list of all health facilities in the selected clusters for the associated household survey was compiled consisting of the health facilities that were closest to each cluster [12]. A dual-frame sampling scheme was used to purposively select the district hospital and a representative sample of nine health facilities using probability proportional to antenatal attendance. Note that community units (level 1) were not included in the sampling frame as they did not provide antenatal care or deliver IPTp nor ITNs at the time of the study. All women attending ANC at the selected health facilities were invited to take part in the study and observations and exit interviews were carried out with all consented participants, except where there were 15 or more ANC attendees on the day at the time of registration, in which case exit interviews were conducted only on every second woman who was observed which followed the registration order.
Data collection procedure
Data on characteristics of the health facility, demographic characteristics of the staff involved in delivery of ANC services, supplies and equipment necessary for ANC available on the day of the survey as well as stock records for the previous 1 year were collected from the health facility staff person in charge (or their representative on the day of the survey) using a structured questionnaire. A trained fieldworker was stationed at different points in the health facility to follow all the ANC processes: at registration desk, in the ANC consultation room, and at the exit to follow participants to the laboratory or pharmacy and to conduct exit interviews. All ANC attendees on each day of the survey were invited to take part and, following provision of informed consent, participant characteristics were collected and all ANC processes observed and recorded using a structured checklist. The exit interviews were conducted immediately following the observations. The data collected through exit interviews included women’s knowledge of the treatments they received during ANC and on the dosing regimen of anti-malarials as well as measures of cost of prevention of malaria in pregnancy to their household.
Data analysis
Data were collected on paper forms and were double entered into EpiData (Version 3.1, EpiData Association, Odense, Denmark) and validated before being transferred to Stata (Version 13, 2013; College Station, TX) for analysis. Each intermediate step required for the effective delivery of IPTp and ITNs through ANC as per WHO guidelines [6] and Kenyan national policy [13] (Fig. 1) defined the framework for analysis (Fig. 2). The data collected were used to quantify: (1) the proportion of ANC attendees who successfully completed a designated intermediate step in the delivery of each intervention from those who completed the previous step, (2) the cumulative systems effectiveness corresponding to the proportion of eligible ANC attendees who successfully completed all intermediate steps to the designated point in the overall delivery process, and (3) design effect (DE) and intra-cluster correlation coefficients (ICC) between health facilities for each intermediate step in the delivery of IPTp and ITNs. Predictors of ineffective intermediate steps will be described in a separate paper together with results from the qualitative data.
For the IPTp effectiveness analysis, IPTp eligibility was defined as per national policy at the time of the survey to include: reporting having felt the baby move (i.e. quickening) or estimated gestational age of 16 weeks or more (to ensure the mother was well past the first trimester, when SP is contraindicated); and not being HIV positive (as a proxy for no concurrent cotrimoxazole use). As information on dates of previous ANC visits was not collected, it was not possible to assess if the last SP dose was within 1 month, but this is unlikely to be a frequent occurrence.
For ITNs, the effectiveness analyses were restricted to participants presenting for their first ANC visit for the current pregnancy. Sample weights for observations were estimated based upon the sampling probability of each selected health facility [14]. Analyses were stratified for level 4 facilities (primary hospitals) and level 2 or 3 (health centres and dispensaries) as a previous study in Mali showed that the systems effectiveness for the delivery of IPTp differed at different levels of the health system [15]. As previously defined, intermediate steps were classified as ‘ineffective’ if fewer than 80 % of participants successfully completed that step [16]. This is based on the fact that the Roll Back Malaria Partnership target for 2015 was 100 % coverage for MiP interventions, which would mean that 100 % of women should pass each step of the delivery process for all to get the interventions. A realistic goal is that at least 80 % of women receive the interventions (the target for 2010) for which each step in the delivery should be at least 80 %.
Intermediate steps effectiveness
The effectiveness of each intermediate step in the delivery of MiP interventions was calculated by estimating the proportion of women who successfully completed each step [15, 16]. The intermediate steps for IPTp according to policy for eligible women (not on cotrimoxazole prophylaxis for the prevention of opportunistic infections associated with HIV infection, and not administered another anti-malarial for treatment of malaria) were: (1): attend ANC consultation after quickening (estimated at around 16 weeks gestation); (2): attend ANC when and where SP is in stock; (3): be given any SP; (4): be given three tablets of SP; (5): be given SP by DOT or leaving the facility with three tablets of SP and able to report correctly how it should be taken. For ITNs the analysis was restricted to first ANC visits and there were four intermediate steps: (1) attend ANC for the first visit; (2) attend ANC when and where ITNs are in stock; (3) be offered an ITN and (4) take the ITN. Where ITN stock information was unavailable and no clients received ITNs from that facility on the day of the survey, these were classified as stock-outs for the effectiveness analysis.
The data were collected via structured observations except for ‘having three tablets of SP at exit with knowledge of correct regimen’, which was obtained through exit interviews with ANC clients.
Cumulative systems effectiveness
The overall cumulative systems effectiveness was estimated as the proportion of eligible pregnant women who successfully received each MiP intervention according to policy on the specific ANC visit. For IPTp two cumulative systems effectiveness estimates were calculated (1) a ‘per policy’ estimate of being given three tablets of SP by directly observed therapy (DOT) and (2) being given three tablets of SP by DOT or without DOT, but women without DOT had three tablets of SP upon leaving the health facility and knew the correct dosing to take home (and, therefore, had the potential to receive a dose of IPTp-SP).
The design effect and intra-cluster correlation coefficients
Design effect (DE) and intra-cluster correlation coefficients (ICC) were used to measure the level of clustering and correlation within health facilities. DE was calculated as the difference between the variance based on the clusters used compared with a modelled variance if a simple random sample was used. DEs were calculated for each intermediate step (described above and in Fig. 2) aggregated for all health facilities combined and for health facilities clustered according to level 4 or level 2 and 3 combined. ICC and DE are closely related and can be derived from one to the other using the following formula: \({\text{DE}} = {\text{ICC }} \times \left( {{\text{m}} - 1} \right) + 1\) where m is average number per cluster. This allows different ways of looking at clustering [17].