Before discussing the results in more detail a number of limitations in the provider and mystery-shopper surveys should be highlighted. Shops that had undergone Tibamal training were identified by asking the respondent if any of the staff had attended the training, but it is possible that some responses were inaccurate due to recall bias or lack of awareness of training of fellow staff members. In addition, given the prior consent process, it is also possible that retailers were suspicious and therefore altered their practice and the advice they gave to the mystery shopper. However, if this were the case then the data would display ‘best practice’ of providers, while still showing considerable room for improvement, especially in areas such as the provision of appropriate counselling. In the scenario, the mystery shoppers waited for the retailer to recommend treatment and paid whatever price they were asked to. Other data (Kangwana and colleagues, unpublished observations from Tibamal focus group discussions and provider survey reports) suggest that in practice the consumer often asks for a specific treatment instead of the provider recommending it, so real-life interactions may be somewhat different. It is also possible that there was some contamination of the control arm outlets, which could have heard some of the communication activities. However, results indicated that such exposure was low, with only 1% of control arm respondents saying that they had attended the Tibamal training, 14% having heard of Tibamal, and no outlets stocking Tibamal.
The intervention was able to significantly improve the percentage of outlets stocking AL, and more than 90% of the AL available at follow-up in the intervention arm was Tibamal. This indicates a willingness of shopkeepers to take part in the intervention and make the treatment available in their outlets. The intervention had an effect on most provider knowledge indicators. Significantly more providers in the intervention arm compared to the control arm knew AL was the first-line treatment for uncomplicated malaria and knew fever as a symptom of uncomplicated malaria. Providers were also more knowledgeable on correct dispensing practices for AL, than those in the control arm. However, although the difference observed between the arms was significant, there remains a need for renewed effort to improve these components of knowledge and prescription. For example, for the scenario of ‘what to do if the child does not get better’, only 66% of providers could give the correct response. In addition, findings from the mystery-shopper survey revealed that knowledge was not always transferred into practice, since no significant improvements were observed in any of the four appropriate counselling indicators.
The mystery-shopper survey revealed that the intervention not only encouraged shopkeepers to stock AL but also significantly encouraged them to dispense AL to clinically diagnosed cases of uncomplicated malaria. This is in line with findings from the household surveys, which showed that a significantly greater percentage of febrile children in the intervention arm were treated with AL compared to the control arm at follow-up . The findings in this paper on changes in provider behaviour also serve to strengthen the claim that the changes observed in the household survey were very likely due to the Tibamal intervention. Encouragingly, the mystery shopper data showed providers adhering to the recommended retail price of Tibamal. Findings from the household survey also revealed that >95% of caregivers purchased Tibamal at its recommended retail price . Printing of the recommended price on Tibamal packaging as well as making consumers aware of the subsidy through the community activities may have contributed to providers not inflating Tibamal prices. Also, the way in which the pricing was structured meant that even with the subsidy, the retailer mark-up would be greater than for other more commonly prescribed anti-malarials, which may have facilitated both stocking of Tibamal by retailers and appropriate pricing.
The intervention did not significantly increase the percentage of providers that would refer complicated cases of malaria directly to a health facility. The data show that even without the intervention, around 80% of shopkeepers stated that they would automatically refer these cases to a health facility, but since immediate specialized care is required for complicated cases, one would hope to see a referral rate of close to 100%. The intervention was also not able to significantly increase the percentage of providers asking for at least one danger sign, an important way of identifying severe cases. It could be that providers instead tended to rely on the consumer to provide all the necessary information without being probed. Identifying and implementing ways to improve enquiries about danger signs is therefore important, in addition to providing caregivers with the knowledge of when to bypass the retail sector and go straight to a health facility. Although the intervention improved the share of AL among mystery-shopper purchases it had no significant impact on decreasing the availability of non-ACT in outlets. There was a decline in non-ACT availability at follow-up, but this was observed in both arms and was thought to be as a result of a government directive to halt the production and supply of less effective monotherapy at the time of the survey. Availability of artemisinin monotherapy was not a concern, with less than 5% of outlets stocking this treatment, probably due to low demand as a result of its high cost compared to other anti-malarial monotherapy .
Overall, it seemed that the greater exposure shopkeepers had to all components in the intervention, the better they tended to perform. Outlets that had received training seemed to perform better than all outlets in the intervention arm, and outlets that had received both training and job aids seemed to perform better than outlets just exposed to training. This shows the importance of ensuring that implementation of the intervention is as ‘ideal’ as possible ; had higher coverage of training and job aids been achieved, even more substantial improvements in provider behaviour and treatment coverage would likely have been achieved.
These findings are of particular importance given the current roll out of similar ACT subsidies under the Affordable Medicines Facility - malaria (AMFm) on a national scale in Kenya and seven other countries, also accompanied by training and communications activities . In Kenya AMFm training of retail providers was limited to registered pharmacists who were the only retailers officially allowed to stock the AMFm subsidized product, although in practice it was widely available in unregistered pharmacies . By demonstrating variation in performance in relation to intervention intensity, and highlighting areas of particularly weak provision, these results can be used to identify potential strategies to enhance provision of subsidized ACT under AMFm and other similar subsidy mechanisms.
The authors of this paper are not aware of any other studies that explore the effect of an intervention including an ACT subsidy on the performance of private sector retailers in the treatment of presumptive malaria. Several studies in sub-Saharan Africa have however evaluated the effect of other interventions to improve the quality of care received from the retail sector in the treatment of presumptive malaria, mostly with anti-malarial monotherapy. The majority of the interventions included training of either providers or users, and combined this with other supporting activities such as provision of job aids, follow-up monitoring and provision of prepackaged anti-malarial treatment with pictorials to guide administration. The outcomes of the interventions varied between studies but the majority showed positive outcomes. Overall the interventions were able to improve provider knowledge on signs and symptoms of malaria [31, 32], and the proportion of providers giving correct treatment and dose [14, 33]. Some interventions increased provision of correct advice on administration [12, 31], and asking for danger signs . Interventions were also able to improve the availability of anti-malarials in outlets [15, 31]. Studies differed in their design, data analysis techniques used and the type of outcome measures used. Few studies carried out hypothesis testing on their outcome results so it is difficult to interpret the importance of any observed differences, and some studies had limitations, for example very small sample sizes or no appropriate comparison group. All these factors make it difficult to make a direct comparison of outcomes in these studies with the data presented here.
Exploration of the context in which the survey took place, together with interviews with participants in the intervention through focus group discussions (Kedenge and colleagues, unpublished observations from the Tibamal focus group discussion report), have provided some insight into the findings and how the results could have been further improved. At follow-up, only 43% of outlets in the intervention arm were identified as trained. Several possible reasons were given including that some businesses trained at baseline had closed due to lack of capital; some businesses relocated to other areas or had changed their type of business, for example, from a general store to a bicycle spare part shop. Also, recently opened but untrained shops had not had the opportunity to be trained. This highlights the need to hold regular courses to ensure that a well-trained cadre of shopkeepers is maintained. In addition some providers thought that the training should have been longer, refresher courses should have been included and the training could have better catered for illiterate shopkeepers. Furthermore in response to the low percentage of patients being counselled, some shopkeepers were said to work in a very busy environment, having to attend to more than one customer at a time, making it difficult to discuss these issues with caregivers in any detail. As this is unlikely to change, this highlights the merits of also providing this information directly to consumers through communication activities.
Care should be taken when extrapolating or generalising the findings of this pilot to other areas. Since Tibamal was being supplied directly to outlets, there is the possibility that monthly contact with PSI staff distributing AL may have had the effect of influencing providers to work at their best. Other factors affecting the generalizability of these findings have been discussed by Kangwana et al..
Finally, the World Health Organization now recommends that all suspected cases of malaria should be parasitologically diagnosed before treatment, where diagnostics are available . Therefore further research is required into understanding how diagnostics will change provider practices, and assessing strategies to ensure diagnostics are used appropriately and supplied at an affordable price.