Skip to main content

Methodological approaches for analysing data from therapeutic efficacy studies

Abstract

Several anti-malarial drugs have been evaluated in randomized clinical trials to treat acute uncomplicated Plasmodium falciparum malaria. The outcome of anti-malarial drug efficacy studies is classified into one of four possible outcomes defined by the World Health Organization: adequate clinical and parasitological response, late parasitological failure, late clinical failure, early treatment failure. These four ordered categories are ordinal data, which are reduced to either a binary outcome (i.e., treatment success and treatment failure) to calculate the proportions of treatment failure or to time-to-event outcome for Kaplan–Meier survival analysis. The arbitrary transition from 4-level ordered categories to 2-level type categories results in a loss of statistical power. In the opinion of the authors, this outcome can be considered as ordinal at a fixed endpoint or at longitudinal endpoints. Alternative statistical methods can be applied to 4-level ordinal categories of therapeutic response to optimize data exploitation. Furthermore, network meta-analysis is useful not only for direct comparison of drugs which were evaluated together in a randomized design, but also for indirect comparison of different artemisinin-based combinations across different clinical studies using a common drug comparator, with the aim to determine the ranking order of drug efficacy. Previous works conducted in Cameroonian children served as data source to illustrate the feasibility of these novel statistical approaches. Data analysis based on ordinal end-point may be helpful to gain further insight into anti-malarial drug efficacy.

Background

Clinical efficacy of artemisinin-based combination therapy (ACT) has been monitored in many countries using the standard World Health Organization (WHO) protocol [1,2,3,4]. The following categorical outcome is used in WHO-recommended protocol: adequate clinical and parasitological response (ACPR), late parasitological failure, late clinical failure, and early treatment failure [1, 5]. The definitions of these four possible outcomes in Plasmodium falciparum-infected patients suggest that there is an explicit order in terms of disease severity and that these outcomes can be considered as ordinal variables. However, in most studies, this four-level primary outcome is reduced to a binary outcome (i.e., ACPR versus failure) to calculate the proportion of cured patients in per-protocol population [1, 6,7,8]. Kaplan–Meier curve analyses time-to-event outcomes to compensate partially for the loss of information, but data may be censored for various reasons (exclusion, withdrawal, loss-to-follow-up, clinical aggravation, or reinfection). These two methods of data interpretation may restrict the full exploitation of clinical results.

Several alternative approaches have been used to identify the most efficacious artemisinin-based combination that is useful to control and eliminate malaria [6]. This paper presents three approaches and areas of reflexion on how statistical analysis of existing database on anti-malarial drug efficacy can provide still unmined information. These approaches were presented in detail in previous works using data from therapeutic efficacy studies conducted in Cameroon [9,10,11]. They include (i) analysis of the ordinal outcome at a single time-point (day 14); (ii) analysis of the ordinal outcome over several time-points (day 14, day 21, and day 28); and (iii) network meta-analysis (NMA).

Ordinal outcome with a single fixed time-point

Proportional odds models are regression models that may be more suitable to accommodate the 4-level ordinal outcome since these approaches take into consideration the order of the categories and adjust the models subsequently on either an agglomerated data or individual data. They can be used in different analytic settings and were found in some fields to provide greater power than time-to-event and binary endpoints [12]. The models are based on a logistic link function and account for both fixed and random effects, as well as baseline covariates [13]. However, difficulties may arise in the estimation process when some categories are not observed in a study. To circumvent this problem, the model has several extensions to account for categories with very few or no observations, especially when the efficacy of a test drug is very high (i.e. > 90%). The complementary log–log link function provides suitable interpretation of results, which is similar to that of Cox-proportional hazard model.

Despite these limitations, proportional odds models have been recommended by some authors to combine the results of different types of clinical trials, including longitudinal and one-off studies, in which the outcome is categorically ordered [14, 15]. Ordinal logistic regression assumes that the coefficients that describe the relationship between each pair of outcomes are the same. This relationship is tested by a graphical method and likelihood comparison of the model with and without covariates [15, 16].

This approach was applied to clinical studies conducted in Cameroon at the time when the WHO was recommending a 14-day follow-up for areas of intense transmission [5, 9, 10]. Individual patient data were available. The interpretation of the proportional odds model accounts for the order across categories in the analytic setting. Categories can be ordered from the worst to the best or vice-versa. The odds ratio of comparison of drug A versus drug B is interpreted as either a progression towards success or progression towards failure [9]. Although the results of this retrospective study can no longer be of help for drug policy change since monotherapies are not used for the treatment of acute uncomplicated malaria, the application of the method illustrates the potential of alternative statistical methods. This method is illustrated in Additional File 1 using a simulated data set.

Ordinal outcome with several time-points

The first approach was extended to data from P. falciparum-infected patients evaluated on days 14, 21, and 28 to account for time-effect in the ordinal regression model and correlation among individual responses to treatment [17]. In such cases, data analysis becomes more complex when multiple treatments are involved, leading to incomplete block design for treatment arms between different trials and the presence of variability. In addition, trials that are not directly connected to others are a potential source of wide variance and are removed from analysis [18, 19]. A statistical approach to handle an incomplete design is to use a proportional odds model where individual log-residual variance is modelled as a linear mixed model that accounts for time-covariate related to the outcome [20]. The effects of individual covariates at inclusion, such as parasitaemia, fever, sex, and weight, are important factors that may decrease heterogeneity across studies and should be accounted for.

The use of direct and indirect comparison has increased over recent years [21, 22]. Indirect comparison is performed using a common comparator drug between treatment regimens that are not linked because they were not compared in a randomized clinical study [10]. By contrast, direct comparison implies a direct link between two treatments with the number of studies comparing both treatments. Bonferroni correction is recommended and is used to estimate the common type 1 error for all comparisons, which results in large confidence intervals. Linear mixed models can be trickier in handling because of complex maximum likelihood that they generate and the difficulty in integrating the likelihood. Despite these difficulties in the formulation of prior distributions, Bayesian methods have been the most commonly used tests to compensate the look-elsewhere effect in such cases because of their flexibility.

To illustrate this approach, clinical studies conducted in Cameroon revealed that, compared to artesunate-amodiaquine (ASAQ), dihydroartemisinin-piperaquine (DHPP) was significantly more efficacious, i.e. there was a positive progression towards ACPR from day 14 to day 28 with DHPP, suggesting that more treatment failures occurred during day 14 and day 21 for ASAQ compared to DHPP [10]. Progression to success was similar between ASAQ and artemether-lumefantrine (AMLM), in agreement with other studies conducted in Africa [23,24,25,26,27]. The apparent superiority of DHPP was assessed in a larger patient population and is in agreement with pharmacokinetic profiles of amodiaquine and lumefantrine, which have shorter elimination half-lives than piperaquine [28].

Network meta-analysis

NMA can be used to extend the second approach to multiple randomized clinical trials with the aim of choosing the best treatment regimen [29, 30]. The modelling process is based on random effect models that account for different sources of variability, including drug formulation, mode of administration, supervised vs unsupervised drug administration, conflicting results reported by each trial, and study design (multicentric vs single centre). When multiplicity is present, some authors opt for a frequentist approach, while others argue for the use of a Bayesian approach [31]. Both of these approaches can be implemented using available statistical packages [32], which are also suitable for single trial proportional model.

Data from randomized studies conducted in Cameroon, and elsewhere in Africa, have been combined to illustrate the utility of NMA using a binary outcome [11, 33]. These analyses showed that DHPP was more effective than AMLM (odds-ratio [OR] = 1.92; 95% confidence interval [CI] 1.30–2.82; 19,163 patients) and that DHPP has the highest probability of being the best choice for treating uncomplicated P. falciparum malaria. A similar study was conducted in Asia, and the network built with 14 treatment regimens revealed that the OR network estimates from both African and Asian studies were comparable. In Asia, DHPP was 2.5 times (95% CI, 1.08–5.8) more efficacious on day 28 than AMLM [34].

One of the disadvantages of NMA is its high sensitivity to highly effective novel ACT in a small sample size because ranking may depend on whether the drug has been widely assessed or not [11]. This problem can be circumvented by ranking treatment regimens from the most tested ACT to the less tested ACT [11]. The methodological approach of ranking treatment also depends on whether a frequentist approach based on P-score and an analog to the surface under the cumulative ranking (SUCRA) or a Bayesian approach with posterior probability is used [35,36,37].

Discussion

The ordinal criterion was applied to data from therapeutic efficacy studies for alternative statistical analysis and data interpretation [9,10,11]. Some authors have argued that ordinal regression models may be superior to analysis of binary outcome for designing clinical trial and evaluating treatment efficacy [12, 38]. In the present opinion paper, it is argued that ordinal data analysis may be helpful to evaluate fixed-time efficacy, or changes in therapeutic responses over time at the individual level where follow-up assessment allows detection of recrudescence/reinfection and evaluate the correlation between therapeutic response and molecular markers of resistance [39]. Modelling the WHO criteria as ordinal criteria results in a gain of information and precision on the estimate of the treatment effect. However, one of its limitations is the difficulty in determining the sample size of a trial given a known treatment effect [40].

Kaplan–Meier analysis is modelled based on the time to failure, reports the risk of recrudescence, and considers several events, such as reinfections, withdrawals, and loss-to-follow-up, as censored [8]. However, it does not consider categorical outcome. Since more robust statistical methods are available for handling categorical outcome [16], it is the opinion of the present authors that clinical studies on anti-malarial drug should account for such outcome since this innovative methodological approach provides information that has been missed heretofore and higher precision on the estimate of the treatment effect [12]. In the context of the emergence and spread of artemisinin resistance [41, 42], it may be expected that an increasing number of patients will respond with one of the failure categories, enabling the optimal use of this approach to compare the efficacy of ACT and identify the most effective drug.

For systematic reviews, NMA constitutes a powerful analytical approach to bring together multiple treatments which have not been compared directly in randomized controlled trials. Treatment ranking is solely possible using NMA, rendering this tool useful to identify the best treatment based on available evidence. NMA also provides a greater statistical precision through its incorporation of indirect evidence, which is not considered in pairwise meta-analysis.

Conclusions

The four-level ordinal outcome derived from the WHO protocol can be better exploited using several statistical tools to analyse agglomerated or individual patient data from single trials or multi-centric trials in which two or more treatments are evaluated. These analytical tools include options for multiple comparison with fixed and mixed effects and determination of the best treatment regimen.

Availability of data and materials

Not applicable.

Abbreviations

ACPR:

Adequate clinical and parasitological response

ACT:

Artemisinin-based combination therapies

AMLM:

Artemether-lumefantrine

ASAQ:

Artesunate-amodiaquine

95% CI:

95% Confidence interval

DHPP:

Dihydroartemisinin-piperaquine

NMA:

Network meta-analysis

OR:

Odds-ratio

PCR:

Polymerase chain reaction

SUCRA:

Surface under the cumulative ranking

WHO:

World Health Organization

References

  1. 1.

    WHO. Assessment and monitoring of anti-malarial drug efficacy for the treatment of uncomplicated falciparum malaria. Geneva: World Health Organization; 2003.

  2. 2.

    WHO. Global report on antimalarial drug efficacy and drug resistance: 2000–2010. Geneva: World Health Organization; 2010.

  3. 3.

    WHO. Guidelines for the treatment of malaria. 3rd Edn. Geneva: World Health Organization; 2015.

  4. 4.

    WHO. Report on antimalarial drug efficacy, resistance and response: 10 years of surveillance (2010–2019). Geneva: World Health Organization; 2020.

  5. 5.

    WHO. Assessment of therapeutic efficacy of antimalarial drugs for uncomplicated falciparum malaria in areas with intense transmission. WHO/MAL/96.1077, Geneva: World Health Organization; 1996.

  6. 6.

    Dahal P, Simpson JA, Dorsey G, Guérin PJ, Price RN, Stepniewska K. Statistical methods to derive efficacy estimates of anti-malarials for uncomplicated Plasmodium falciparum malaria: pitfalls and challenges. Malar J. 2017;16:430.

    Article  Google Scholar 

  7. 7.

    Stepniewska K, White NJ. Some considerations in the design and interpretation of antimalarial drug trials in uncomplicated falciparum malaria. Malar J. 2006;5:127.

    Article  Google Scholar 

  8. 8.

    WorldWide Antimalarial Resistance Network Methodology Study Group. Competing risk events in antimalarial drug trials in uncomplicated Plasmodium falciparum malaria: a WorldWide Antimalarial Resistance Network individual participant data meta-analysis. Malar J. 2019;18:225.

    Article  Google Scholar 

  9. 9.

    Whegang SY, Basco LK, Gwét H, Thalabard JC. Analysis of an ordinal outcome in a multicentric randomized controlled trial: application to a 3-arm anti-malarial drug trial in Cameroon. BMC Med Res Methodol. 2010;10:58.

    Article  Google Scholar 

  10. 10.

    Whegang Youdom S, Samson A, Basco LK, Thalabard JC. Multiple treatment comparisons in a series of anti-malarial trials with an ordinal primary outcome and repeated treatment evaluations. Malar J. 2012;11:147.

    Article  Google Scholar 

  11. 11.

    Whegang Youdom S, Tahar R, Basco LK. Comparison of anti-malarial drugs efficacy in the treatment of uncomplicated malaria in African children and adults using network meta-analysis. Malar J. 2017;16:311.

    Article  Google Scholar 

  12. 12.

    Peterson RL, Vock DM, Babiker A, Powers JH 3rd, Hunsberger S, Angus B, et al. Comparison of an ordinal endpoint to time-to-event, longitudinal, and binary endpoints for use in evaluating treatments for severe influenza requiring hospitalization. Contemp Clin Trials Commun. 2019;15:100401.

    Article  Google Scholar 

  13. 13.

    McCullagh P. Regression models for ordinal data. J R Statist Soc B. 1980;42:109–42.

    Google Scholar 

  14. 14.

    Agresti A. An introduction to categorical data analysis. 2nd ed. New Jersey: John Wiley & Sons, Inc.; 2007.

    Book  Google Scholar 

  15. 15.

    Whitehead A, Omar RZ, Higgins JP, Savaluny E, Turner RM, Thompson SG. Meta-analysis of ordinal outcomes using individual patient data. Stat Med. 2001;20:2243–60.

    CAS  Article  Google Scholar 

  16. 16.

    Harrell FE Jr. Ordinal logistic regression. In: Regression modeling strategies. Springer Series in Statistics. Springer, New York, NY; 2001. p. 331–43.

  17. 17.

    Djalle D, Njuimo SP, Manirakiza A, Laganier R, Le Faou A, Rogier C. Efficacy and safety of artemether + lumefantrine, artesunate + sulphamethoxypyrazine-pyrimethamine and artesunate + amodiaquine and sulphadoxine–pyrimethamine + amodiaquine in the treatment of uncomplicated falciparum malaria in Bangui, Central African Republic: a randomized trial. Malar J. 2014;13:9.

    Article  Google Scholar 

  18. 18.

    Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med. 2004;23:3105–24.

    CAS  Article  Google Scholar 

  19. 19.

    Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU technical support document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. London: National Institute for Health and Care Excellence (NICE); 2014. https://www.ncbi.nlm.nih.gov/books/NBK310366/

  20. 20.

    Foulley JL, Jaffrézic F. Modelling and estimating heterogeneous variances in threshold models for ordinal discrete data via Winbugs/Openbugs. Comput Methods Programs Biomed. 2010;97:19–27.

    Article  Google Scholar 

  21. 21.

    Jansen JP, Fleurence R, Devine B, Itzler R, Barrett A, Hawkins N, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR task force on indirect treatment comparisons good research practices: Part 1. Value Health. 2011;14:417–28.

    Article  Google Scholar 

  22. 22.

    Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, et al. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on indirect treatment comparisons good research practices: part 2. Value Health. 2011;14:429–37.

    Article  Google Scholar 

  23. 23.

    Egunsola O, Oshikoya KA. Comparative safety of artemether-lumefantrine and other artemisinin-based combinations in children: a systematic review. Malar J. 2013;12:385.

    Article  Google Scholar 

  24. 24.

    Béliveau A, Goring S, Platt RW, Gustafson P. Network meta-analysis of disconnected networks: how dangerous are random baseline treatment effects? Res Synth Methods. 2017;8:465–74.

    Article  Google Scholar 

  25. 25.

    Konaté A, Barro-Kiki PCM, Angora KE, Bédia-Tanoh AV, Djohan V, Kassi KF, et al. Efficacy and tolerability of artesunate-amodiaquine versus artemether-lumefantrine in the treatment of uncomplicated Plasmodium falciparum malaria at two sentinel sites across Côte d’Ivoire. Ann Parasitol. 2018;64:49–57.

    PubMed  Google Scholar 

  26. 26.

    Zongo I, Compaoré YD, Nikiéma F, Zongo M, Barry N, Somé FA, et al. Efficacy of artemether-lumefantrine and artesunate-amodiaquine as first line therapy of uncomplicated malaria in Burkina Faso, 11 years after policy change. Pan Afr Med J. 2020;35:68.

    Article  Google Scholar 

  27. 27.

    Saito M, Mansoor R, Kennon K, Anvikar AR, Ashley EA, Chandramohan D, et al. Efficacy and tolerability of artemisinin-based and quinine-based treatments for uncomplicated falciparum malaria in pregnancy: a systematic review and individual patient data meta-analysis. Lancet Infect Dis. 2020;20:943–52.

    CAS  Article  Google Scholar 

  28. 28.

    Grandesso F, Guindo O, Woi Messe L, Makarimi R, Traore A, Dama S, et al. Efficacy of artesunate-amodiaquine, dihydroartemisinin-piperaquine and artemether-lumefantrine for the treatment of uncomplicated Plasmodium falciparum malaria in Maradi. Niger Malar J. 2018;17:52.

    Article  Google Scholar 

  29. 29.

    Cameron C, Fireman B, Hutton B, Clifford T, Coyle D, Wells G, et al. Network meta-analysis incorporating randomized controlled trials and non-randomized comparative cohort studies for assessing the safety and effectiveness of medical treatments: challenges and opportunities. Syst Rev. 2015;4:147.

    Article  Google Scholar 

  30. 30.

    Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods. 2012;3:98–110.

    CAS  Article  Google Scholar 

  31. 31.

    Schmid CH, Trikalinos TA, Olkin I. Bayesian network meta-analysis for unordered categorical outcomes with incomplete data. Res Synth Methods. 2014;5:162–85.

    Article  Google Scholar 

  32. 32.

    Béliveau A, Boyne DJ, Slater J, Brenner D, Arora P. BUGSnet: an R package to facilitate the conduct and reporting of Bayesian network meta-analyses. BMC Med Res Methodol. 2019;19:196.

    Article  Google Scholar 

  33. 33.

    Whegang Youdom S, Chiabi A, Basco LK. Monitoring the efficacy and safety of artemisinin-based combination therapies: a review and network meta-analysis of antimalarial therapeutic efficacy trials in Cameroon. Drugs R D. 2019;19:1–14.

    CAS  Article  Google Scholar 

  34. 34.

    Naing C, Whittaker MA, Htet NH, Aye SN, Mak JW. Efficacy of antimalarial drugs for treatment of uncomplicated falciparum malaria in Asian region: A network meta-analysis. PLoS ONE. 2019;14:e0225882.

    CAS  Article  Google Scholar 

  35. 35.

    Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JP. Evaluating the quality of evidence from a network meta-analysis. PLoS ONE. 2014;9:e99682.

    Article  Google Scholar 

  36. 36.

    Rücker G, Schwarzer G. Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med Res Methodol. 2015;15:58.

    Article  Google Scholar 

  37. 37.

    Efthimiou O, Debray TP, van Valkenhoef G, Trelle S, Panayidou K, Moons KG, Reitsma JB, Shang A, Salanti G; GetReal Methods Review Group. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods. 2016;7:236-63.

  38. 38.

    D’Amico G, Abraldes JG, Rebora P, Valsecchi MG, Garcia-Tsao G. Ordinal outcomes are superior to binary outcomes for designing and evaluating clinical trials in compensated cirrhosis. Hepatology. 2020;72:1029–42.

    Article  Google Scholar 

  39. 39.

    Stanley CC, Kazembe LN, Mukaka M, Otwombe KN, Buchwald AG, Hudgens MG, et al. Systematic review of analytical methods applied to longitudinal studies of malaria. Malar J. 2019;18:254.

    Article  Google Scholar 

  40. 40.

    Manuguerra M, Heller GZ. Ordinal regression models for continuous scales. Int J Biostat. 2010;6:14.

    Article  Google Scholar 

  41. 41.

    Lin JT, Juliano JJ, Wongsrichanalai C. Drug-resistant malaria: the era of ACT. Curr Infect Dis Rep. 2010;12:165–73.

    Article  Google Scholar 

  42. 42.

    Nsanzabana C. Resistance to artemisinin combination therapies (ACTs): do not forget the partner drug! Trop Med Infect Dis. 2019;4:26.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the Cameroonian children and their parents or legal guardians who participated in the clinical studies cited in the references. The authors are also grateful to Pr Jean-Christophe Thalabard (University Paris Descartes-France) who was one of the PhD supervisors of the first author of this work.

Funding

None.

Author information

Affiliations

Authors

Contributions

SWY prepared the draft with a critical reflexion about novel methodological approaches to analyse database on therapeutic efficacy studies on anti-malarial drugs. LKB helped to improve the draft. Both authors read and approved the final manuscript.

Authors’ information

SWY was trained under the mentorship of Professor Jean-Christophe Thalabard (University of Paris Descartes, France), Professor Henri Gwét (Ecole Nationale Supérieure Polytechnique, Université de Yaoundé I, Cameroon), and LKB during her doctoral studies on statistical analysis of therapeutic efficacy studies conducted in Cameroonian children. SWY graduated from the National Advanced School of Engineering (University of Yaoundé I) with a PhD in applied mathematics (statistics and epidemiology). She is currently an assistant lecturer in Statistics at the University of Dschang, Cameroon.

Corresponding author

Correspondence to Solange Whegang Youdom.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

: Illustration of data analysis with an ordinal outcome: WHO criteria for anti-malarial trial using a fixed effect model. Step by step analysis and interpretation of ordinal outcomes using R.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Whegang Youdom, S., Basco, L.K. Methodological approaches for analysing data from therapeutic efficacy studies. Malar J 20, 228 (2021). https://doi.org/10.1186/s12936-021-03768-1

Download citation

Keywords

  • Plasmodium falciparum
  • Drug resistance
  • Artemisinin
  • Ordinal outcome
  • Multiple comparison
  • Network meta-analysis
  • Ranking
  • Analytics
  • Data science