Implementation of a reference standard and proficiency testing programme by the World Wide Antimalarial Resistance Network (WWARN)
© Lourens et al; licensee BioMed Central Ltd. 2010
Received: 9 October 2010
Accepted: 25 December 2010
Published: 25 December 2010
The Worldwide Antimalarial Resistance Network (WWARN) is a global collaboration to support the objective that anyone affected by malaria receives effective and safe drug treatment. The Pharmacology module aims to inform optimal anti-malarial drug selection. There is an urgent need to define the drug exposure - effect relationship for most anti-malarial drugs. Few anti-malarials have had their therapeutic blood concentration levels defined. One of the main challenges in assessing safety and efficacy data in relation to drug concentrations is the comparability of data generated from different laboratories. To explain differences in anti-malarial pharmacokinetics in studies with different measurement laboratories it is necessary to confirm the accuracy of the assay methods. This requires the establishment of an external quality assurance process to assure results that can be compared. This paper describes this process.
The pharmacology module of WWARN has established a quality assurance/quality control (QA/QC) programme consisting of two separate components:
1. A proficiency testing programme where blank human plasma spiked with certified reference material (CRM) in different concentrations is sent out to participating bioanalytical laboratories.
2. A certified reference standard programme where accurately weighed amounts of certified anti-malarial reference standards, metabolites, and internal standards are sent to participating bioanalytical and in vitro laboratories.
The proficiency testing programme is designed as a cooperative effort to help participating laboratories assess their ability to carry out drug analysis, resolve any potential problem areas and to improve their results - and, in so doing, to improve the quality of anti-malarial pharmacokinetic data published and shared with WWARN.
By utilizing the same source of standards for all laboratories, it is possible to minimize bias arising from poor quality reference standards. By providing anti-malarial drug standards from a central point, it is possible to lower the cost of these standards.
The Worldwide Antimalarial Resistance Network (WWARN) is a global collaboration, established in 2009, with the support of the Bill and Melinda Gates Foundation. The aim is to support the objective that anyone affected by malaria receives effective and safe drug treatment. Working closely with WHO, WWARN has the goal of providing urgently needed, comprehensive, timely and quality assured information to track the emergence and spread of anti-malarial drug resistance so that global efforts to control the disease are founded on reliable information. WWARN validates and provides tools to obtain this information and shares data from a continuously updated database. The Network supports and enhances relevant research capacity in malaria-endemic countries and is coordinated through regional centres established in malaria-endemic areas.
The Asia Regional Centre is the first such coordinating centre, operating out of Mahidol University in Bangkok, Thailand. WWARN is composed of six complementary modules  Clinical, Pharmacology, In vitro, Molecular, Informatics and Anti-malarial Quality. Achieving adequate anti-malarial drug concentrations in the blood is essential for reliable cure of malaria. The Pharmacology module aims to inform optimal anti-malarial drug selection and dosing by characterizing individual and population blood concentration profiles, thereby assessing the therapeutic ratio in different patient groups, estimating selection pressures and differentiating resistance from under-dosing. This review outlines the approaches taken and methods used by the WWARN Pharmacology module.
Therapeutic failure is not always an indication of resistance to the malaria parasite. It may be the result of poor drug quality, inadequate dosing, vomiting, poor absorption, or reduced drug exposure because of increases in the apparent volume of distribution or clearance. Moreover, it is essential that anti-malarial drug concentration profiles are characterized in all key target populations so that dosage regimens can be optimized. The choice of the optimal dose is dependent on many, often interrelated factors such as tablet size, acceptability, toxicity as well as the pharmacokinetic properties of the drug(s) used (which may vary with e.g. age, body mass, disease severity, coadministered food, and pregnancy). There is always a trade-off between simplicity and accuracy of the mg/kg dose. Low dosage increases the risk of treatment failure while high dosage may increase side effects. As current dose recommendations were often developed for acute uncomplicated malaria in adults, vulnerable populations (e.g. infants, malnourished children or pregnant women) may receive inappropriate dosage of the currently recommended anti-malarials [2–5].
There is an urgent need to define the drug exposure - effect (i.e. the pharmacokinetic-pharmacodynamic; PK-PD) relationship for most anti-malarial drugs. Few anti-malarials have had their therapeutic blood concentration levels defined [6–9]. Studies on artemisinin combinations have shown that a single time point (i.e. the day 7 concentration of the slowly eliminated partner drug) is a valuable determinant of total drug exposure and consequently treatment outcome [6, 7, 9].
One of the main challenges in assessing safety and efficacy data in relation to drug concentrations in various patient populations is the comparability of data generated from different laboratories. The same issues bedevils comparison of in vitro susceptibility assessments . To explain differences in anti-malarial pharmacokinetics in studies with different measurement laboratories (or assays performed at different times) it is necessary to confirm the accuracy of the assay methods. This requires the establishment of an external quality assurance process to assure results that can be compared . This paper describes how this process was set up. The results from different laboratories can now be compared through a proficiency testing programme . Inter-laboratory biases can be minimized through the use of reliable reference standards, implementation of quality systems, and external proficiency testing.
A proficiency testing programme where blank human plasma spiked with certified reference material (CRM) in different concentrations is sent out to participating bioanalytical laboratories. The quality control samples will help laboratories in the tracking of potential problem areas. The QA/QC unit will also provide assistance and advice to help laboratories solve quality or methodological issues that might arise during testing.
A certified reference standard programme where accurately weighed amounts of certified anti-malarial reference standards, metabolites, and internal standards are sent to participating bioanalytical and in vitro laboratories.
The pharmacology module of WWARN is dedicated to provide the necessary tools to ensure that pharmacology data generated around the globe is reliable and of a high standard. Informing optimal anti-malarial selection and dosing requires the global cooperation of researchers active in anti-malarial pharmacology to define adequately therapeutic drug concentrations and facilitate a better understanding of the patient, dosing and disease factors that change anti-malarial exposure sufficiently to compromise the anti-malarial efficacy (or safety). From the above it is evident that a well controlled pharmacology QA/QC programme will not only benefit the quality of data to be uploaded to the WWARN database, but also patients, contributing to the control, and eventual elimination, of malaria.
Quality assurance (QA)
QA aims at improving and stabilizing processes to minimize or avoid issues that lead to unreliable results.
Quality control (QC)
QC also known as external quality assessment (EQA) or proficiency testing (PT) emphasizes testing and blocks the release of unreliable results.
Reference material programme
Proficiency testing (PT) programme
Day-to-day running of the PT programme is the responsibility of the WWARN QA/QC unit while the long-term strategic planning is decided together with an international programme advisory committee comprising an analytical pharmacologist, a clinician and a statistician. The current members of the committee are noted on the QA/QC programme page of the WWARN website . All practices and procedures of the WWARN QA/QC unit are documented in their own quality management system (QMS). Selected procedures will be available through the WWARN web site as a guide to participating laboratories . The WWARN QA/QC unit will keep participants informed about the efficiency of the programme as a whole, any changes that are being introduced, and how any problems have been dealt with. The operation of the programme will be reviewed periodically by an independent accreditation body as part of the WWARN QA/QC units' proficiency testing provider accreditation. Overall direction of the programme will be overseen by the programme advisory committee. The unit makes use of International Harmonized Protocol for the Proficiency Testing of (Chemical) Analytical Laboratories as published in an IUPAC technical report .
Participation in the PT programme is open to all laboratories. The QA/QC unit will provide registration documents upon request. The principal investigator of a laboratory must complete the registration form and return it to the QA/QC unit who will then evaluate the application. The proficiency testing programme is limited to laboratories doing either clinical studies or research on anti-malarial drugs. In the first six months of operation, eight analytical laboratories registered with the WWARN PT programme.
PT samples will be distributed to the participants once per year. All samples will be sent in a single shipment but testing will be assessed in three cycles per year. The first cycle starts in January and ends by 30th April, the second cycle starts in May and ends by 31st August and the last and third cycle starts in September and ends by 31st December. Samples may be requested from the WWARN QA/QC unit at any time within a cycle. However, results must be reported within two weeks after the end of the cycle. The assigned value (i.e. concentration) for each anti-malarial drug substance in each sample will be disclosed to participants together with anonymized results from all participating laboratories after each cycle. Participants should use their preferred analytical method i.e. those used by the participating laboratory for routine analysis and not versions of the method specially adapted for the proficiency test. Each laboratory will be given a unique code, and will only be able to identify their laboratory in the result reports. The results will be subjected to statistical analysis and converted into scores by the WWARN QA/QC unit. All participants will be kept fully informed of the progress of the programme at all times.
The WWARN QA/QC unit will prepare all PT samples in-house using certified reference material and characterized healthy volunteer sodium heparin plasma. Other matrixes will be added in the future. PT samples are spiked with anti-malarial drugs and drug metabolites according to valid internal SOPs. Samples are frozen at -80°C immediately after preparation. The proficiency testing programme makes use of an international courier service with experience in shipping biological material. No materials will be shipped without the required documents in place. The PT programme will initially include PT samples for dihydroartemisinin, chloroquine and lumefantrine but other drugs will be included as the programme progresses after discussions with the advisory committee.
Participating laboratories should report results in a spreadsheet format indicating the units of measurement. Results below the lower limit of quantification (LLOQ) for a specific method in a laboratory should not be reported but the LLOQ for each analyte (drug/metabolite) must be reported so that the standard deviation for proficiency assessment (SDPA) can be calculated on an individual basis for each PT sample. Results submitted after the deadline for that cycle will be rejected, and submitted results cannot be corrected or withdrawn. The reason for this stringent approach is that proficiency testing is intended to test every aspect of the analytical process, including calculating, checking, and reporting a result.
Assessment of performance
where z is the performance score
x is the participant result
x a is the assigned value
SDPA is the standard deviation for proficiency assessment
Assigned value (xa)
formulation (i.e., the nominal value assigned on the basis of proportions of drug reference material used in spiked PT samples)
a consensus value (i.e., an average value derived directly from the reported results of all participating laboratories)
The number of laboratories taking part in the WWARN QA/QC programme determines what method will be used to determine the assigned value. The QA/QC unit will compare the average value of all participating laboratories with the formulation value after each cycle. If large unexplained discrepancies are found, the QA/QC unit will initiate an investigation for the root cause.
Standard deviation for proficiency assessment (SDPA)
The revised Harmonized Protocol (HP) describes several ways in which the SDPA can be obtained. It can, for example, be determined in a proficiency test as the standard deviation of all the laboratory results (excluding significant outliers). However the HP recommends that the SDPA is a set value, which corresponds to the precision needed to perform a certain task. The HP calls this SDPA the "fitness-for-purpose based standard deviation for proficiency assessment .
Example of SDPA for different LLOQ's
Lab 1 LLOQ 1
Lab 2 LLOQ 20
Calculation of SDPA for each PT sample .
1- 3× LLOQ ±25% of the assigned value
>3 - 10× LLOQ ±20% of the assigned value
>10× LLOQ ±15% of the assigned value
|z| < 2: the result is considered satisfactory.
2 < |z| < 3: the result is considered questionable.
|z| > 3: the result is considered unsatisfactory.
The WWARN QA/QC unit will provide a performance report to each participant for each cycle where the overall performance will be appraised as follows :
1. Satisfactory performance is the successful analysis of a drug by the laboratory (More than 80% of all z-scores for a drug < 2). No further action is necessary and the laboratory demonstrated proficiency in carrying out analysis for that particular drug.
2. Questionable performance is when the laboratory produces more than 20% of all z-scores for a drug > 2 but less than 10% > 3. The laboratory is advised to completely investigate every aspect of the questionable and unsatisfactory results (|z| > 2) to prevent recurrence of this problem or any other problem identified during the investigation.
3. Unsatisfactory performance is when the laboratory produces more than 20% of all z-scores for a drug > 3. The laboratory is advised to completely investigate every aspect of the unsatisfactory results to prevent recurrence of this problem or any other problem identified during the investigation.
4. Unsuccessful performance is when the laboratory displays unsatisfactory performance for a drug for two consecutive or two out of three cycles. The laboratory is advised to completely investigate every aspect of the analytical methodology and re-confirm method validation performance.
5.Critical performance is when the laboratory displays unsatisfactory performance for a drug for three consecutive cycles. The laboratory is advised to cease testing until all necessary actions are taken to correct performance
Laboratories that do not meet the criteria for successful performance for a given drug are encouraged to contact the WWARN QA/QC unit. The QA/QC unit will provide assistance and advice to help laboratories solve quality or methodological issues that might arise during testing. Reports issued to participants will be clear and comprehensive and show the distribution of anonymized results from all laboratories together with the participant's performance score. Test results used by the WWARN QA/QC unit in all calculations will be provided, to enable participants to check that their data have been correctly entered. Reports will be made available as quickly as possible after the return of results to the WWARN QA/QC unit. Participants will receive reports in clear and simple format with summary results in graphical form (e.g., as a histogram, bar chart, or other distribution plot) with appropriate summary statistics.
Summary statistics of the programme will be posted by means of anonymous identifiers via the WWARN website. Participants will be identified in reports by code only. Participants in the proficiency testing programme may communicate their own results, including the regular programme reports, privately to a laboratory accreditation or other assessment body when required for the purpose of assessment, or to clients for the purpose of demonstrating analytical capability.
On joining the programme, participants are provided with detailed information, including:
• the range of tests available and the tests the participant has elected to undertake;
• the method of setting performance criteria;
• performance criteria applicable at the time of joining, unless criteria are set separately for each test material;
• the method(s) of determining assigned values, including measurement methods where relevant;
• a summary of the statistical procedures used to obtain participant scores;
• information on interpreting scores;
• conditions pertaining to participation (e.g., avoidance of collusion with other participants);
• the composition and method of selection of the advisory committee; and
• contact details for the WWARN QA/QC unit and any other relevant organizations.
Participants will be advised of any forthcoming changes in programme design or operation, and will be encouraged to provide feedback on all aspects of programme operation and on problems with individual test materials. In this way, participants will contribute to the development of the programme and to alerting the WWARN QA/QC unit to any unanticipated difficulty with its system and PT samples.
The WWARN QA/QC programme is a dynamic programme for the distribution of certified reference standards and proficiency testing samples for anti-malarial drug measurement. It will facilitate clinical pharmacokinetic and in vitro sensitivity studies around the world. The proficiency testing programme is designed as a cooperative effort to help participating laboratories assess their ability to carry out drug analysis, resolve any potential problem areas and to improve their results - and, in so doing, to improve the quality of anti-malarial pharmacokinetic data published and shared with WWARN. To ensure the quality of pharmacokinetic data, it is important that laboratories use validated and accurate methods, but equally important that results are compared with other laboratories. By utilizing the same source of standards for all laboratories, it is possible to minimize bias arising from poor quality reference standards. By providing anti-malarial drug standards from a central point, it is possible to lower the cost of these standards. This process both assesses and empowers. Laboratories which can provide evidence of the accuracy of their analytical methods can prove to their host institutions, sponsors, and professional colleagues that they are performing at a high standard. This innovative approach improves trust, and could help translate evidence into policy and practice. It is hoped that similar programmes can be developed for other anti-infective drug measurement in tropical countries. The WWARN QA/QC unit will review the outcomes of every cycle of the programme, noting, for example, any strengths, weaknesses, specific problems, and opportunities for improvement. Every aspect of the operation of the programme, including any issues identified by the participants is continuously monitored and feedback from participants is encouraged.
This programme will hopefully facilitate for groups to conduct and analyse clinical pharmacokinetic studies of the highest quality and thereby generate desperately needed data on anti-malarial pharmacology and its impact on treatment response. The WWARN QA/QC programme is dedicated to provide a service of the highest standard and to this effect is pursuing proficiency testing provider accreditation from a nationally recognized accreditation body. It is envisaged that accreditation will be obtained as soon as the first cycle of the quality control programme is completed. Participating laboratories will then also be able to use their performance in the WWARN QA/QC programme as additional proof of their standard of service . They may also quote their results in scientific publications. Contact information for the WWARN QA/QC unit can be found through the WWARN website where also registration forms for the proficiency testing programme and/or the reference material programme can be requested .
The authors are grateful to Dr. Val Tate for her assistance in editing this manuscript and to Pak Sadomthian for technical assistance in the day-to-day running of the programme. The authors are grateful to Dr. Patrick Lukulay USP for advice during the implementation of the programme.
- WWARN - World-wide antimalarial resistance network. [http://www.wwarn.org]
- Barnes KI, Little F, Smith PJ, Evans A, Watkins WM, White NJ: Sulfadoxine-pyrimethamine pharmacokinetics in malaria: pediatric dosing implications. Clin Pharmacol Ther. 2006, 80: 582-596. 10.1016/j.clpt.2006.08.016.View ArticlePubMedGoogle Scholar
- Tarning J, McGready R, Lindegardh N, Ashley EA, Pimanpanarak M, Kamanikom B, Annerberg A, Day NP, Stepniewska K, Singhasivanon P, White NJ, Nosten F: Population pharmacokinetics of lumefantrine in pregnant women treated with artemether-lumefantrine for uncomplicated Plasmodium falciparum malaria. Antimicrob Agents Chemother. 2009, 53: 3837-3846. 10.1128/AAC.00195-09.PubMed CentralView ArticlePubMedGoogle Scholar
- McGready R, Stepniewska K, Lindegardh N, Ashley EA, La Y, Singhasivanon P, White NJ, Nosten F: The pharmacokinetics of artemether and lumefantrine in pregnant women with uncomplicated falciparum malaria. Eur J Clin Pharmacol. 2006, 62: 1021-1031. 10.1007/s00228-006-0199-7.View ArticlePubMedGoogle Scholar
- Karunajeewa HA, Salman S, Mueller I, Baiwog F, Gomorrai S, Law I, Page-Sharp M, Rogerson S, Siba P, Ilett KF, Davis TM: Pharmacokinetics of chloroquine and mono-desethylchloroquine in pregnancy. Antimicrob Agents Chemother. 2010, 54: 1186-1192. 10.1128/AAC.01269-09.PubMed CentralView ArticlePubMedGoogle Scholar
- Ezzet F, van Vugt M, Nosten F, Looareesuwan S, White NJ: Pharmacokinetics and pharmacodynamics of lumefantrine (benflumetol) in acute falciparum malaria. Antimicrob Agents Chemother. 2000, 44: 697-704. 10.1128/AAC.44.3.697-704.2000.PubMed CentralView ArticlePubMedGoogle Scholar
- Simpson JA, Watkins ER, Price RN, Aarons L, Kyle DE, White NJ: Mefloquine pharmacokinetic-pharmacodynamic models: implications for dosing and resistance. Antimicrob Agents Chemother. 2000, 44: 3414-3424. 10.1128/AAC.44.12.3414-3424.2000.PubMed CentralView ArticlePubMedGoogle Scholar
- Price RN, Uhlemann AC, van Vugt M, Brockman A, Hutagalung R, Nair S, Nash D, Singhasivanon P, Anderson TJ, Krishna S, White NJ, Nosten F: Molecular and pharmacological determinants of the therapeutic response to artemether-lumefantrine in multidrug-resistant Plasmodium falciparum malaria. Clin Infect Dis. 2006, 42: 1570-1577. 10.1086/503423.PubMed CentralView ArticlePubMedGoogle Scholar
- Price RN, Hasugian AR, Ratcliff A, Siswantoro H, Purba HL, Kenangalem E, Lindegardh N, Penttinen P, Laihad F, Ebsworth EP, Anstey NM, Tjitra E: Clinical and pharmacological determinants of the therapeutic response to dihydroartemisinin-piperaquine for drug-resistant malaria. Antimicrob Agents Chemother. 2007, 51: 4090-4097. 10.1128/AAC.00486-07.PubMed CentralView ArticlePubMedGoogle Scholar
- Bacon DJ, Jambou R, Fandeur T, Le Bras J, Wongsrichanalai C, Fukuda MM, Ringwald P, Sibley CH, Kyle DE: World Antimalarial Resistance Network (WARN) II: in vitro antimalarial drug susceptibility. Malar J. 2007, 6: 120-10.1186/1475-2875-6-120.PubMed CentralView ArticlePubMedGoogle Scholar
- Sibley CH, Barnes KI, Watkins WM, Plowe CV: A network to monitor antimalarial drug resistance: a plan for moving forward. Trends Parasitol. 2008, 24: 43-48. 10.1016/j.pt.2007.09.008.View ArticlePubMedGoogle Scholar
- Barnes KI, Lindegardh N, Ogundahunsi O, Olliaro P, Plowe CV, Randrianarivelojosia M, Gbotosho GO, Watkins WM, Sibley CH, White NJ: World Antimalarial Resistance Network (WARN) IV: clinical pharmacology. Malar J. 2007, 6: 122-10.1186/1475-2875-6-122.PubMed CentralView ArticlePubMedGoogle Scholar
- The International Harmonized Protocol for the Proficiency testing of Analytical Chemistry Laboratories. Pure Appl Chem. 2006, 78: 145-196. 10.1351/pac200678010145.Google Scholar
- Vander Heyden Y, Smeyers-Verbeke J: Set-up and evaluation of interlaboratory studies. J Chromatogr A. 2007, 1158: 158-167. 10.1016/j.chroma.2007.02.053.View ArticlePubMedGoogle Scholar
- Quartz - Forensic blood toxicology proficiency testing scheme. LGC Standards Proficiency Testing.Google Scholar
- Hund E, Massart DL, Smeyers-Verbeke J: Inter-laboratory studies in analytical chemistry. Analytica Chimica Acta. 2000, 423: 145-165. 10.1016/S0003-2670(00)01115-6.View ArticleGoogle Scholar
- Laboratory Accreditation Program (LAP) Proficiency Testing Response Requirements and Definitions. The College of American Pathologists (CAP).Google Scholar
- Viswanathan CT: Regulatory observations in bioanalytical determinations. Bioanalysis. 2010, 2: 1325-1329. 10.4155/bio.10.85.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<url>http://creativecommons.org/licenses/by/2.0</url>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.