Research | Open | Published:
Malaria rapid diagnostic kits: quality of packaging, design and labelling of boxes and components and readability and accuracy of information inserts
Malaria Journalvolume 10, Article number: 39 (2011)
The present study assessed malaria RDT kits for adequate and correct packaging, design and labelling of boxes and components. Information inserts were studied for readability and accuracy of information.
Criteria for packaging, design, labelling and information were compiled from Directive 98/79 of the European Community (EC), relevant World Health Organization (WHO) documents and studies on end-users' performance of RDTs. Typography and readability level (Flesch-Kincaid grade level) were assessed.
Forty-two RDT kits from 22 manufacturers were assessed, 35 of which had evidence of good manufacturing practice according to available information (i.e. CE-label affixed or inclusion in the WHO list of ISO13485:2003 certified manufacturers). Shortcomings in devices were (i) insufficient place for writing sample identification (n = 40) and (ii) ambiguous labelling of the reading window (n = 6). Buffer vial labels were lacking essential information (n = 24) or were of poor quality (n = 16). Information inserts had elevated readability levels (median Flesch Kincaid grade 8.9, range 7.1 - 12.9) and user-unfriendly typography (median font size 8, range 5 - 10). Inadequacies included (i) no referral to biosafety (n = 18), (ii) critical differences between depicted and real devices (n = 8), (iii) figures with unrealistic colours (n = 4), (iv) incomplete information about RDT line interpretations (n = 31) and no data on test characteristics (n = 8). Other problems included (i) kit names that referred to Plasmodium vivax although targeting a pan-species Plasmodium antigen (n = 4), (ii) not stating the identity of the pan-species antigen (n = 2) and (iii) slight but numerous differences in names displayed on boxes, device packages and information inserts. Three CE labelled RDT kits produced outside the EC had no authorized representative affixed and the shape and relative dimensions of the CE symbol affixed did not comply with the Directive 98/79/EC. Overall, RDTs with evidence of GMP scored better compared to those without but inadequacies were observed in both groups.
Overall, malaria RDTs showed shortcomings in quality of construction, design and labelling of boxes, device packages, devices and buffers. Information inserts were difficult to read and lacked relevant information.
The use of malaria RDTs is rapidly expanding
Prompt parasitological confirmation by microscopy or alternatively by RDTs is recommended in all patients suspected of malaria before treatment is started . As a consequence, malaria rapid diagnostic tests (RDTs) are increasingly used as a diagnostic tool in both malaria endemic and non-endemic settings: in 2007, more than 70,000,000 tests were performed .
Malaria RDTs are so-called immunochromatographic tests that detect Plasmodium antigens in the blood by an antigen-antibody reaction on a nitrocellulose strip. The antigen-antibody complex is conjugated to colloidal gold, and a positive result is visible as a cherry- or purple-red coloured line. Apart from a control line, there are one, two or three test lines: the so-called two-band tests comprise a control line and a single test line, and are mostly designed to diagnose Plasmodium falciparum. Their targets are either histidine-rich protein-2 (HRP-2) or P. falciparum-specific parasite lactate dehydrogenase (Pf-pLDH). Three-band RDTs display a second test line mostly targeting antigens common to the four species such as pan-Plasmodium-specific parasite lactate dehydrogenase (pan-pLDH) or aldolase. The four-band RDTs have an additional third test line targeting Plasmodium vivax-specific pLDH (Pv-pLDH).
Written instructions add to the correct performance and interpretation of RDTs
RDTs are accurate and robust but they have limitations linked to design, production and distribution [3–8]. In addition, there are errors at the level of the end-user, which apply to both laboratory staff and field workers and are related to sampling, testing and interpretation of RDTs [9, 10]. Clearly written instructions can add to the comprehensibility and maximize RDT kit performance [9, 10]. On this basis, the World Health Organization (WHO) designed easy-to-read generic job aids .
During field visits in Africa, teams of the Institute of Tropical Medicine (ITM) occasionally noted shortcomings in RDT kit boxes, content and instructions. In addition, part of the interpretation errors that were observed during a recent external quality assessment (EQA) on RDTs were shown to be related to errors in the information inserts of the RDT kits used . Inspection of these information inserts also revealed a large variety in layout and readability, as well as variations in the adequacy of labelling of RDT boxes and devices.
Objectives of the present study
In view of the observations above, it was decided (i) to assess malaria RDT kits for adequate and correct design, construction and labelling of boxes and components, and (ii) to study the readability and accuracy of their information inserts.
Selection of RDT kits
Malaria RDTs marketed as devices consisting of cassettes, cardboard boxes and hybrids (nitrocellulose strips to be dipped into plastic wells) were selected. They were checked for the presence of the CE label and evidence of good manufacturing practice (GMP) based on their inclusion in the WHO lists of RDT manufacturers and distributors complying with ISO13485:2003 or US FDA 21 CFR 820 production norms .
Criteria used for RDT kit assessment and procedure
For packaging, design and labelling, assessment criteria were compiled from requirements listed in regulatory documents such as the Directive 98/79/EC and the European Community (EC) as well as relevant WHO documents [14–17]. Criteria for information inserts and device design from studies on end-users' performance and RDT instructions were pooled [5, 7, 9, 10, 18–24]. Inadequacies were defined as listed in Table 1.
RDT kit package, device package, device and buffer vial
The RDT kit packages were assessed for type (box versus plastic bag), material (simple and plasticized cardboard) and the presence and quality of the printed information. Information displayed on the package considered as essential included the RDT kits and manufacturer's names, expiry date, number of tests included, storage requirements and a reminder to read the instructions before use. A referral to the intended use of the RDT kit was looked for, either by the RDT kit name or by an additional text. The expiry date mentioned on the box was matched with those of the other RDT kit components. For CE labelled RDT kits produced by companies outside the European Economic Area, the affixing of the so-called authorized representative of the company in the EC (EC-REP) was assessed. Kits were assessed for sampling material needed included or not.
The package of the test device was checked for quality (humidity-proof material) and essential information including RDT kit name, lot number and expiry date. In addition, the desiccant was checked for composition, warning label and presence of colour indicator.
The RDT devices (cassette or cardboard housing the nitrocellulose strip) were assessed for clearness of design and construction including referral to the RDT kit's name. The space allocated for sample identification was evaluated for dimensions and ease of writing. A space of minimal 0.5 cm height and 4 cm wide was considered as adequate for handwriting of sample identification. The labelling of buffer wells, sample wells and reading windows including the places of appearance of the control and test lines (further referred to as reading label) were assessed for visibility and unambiguous interpretation.
The buffer vials were assessed for leak-proof closure, and their labels for quality of adherence and print. The information displayed on the label was assessed for the presence of RDT kit name, lot number, expiry date and storage conditions.
RDT information insert
RDT kits were checked for the presence of an information insert and a job aids (short procedure version), of which date of release and version number were assessed.
Layout and figures
The figures were counted and their dimensions measured. Their total surface area was calculated and expressed as a percentage of the total surface of the information insert. The figures were assessed for their concept (pure black and white versus use of colours) and conformity with the real devices.
The font size of the predominant letter type used (excluding the bibliography section) was measured in Cicero using a typometer (Rotring-werke Riepe KG, Hamburg, Germany) as the "kp" distance from the top of the highest ascender (top of the lower case letter k) to the bottom of the lowest descender (bottom of the lower case letter p). The opening of the characters was visually assessed for the characters "c, o and a", by covering them for their lower two-thirds and checking whether they were still correctly readable (open letter type) versus read as an "o" (closed letter type). The interline spacing was assessed by measuring in Cicero with a typometer the distance between the base line of two successive rows and then subtracting the font size. Fonts of open letter types and interline spacing equal or larger than 2 are better readable compared to fonts of closed letter types and interline spacing smaller than 2, especially at larger text columns. For patient education materials and health instructions, font sizes of 12 or larger are recommended [25, 26].
For assessment of the readability level, the sessions about blood sampling, procedure and interpretation in the English text version were copied or retyped in Microsoft Word (Microsoft Corp., Redmond, WA, U.S.A.) and checked for correct spelling and syntax construction. Follow-up editing was performed as described elsewhere . Next, the text fragments were copied into an on-line readability assessment tool, which generates different reading indices . The Flesch-Kincaid grade level  was calculated. This grade-level expresses the U.S. grade-level equivalency of the skills required to read a particular document. For patient education materials and health related information, the recommended level is ≤ 6th grade level [25, 30].
Accuracy and relevance of information
The following items were actively looked for: description of the RDT test principle, target antigens, listing of required materials provided and not, description of sampling procedures and biosafety precautions. The RDT test procedures were studied with reference to common errors made by end-users in the field (Table 2) [5, 9–11, 18, 19, 23]. The interpretation section was assessed for the complete description of invalid results and Plasmodium species differentiation as well as for listing causes of false negative and false positive results. The description of test characteristics was assessed for mentioning the diagnostic accuracies related to the different Plasmodium species and parasite densities. Bibliographic references were checked for relevance with regard to RDT performance in general and information on the RDT kit's performance in particular.
Assessment, data registration and statistical analysis
Two observers trained in the use of RDTs independently assessed the RDTs according to the described criteria. Discrepant observations were discussed together with the other investigators and a consensus was reached. Data were registered in an Excel sheet (Microsoft Corp., Redmond, WA, U.S.A.).
Panel of RDT kits
For the purpose of this study, 51 RDT kits were ordered at 29 companies. Seven companies (representing nine RDT kits) did not reply despite several reminders. The final panel consisted of 42 RDT kits from 22 companies. Nearly all (39/42, 93%) RDT kit formats were cassettes, further there were one cardboard and two hybrid kits. Two RDT kits consisted of individually wrapped RDT packages containing all materials for a single test (cassette, disinfectant, lancet and buffer). They will be further referred to as "Single RDT kits". Table 3 lists the RDTs according to their evidence of GMP. Table 1 lists the number of the RDTs with inadequacies in boxes, devices, buffer vials and information inserts.
RTD kit package, device and buffer vial
RDT kit package
Thirty-eight RDT kits arrived as cardboard boxes; four kits arrived in plastic bags. Two of these plastic bags contained a cardboard box to be folded by the end-user, resulting in a total of 40 boxes and two plastic bags as the package of use on the bench. All but one box displayed an indication in the RDT kit's name or in the test description that the RDT kit was intended for malaria diagnosis. The two plastic bags did not display any information. A company name was listed on all the 40 boxes, but for 12 kits, it was not clearly mentioned whether this name represented the manufacturer or the distributor.
The lot number and expiry date were listed on all boxes. One of the Single RDT kits showed both lot number and expiry date on the outer box containing the single packages, but not on the single packages themselves. There were no discrepancies between the expiry dates on the RDT kit box and those of the contents except for two buffer vials with expiry dates extending those printed on the RDT kit box.
The number of tests included and a reminder to read the instructions before use were not displayed on four and ten boxes respectively. All the 40 boxes showed information on storage temperature requirements, by written text, symbols or both. Apart from a single symbol, i.e. a penguin expressing "do not freeze", all symbols were internationally recognized symbols complying with EN 980:2008 or FDA 2004, 21 CFR 809.10 and 21 CFR Parts 610 and 660. Capillary blood sampling systems were included in eight kits and proposed as optional in ten other kits. Blood transfer systems were missing in three kits.
Four of 42 device packages were not made of humidity-resistant material. Most of the packages were easy to open by tearing a pre-cut lid of the package. However, for three packages, scissors had to be used to open the packages properly.
A desiccant was present in all but one package. Three of the 41 desiccants did not show a warning that the desiccant was harmful: two of them (from one manufacturer) were tablets looking like drug pills. Seventeen desiccants (including the two tablets) had no colour indicator of humidity saturation.
Space for writing was too small in 40 of the 42 devices (Figure 1). For two cassettes, a felt pen was required as a standard pen failed to mark.
Most cassettes (35/39) had separate wells for sample and buffer application, four had a single well for sample and buffer application. There was no uniform labelling of the wells: for instance characters "S" and "A" were used randomly for the sample well, buffer well and combined sample/buffer well (Figure 2). Fifteen cassettes showed at the distal end a window or holes that might be confused with a sample or buffer well (Figure 1 and 2).
The reading label was indicated with acronyms (n = 20), characters or numbers. Acronyms included abbreviations such as "Pf" or "pan", they were printed on the plastic housing or on a label and were well readable. Characters such as "C" (control line) and "T" and numbers were embedded in the plastic housings and were more difficult to distinguish (Figure 2). In one cassette, characters were printed on a label, which was not well fixed (Figure 1). In 14 cassettes, two labels were displayed at either side of the reading window (Figure 2), and one three-band cassette had no reading label at all (Figure 2).
RDT buffer vials
The two Single kit RDTs contained a small buffer plastic ampoule in each device package, which were too small to display information. Fifteen buffer vials required clockwise tightening the vial cap to pierce the dropper vial nozzle, but for five of them, this was not mentioned in the information insert. For 13 vials, the label was not well fixed and the printed information on three of these labels was not humidity-resistant. Lot number and expiry date were not listed on five vials and storage conditions were missing on 10 vials.
RDT information insert
The information inserts of the two Single RDT kits were not considered: one of them contained a simple job aids explaining the procedure by figures only, the other contained a shortened version of the information insert of the same RDT marketed as laboratory kit. All of the remaining 40 RDT kits contained an information insert of which seven had an additional job aids. Either version number or date of issue was missing in 11 and 13 of them; in five, both were missing.
Layout and figures
All 40 inserts included figures. The median number of figures per information insert was 8.5 (range 2 - 25) and figures accounted for a median surface ratio of 7.2% (range 0.4% - 33%) of the entire insert. The median size of the figures was 2.4 × 2.0 cm, the smallest and largest figure measured respectively 1.0 × 0.3 cm and 6.7 × 8.0 cm.
All inserts used figures to illustrate the interpretation section. Other figures depicted blood sampling (n = 17), application of sample and buffer (n = 21) and a clock indicating the correct reading delay (n = 10). Fourteen inserts used red colour to indicate control and test lines but in four, they were pictured green or blue. Most inserts (n = 35) showed differences between depicted and real devices of which some were major, such as discrepancies between characters used for sample and buffer well identification (n = 4) and differences in labelling of the reading window (n = 5) (Figures 3 and 4). One insert mentioned a reading delay of 15 minutes, but the illustration mentioned a reading delay of 20 minutes (Figure 5).
Typographic features and readability
Figure 6 displays the font size and line spacing of the information inserts. Median font size was 8 ± 1.3, none of them exceeded 10. User-unfriendly typographic features included combinations of font sizes of eight or smaller with a closed letter type (n = 10) or with line spacing lower than two and more than 12 words per line (n = 14) (Figure 7). Median readability level was grade 8.9 (range 7.1 - 12.9) and 18 and four of the inserts' readability levels were above grade 9 and 10 respectively (Figure 8). Readability levels of the job aids were also high (median 8.5, range 5.1 - 9.4), and six out of seven exceeded the readability level of the most recent WHO job aids. Five inserts showed prints of very poor quality hindering reading of the text (Figure 9).
Accuracy and relevance of information
The RDT kit's principle was described in all 40 information inserts. All but two inserts mentioned the materials provided in the RDT kit, and seven inserts provided a complete list of the materials required.
All RDT kit inserts mentioned the required specimen (in all cases both capillary and venous blood), all but one mentioned the anticoagulant to be used. Capillary blood sampling through finger prick was described in 35 inserts, of which one also added sampling by venipuncture. By contrast, the heel prick was not described in any information insert.
Biosafety precautions included the use of gloves (depicted or mentioned in 21 inserts, Figure 10) and safe waste disposal (addressed in 16 inserts), but 18 inserts did not mention any information on biosafety.
From Table 2 it is clear that a number of critical steps in RDT procedures were addressed by only part of the RDT inserts. Among them, there were relevant steps such as writing down sample identification, correct positioning of the transfer and buffer vial and the need for an adequate light source.
The complete array of all control and test line combinations was listed by only nine inserts. Fourteen inserts mentioned the absence of all lines as an invalid result but not the presence of a test line in the absence of a control line (Figure 9). In addition, errors in the interpretation of test lines were observed. For instance, the combination of a Pf-specific and a pan-specific test line was interpreted as P. falciparum without mentioning the possibility of a mixed infection (12/34 three- and four-band RDTs). Likewise, the combination of a Pv-specific and a pan-specific test line in case of a four-band RDT was interpreted as a Plasmodium vivax infection without mentioning the possibility of a mixed infection with Plasmodium ovale and Plasmodium malariae (6/7 RDTs). In addition, a visible pan-pLDH line was interpreted as a P. vivax infection (instead of non-falciparum species) in two inserts.
Few inserts mentioned causes of false positive and false negative results (Table 2). One insert recommended to repeat the test in case of a negative RDT result and persistent suspicion of malaria, another warned about the prozone effect as a cause of a false negative result (Figure 4). RDT test results during treatment follow-up were addressed in 13 inserts, but the information listed in nine inserts was presented in a scattered way and only one insert clearly mentioned that HRP-2 persistence does not indicate a failed therapeutic response.
RDT test characteristics
Eight inserts did not provide information on sensitivity or specificity. Diagnostic characteristics were mostly expressed for P. falciparum and P. vivax (n = 31 and n = 22 respectively), only one insert mentioned test characteristics for P. ovale and P. malariae. Sensitivity for P. falciparum was expressed by parasite density range in 10 inserts.
Bibliography cited in the information inserts
In total, 45 different references were used in the bibliography of the information inserts. One third of them referred to the original description of the target antigens, another 12 referred to general information on malaria and its diagnosis. Thirteen inserts cited evaluation studies of RDTs, but only three RDT kits referred to product-related studies. Two panels of identical references were shared by nine and eight inserts respectively.
RDT kits' names
Inconsistencies in the RDT kit names and referrals to target antigens were observed. For instance, four RDT kits had names referring to P. vivax although they used a pan-species Plasmodium antigen. Two other inserts did not mention the identity of the pan-species antigen (aldolase versus pan-pLDH). Five RDT kits from one manufacturer were supplied in identical boxes, carrying the same names and identical prints. Furthermore, there were slight but numerous differences between names as displayed on boxes versus those noted on device packages (eight had no brand name affixed), devices (15 differences) and information inserts (eight differences). Similar observations were made for buffer vials: five vials displayed only the manufacturer's name and one vial did not show brand nor manufacturer's name.
RDT kits' duplicates
During assessment of the RDT kits, apparent similarities between different RDT brands were observed. These similarities concerned, amongst others, design and shape of the device and content and layout of the information insert (e.g. numbers of samples used for calculation of test characteristics). In that way, six products were assumed to represent a common design and production platform for 16 different RDT brands.
Relation with CE marking and WHO listing
Overall, RDTs with evidence of GMP (n = 35) scored better compared to those without (n = 5), although inadequacies, errors and omissions were observed between both groups. Three CE labelled RDT kits produced outside European Economic Area (EEA) had no EC-REP indicated neither on the box nor in the information insert. For these kits, the CE symbol as displayed on the package had not the shape and relative dimensions of the 98/79/EC directive (Figure 11). Six of the eight RDT kits that did not mention data on sensitivity and specificity were CE labelled.
Previous studies demonstrated that RDT manufacturers' instructions are insufficient to ensure accurate test performance by community health care workers, and well-designed instructions such as the WHO generic job aids have proven to increase performance [9, 10, 19, 31]. In practice however, such job aids still need to be adapted to the particular RDT brand used on site and, depending on the chains of supply, different RDT brands and versions may be available. For market exploration and choice of RDTs, laboratory managers will orient to RDT kits' names and labelling. Clear design and labelling of RDT kit components will contribute to correct storage and use; and laboratory staff will rely on RDT inserts for background information, adaptations of the generic procedures, interpretation and trouble-shooting.
The design of the present study has its limitations. For instance, mainly RDTs marketed as cassettes were considered. However, cassettes are the mostly used platform and preferred by end-users over the strip format . Besides, only a part of the marketed brands was evaluated; however, this evaluation studied about half of the 80 brands worldwide-marketed  including those frequently used in endemic and non-endemic settings.
With regard to the assessment of the information inserts, it should further be noted that the Flesch-Kincaid label (as discussed below) is only a proxy measure of readability. In addition, the layout of the inserts was assessed for typography but not for other features such as adequate use of headings, bullets, boldfacing, and amount of white space [32, 33]. Finally, although RDTs were presently assessed against compiled criteria based on relevant documents, they were not evaluated by end-users in a real-life setting. On the other hand, as far as known, this is the first time that in vitro diagnostic medical devices (IVDs) were assessed for these characteristics.
Problems in design and labelling of the RDT kits' components
Despite these limitations, much information was generated on the quality and appropriateness of RDT packaging, labels and inserts. Apart from two exceptions (the plastic bags), boxes were well labelled, but shortcomings in labelling of device packages and buffer vials were more numerous. The absence of blood sampling and transfer system which was observed in nearly a quarter of RDT kits may create logistical problems when used in field settings. The problems in device design were of most concern. Characters indicating wells and reading labels that were embedded in the plastic cassette housings are difficult to distinguish and the simultaneous presence of two reading labels may cause confusion. Standardized and unequivocal characters or acronyms should be used for designating wells and reading labels and clear labelling with contrasting print should be ensured.
Some design issues were not compiled from previous studies but originated from ITM observations. Some of them may look trivial but have consequences in daily practice. For instance, device packages without pre-cut lids require scissors to open, which is neither safe nor practical in busy and remote settings. The space allocated for sample identification on most cassettes was large enough for writing down a sample number but not a patient name: this may meet the requirements in computerized settings but not those in a non-computerized field setting, where full names are written as recommended by WHO job aids instructions . Likewise, it is evident that use of a felt pen for writing down sample identification is inappropriate for a field setting. Another example was the clockwise tightening of the buffer vial's cap for piercing the dropper bottle nozzle: ITM teams observed that laboratory staff unaware of this procedure simply cut off the distal end from the nozzle. The resulting opening tended to be very wide resulting in a too large volume of buffer added to each test and early emptying of the buffer vial. The RDT cassettes that remained without buffer were run with other buffers or with injection water, at the risk of causing false positive results .
Readability level and typography of the information insert
The Flesch-Kincaid grade level used in the present study has been demonstrated to be reliable and valid and is frequently used in health care issues such as consumer medical information (CMI) and patient education materials [27, 30]. Its use in the present context should be interpreted with caution. Like any other reading formula, the Flesch-Kincaid readability tool assesses text structure but does not take into account the content. It further refers only to US grade levels and applies to English language. Other factors such as motivation and previous experience may influence comprehension, and linguistic and cultural issues may interfere . Despite these limitations, it is of note that readability levels of all inserts exceeded the 6th grade level, while the recommended level for health related information is equal or lower than this level [27, 30]. The readability level of the job aids scored slightly better, but still higher compared to the WHO generic job aids. Too elevated reading levels have been consistently demonstrated in CMI documents such as those of home pregnancy tests, blood glucose monitoring and home blood pressure monitor equipment as well as in patient education brochures [25–27, 32, 34, 35]. It should also be taken into account that end-users in endemic settings are likely to be non-native speakers of the language of the information insert (e.g. English, French, Portuguese), and may operate in stressful situations such as environmental disasters and war [36, 37] which decrease actual reading levels . Apart from the bad quality prints and small figures, the observed user-unfriendly typographic features may add to the decreased readability of the inserts, which were consistent with those documented for CMI materials [25, 30, 32, 34, 35].
Content of the information insert
The lack of referral to biosafety procedures in nearly half of the inserts was striking and unacceptable, in particular because this is clearly mentioned in the WHO generic job aids . Likewise were the differences between depicted and real devices and the use of non-realistic colours for depicting test lines which do not comply with WHO recommendations . Numbers and sizes of illustrations did not comply with the established standards for patient education materials and CMI [25, 35, 39].
The shortcomings in the RDT test interpretation session were in line with observations made during a recent external quality assessment on RDTs in a non-endemic setting. During that session, not reporting a mixed infection in case of the simultaneous presence of P. falciparum- and pan-specific test lines was demonstrated to be linked to the information inserts of the RDT kits used . Although extremely rare in the experience at ITM, the presence of a visible test line in the absence of a control line points to invalid test results and should be added to the spectrum of possible line combinations, preferably with a picture. In addition, the persistence of HRP-2 after successful treatment and the production of pLDH by gametocytes should be clearly mentioned.
The poor description of diagnostic characteristics in the information inserts was another concern. Although current directives and recommendations do not specify details about origin, numbers and statistical validity of these test characteristics, manufacturers should be encouraged to provide as detailed and sound data as possible, including data on sensitivity in relation to parasite density and Plasmodium species. The low diagnostic sensitivity for P. malariae and P. ovale is well known : few studies have included enough samples to provide reliable data for both species. Those that did mostly found a poor sensitivity, in particular for P. malariae [40–44]. In the absence of a thorough evaluation for both species, one could consider adding a statement mentioning the low overall diagnostic sensitivity for both species to the information insert, in order to avoid unrealistic expectations by the end-user relying on the pan-Plasmodium species nature of the targeted antigen .
The cited bibliography mainly referred to the original papers on the description of the antigens or general RDT evaluations. In addition to references addressing the RDT kit itself, references to one or more of the recent reviews on RDTs or WHO/FIND documents could be added, as they contain relevant information on the use and limitations of RDTs.
Names and duplicates of RDTs, relation to GMP and CE labelling
Among the inadequacies, erratic and inconsistent names were a frequent finding: they ranged from minor differences in RDT brand names as displayed on boxes, devices and their packages and information inserts to brand names suggesting P. vivax despite using a pan-pLDH target. In addition, shortcomings with regard to clear specification of target antigens were noted. For a laboratory manager finding his way among many other diagnostics and supplies, it is essential to get a quick and reliable idea about the intended use and the target antigens of RDTs. An unequivocal code for naming and short test descriptions should be considered, with mentioning of the (abbreviated) antigens as a requisite (e.g. Pf-pLDH, HRP-2 etc.).
With regard to the presumed RDT kit duplicates (kits presenting with similar presentation suggesting a shared design and production), it should be noted that WHO and FIND recognize this phenomenon . WHO defines so-called "re-branded" products as products manufactured under identical conditions at the same manufacturing site as the original product, but labelled with a different product name and identifier. WHO encourages, in such case, joint application for the prequalification program or test evaluations [45–47]. The CE recognizes also the "re-branding" for commercial interests. The coexistence of multiple names for the same product however may create difficulties for instance in retrieving published information on test evaluations and may add to the complexity of post marketing surveillance, including traceability in case of batch recalls. To prevent these problems, the requirement of "re-branded" RDT kits to mention the original manufacturer should be considered. In addition, any RDT kit label (whether original or re-branded) should clearly distinguish the names of the manufacturer from that of the local distributor.
Shortcomings and errors were observed among CE-labelled and WHO-listed RDTs. Not affixing the EC-REP when required and not mentioning information on RDT test characteristics do not conform the 98/79/EC Directive . It should further be noted that in case of malaria RDTs, the CE-label by itself is not a guarantee for intrinsic quality of performance. The 98/79/EC Directive includes the "Annex II", which lists diagnostics for which market release of any new lot has to be preceded by testing and approval by a competent authority, the so-called notified body. For diagnostics that are not listed in the "Annex II" (such as malaria RDTs), such testing and authorization are not required. Acquisition of the CE-label for these diagnostics is a purely administrative process, in which the manufacturer himself draws up the EC declaration of conformity. Unfortunately, the majority of laboratory and medical staff are unaware of this procedure. This can create a sense of "over-confidence" in CE-labelled products, based on the perception of quality associated with European labels. The inclusion of RDTs for malaria and other tropical diseases in the "Annex II" could represent a significant support for countries with weak regulatory overview.
What can be done to improve the quality of RDT package and information inserts?
Many shortcomings such as incomplete and incorrect labelling of boxes, device packages, cassettes and buffer vials can be easily remediated at minimal costs. End-users and manufactures should reach consensus on uniform codes for labelling wells and reading windows and gradually reinforce the requirements for inserts and packaging. Generic recommendations as how to layout and how to appropriately design information inserts as well as use of figures can be found in the literature on CMI [25, 35, 39], RDT specific guidelines have been issued by the WHO . Readability, cultural and linguistic backgrounds should be taken into account , and all texts and figures should be assessed for appropriateness and comprehension among the targeted end-users [10, 49]. For the content of RDTs, the reference documents that were used to compile Tables 1 and 2 can give guidance. Examples of such checklists are added as additional files (Additional file 1 Table S1 and Additional file 2 Table S2). The information inserts should highlight key points in performance and interpretation in order to reduce the likelihood of potential errors.
With regard to user-friendliness and adequacy of RDT presentation and instructions, interesting features not listed on the compiled criteria were noted. For instance, some device packages carried short instructions for use based upon the generic WHO job aids , witch may increase the test performance [9, 10, 19, 31]. The availability of job aids for the different brands on the company websites in an adaptable text format may help Malaria National Programs to translate it in the end-user language and to adapt it on the local context. In addition, some RDT kits provided a glossary with explanation of the affixed symbols, which may help in their comprehension and acquisition and another five had all essential information printed on a single (lateral) side of the box, contributing to easy storage (Figure 12). Another asset was the presence of more than one buffer vial per RDT box, as shortage and replacement of buffer vials is a common problem in resource limited settings . Likewise, the above described observations and corrective measures could be extended to other IVDs, such as human immunodeficiency virus RDTs.
Of course, it should be noted that adequate packages and information inserts by themselves are not a guarantee for competent use of RDTs. Simply distributing the RDTs and instructions does not work, and RDT instructions on their own will not change professional behaviours . Thorough training and performance monitoring are needed for correct performance .
In conclusion, malaria RDTs showed shortcomings with regard to quality of construction, design and labelling of boxes, device packages, devices and buffers. Information inserts were difficult to read and lacked relevant information. Particular problems were observed in the consistency and appropriateness of RDT brand names and in the referral to the antigens used. In general, CE-labelled and WHO-listed RDTs scored better compared to those without but inadequacies were observed among these RDTs. Addressing the quality of RTD package and information inserts in evaluation programs such as the WHO/FIND products testing program could stimulate the manufacturers to remediate these shortcomings. Likewise, inclusion of malaria RDTs in the "Annex" II of the 98/79/EC directive might represent a powerful support from the European Community towards the quality of in-vitro diagnostics in tropical countries.
Consumer medical information
European Authorized Representative
European Economic Area
Food and drug administration
Foundation for Innovative New Diagnostics
Faculty of Health, Medicine and Life Sciences
Good manufacturing practice
Histidine-rich protein 2
International organization for standardization
Institute of Tropical Medicine
in vitro diagnostic medical devices
- P :
pan Plasmodium-specific parasite lactate dehydrogenase
Plasmodium falciparum-specific parasite lactate dehydrogenase
parasite lactate dehydrogenase
Plasmodium vivax-specific parasite lactate dehydrogenase
Rapid diagnostic test(s)
World Health Organization.
World Health Organization: Guidelines for the treatment of malaria. 2010, [http://whqlibdoc.who.int/publications/2010/9789241547925_eng.pdf]second
Bell D, Perkins MD: Making malaria testing relevant: beyond test purchase. Trans R Soc Trop Med Hyg. 2008, 102: 1064-1066. 10.1016/j.trstmh.2008.05.007.
Chiodini PL, Bowers K, Jorgensen P, Barnwell JW, Grady KK, Luchavez J, Moody AH, Cenizal A, Bell D: The heat stability of Plasmodium lactate dehydrogenase-based and histidine-rich protein 2-based malaria rapid diagnostic tests. Trans R Soc Trop Med Hyg. 2007, 101: 331-337. 10.1016/j.trstmh.2006.09.007.
Gillet P, Mori M, Van Esbroeck M, Van Den Ende J, Jacobs J: Assessment of the prozone effect in malaria rapid diagnostic tests. Malar J. 2009, 8: 271-10.1186/1475-2875-8-271.
Gillet P, Mori M, Van Den Ende J, Jacobs J: Buffer substitution in malaria rapid diagnostic tests causes false-positive results. Malar J. 2010, 9: 215-10.1186/1475-2875-9-215.
Jorgensen P, Chanthap L, Rebueno A, Tsuyuoka R, Bell D: Malaria rapid diagnostic tests in tropical climates: the need for a cool chain. Am J Trop Med Hyg. 2006, 74: 750-754.
McMorrow ML, Masanja MI, Abdulla SM, Kahigwa E, Kachur SP: Challenges in routine implementation and quality control of rapid diagnostic tests for malaria--Rufiji District, Tanzania. Am J Trop Med Hyg. 2008, 79: 385-390.
World Health Organization: Lot-testing: Using quality control parasite dilutions to test the sensitivity of malaria RDTs. 2010, [http://www.wpro.who.int/sites/rdt/using_rdts/qa/lot_testing.htm]
Rennie W: Developing and testing a generic job aid for malaria rapid diagnostic tests (RDTs). 2004, [http://www.wpro.who.int/internet/resources.ashx/RDT/docs/pdf_version/Developing_and_testing_an_RDT_Job_Aid.pdf]
Rennie W, Phetsouvanh R, Lupisan S, Vanisaveth V, Hongvanthong B, Phompida S, Alday P, Fulache M, Lumagui R, Jorgensen P, Bell D, Harvey S: Minimising human error in malaria rapid diagnosis: clarity of written instructions and health worker performance. Trans R Soc Trop Med Hyg. 2007, 101: 9-18. 10.1016/j.trstmh.2006.03.011.
World Health Organization: How to use a Rapid Diagnostic Test (RDT): A guide for training at a village and clinic level. 2008, [http://www.wpro.who.int/sites/rdt/using_rdts/training/rdt_training_falciparum.htm]
World Health Organization: List of known commercially-available antigen-detecting malaria RDTs with adequate evidence of good manufacturing practice. 2009, [http://www.wpro.who.int/internet/resources.ashx/RDT/docs/MD_table34+(1)_totallistofISO131485criteria.pdf]
Maltha J, Gillet P, Cnops L, Van Den Ende J, Van Esbroeck M, Jacobs J: Malaria rapid diagnostic tests: Plasmodium falciparum infections with high parasite densities may generate false positive Plasmodium vivax pLDH lines. Malar J. 2010, 9: 198-10.1186/1475-2875-9-198.
World Health Organization: Hepatitis B Assays: Operational Characteristics Report 1 & 2. 2004, [http://whqlibdoc.who.int/publications/2004/9241592206.pdf]
World Health Organization: The Use of Rapid Malaria Diagnostic Tests, Second Edition. 2006, [http://www.wpro.who.int/NR/rdonlyres/A30D47E1-1612-4674-8DF8-FCA031CDB9BA/0/Reduced_web2_MalariaRDT_20062ndedition.pdf]
World Health Organization: Notes on job aids for malaria rapid diagnostic tests (RDTs). 2006, [http://www.wpro.who.int/NR/rdonlyres/06B185E0-7E63-4991-B8F5-B3D62D6C8373/0/SHORTNOTESONJOBAIDfinal.pdf]
World Health Organization: HIV Rapid Assays: Operational Characteristics Report 16. 2009, [http://www.who.int/entity/diagnostics_laboratory/publications/Report16_final.pdf]
Gillet P, Mukadi P, Vernelen K, Van Esbroeck M, Muyembe J-J, Bruggeman C, Jacobs J: External quality assessment on the use of malaria rapid diagnostic tests in a non-endemic setting. Malar J. 2010, 9: 359-10.1186/1475-2875-9-359.
Harvey SA, Jennings L, Chinyama M, Masaninga F, Mulholland K, Bell DR: Improving community health worker use of malaria rapid diagnostic tests in Zambia: package instructions, job aid and job aid-plus-training. Malar J. 2008, 7: 160-10.1186/1475-2875-7-160.
Knebel E: The Use of Manual Job Aids by Health Care Providers: What Do We Know?. 2000, [http://www.qaproject.org/pubs/PDFs/ISSUESJA.PDF]
Moonasar D, Goga AE, Kruger PS, La Cock C, Maharaj R, Frean J, Chandramohan D: Field evaluation of a malaria rapid diagnostic test (ICT Pf). S Afr Med J. 2009, 99: 810-813.
Murray CK, Bell D, Gasser RA, Wongsrichanalai C: Rapid diagnostic testing for malaria. Trop Med Int Health. 2003, 8: 876-883. 10.1046/j.1365-3156.2003.01115.x.
Seidahmed OM, Mohamedein MM, Elsir AA, Ali FT, Malik eF, Ahmed ES: End-user errors in applying two malaria rapid diagnostic tests in a remote area of Sudan. Trop Med Int Health. 2008, 13: 406-409. 10.1111/j.1365-3156.2008.02015.x.
Tavrow P, Shabahang J, Makama S: Vendor-to-vendor education to improve malaria treatment by private drug outlets in Bungoma District, Kenya. Malar J. 2003, 2: 10-10.1186/1475-2875-2-10.
Wallace LS, Keenum AJ, Roskos SE, Blake GH, Colwell ST, Weiss BD: Suitability and readability of consumer medical information accompanying prescription medication samples. Patient Educ Couns. 2008, 70: 420-425. 10.1016/j.pec.2007.11.017.
Wallace LS, Zite NB, Homewood VJ: Making sense of home pregnancy test instructions. J Womens Health (Larchmt ). 2009, 18: 363-368. 10.1089/jwh.2008.0985.
Wallace LS, Keenum AJ, Roskos SE, Koopman RJ, Young KG: Blood glucose monitor quick reference guides: are they suitable for patients?. Diabetes Technol Ther. 2008, 10: 11-15. 10.1089/dia.2007.0258.
Tests Document Readability. [http://www.online-utility.org/english/readability_test_and_improve.jsp]
Flesch R: How to Write Plain English. 2004, [http://www.mang.canterbury.ac.nz/writing_guide/writing/flesch.shtml]
Wallace LS, Roskos SE, Weiss BD: Readability characteristics of consumer medication information for asthma inhalation devices. J Asthma. 2006, 43: 375-378. 10.1080/02770900600709856.
Mayxay M, Newton PN, Yeung S, Pongvongsa T, Phompida S, Phetsouvanh R, White NJ: Short communication: An assessment of the use of malaria rapid tests by village health volunteers in rural Laos. Trop Med Int Health. 2004, 9: 325-329. 10.1111/j.1365-3156.2004.01199.x.
Arnold CL, Davis TC, Frempong JO, Humiston SG, Bocchini A, Kennen EM, Lloyd-Puryear M: Assessment of newborn screening parent education materials. Pediatrics. 2006, 117: S320-S325. 10.1542/peds.2005-1934.
White LJ, Jones JS, Felton CW, Pool LC: Informed consent for medical research: common discrepancies and readability. Acad Emerg Med. 1996, 3: 745-750. 10.1111/j.1553-2712.1996.tb03509.x.
Wallace LS, Keenum AJ: Using a home blood pressure monitor: do accompanying instructional materials meet low literacy guidelines?. Blood Press Monit. 2008, 13: 219-223. 10.1097/MBP.0b013e3283057b0a.
Wallace LS, Keenum AJ, DeVoe JE: Evaluation of consumer medical information and oral liquid measuring devices accompanying pediatric prescriptions. Acad Pediatr. 2010, 10: 224-227. 10.1016/j.acap.2010.04.001.
Hashizume M, Kondo H, Murakami T, Kodama M, Nakahara S, Lucas ME, Wakai S: Use of rapid diagnostic tests for malaria in an emergency situation after the flood disaster in Mozambique. Public Health. 2006, 120: 444-447. 10.1016/j.puhe.2005.11.007.
Hawkes M, Katsuva JP, Masumbuko CK: Use and limitations of malaria rapid diagnostic testing by community health workers in war-torn Democratic Republic of Congo. Malar J. 2009, 8: 308-10.1186/1475-2875-8-308.
Doak LG, Doak CC, Meade CD: Strategies to improve cancer education materials. Oncol Nurs Forum. 1996, 23: 1305-1312.
Kaphingst KA, DeJong W: The educational potential of direct-to-consumer prescription drug advertising. Health Aff (Millwood ). 2004, 23: 143-150. 10.1377/hlthaff.23.4.143.
Grobusch MP, Hanscheid T, Zoller T, Jelinek T, Burchard GD: Rapid immunochromatographic malarial antigen detection unreliable for detecting Plasmodium malariae and Plasmodium ovale. Eur J Clin Microbiol Infect Dis. 2002, 21: 818-820. 10.1007/s10096-002-0831-0.
Maltha J, Gillet P, Bottieau E, Cnops L, Van Esbroeck M, Jacobs J: Evaluation of a rapid diagnostic test (CareStart Malaria HRP-2/pLDH (Pf/pan) Combo Test) for the diagnosis of malaria in a reference setting. Malar J. 2010, 9: 171-10.1186/1475-2875-9-171.
Van der Palen M, Gillet P, Bottieau E, Cnops L, Van Esbroeck M, Jacobs J: Test characteristics of two rapid antigen detection tests (SD FK50 and SD FK60) for the diagnosis of malaria in returned travellers. Malar J. 2009, 8: 90-10.1186/1475-2875-8-90.
van Dijk DP, Gillet P, Vlieghe E, Cnops L, Van Esbroeck M, Jacobs J: Evaluation of the Palutop+4 malaria rapid diagnostic test in a non-endemic setting. Malar J. 2009, 8: 293-10.1186/1475-2875-8-293.
van Dijk DP, Gillet P, Vlieghe E, Cnops L, Van Esbroeck M, Jacobs J: Evaluation of the Immunoquick+4 malaria rapid diagnostic test in a non-endemic setting. Eur J Clin Microbiol Infect Dis. 2010, 29: 577-583. 10.1007/s10096-010-0898-y.
World Health Organization: Overview of the prequalification of diagnostics assessment process. 2010, [http://www.who.int/entity/diagnostics_laboratory/evaluations/pqdx_007_pq_overview_document_v3.pdf]
World Health Organization: Malaria Rapid Diagnostic Test Performance; Results of WHO product testing of malaria RDTs: Round 1 (2008). 2009, [http://www.finddiagnostics.org/resource-centre/reports_brochures/malaria-diagnostics-report-2009.html]
World Health Organization: Malaria Rapid Diagnostic Test Performance; Results of WHO product testing of malaria RDTs: Round 2 (2009). 2010, [http://www.finddiagnostics.org/resource-centre/reports_brochures/malaria-diagnostic-test-report-round2.html]
European Commission: European standards in vitro diagnostic medical devices. 1998, [http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:1998:331:0001:0037:EN:PDF]
Taylor HA: Barriers to informed consent. Semin Oncol Nurs. 1999, 15: 89-95. 10.1016/S0749-2081(99)80066-7.
We would like to thank Peter Verbruggen, Sint Lucas Antwerpen, Karel de Grote-Hogeschool, Antwerp, Belgium for advices about assessing typography and Dionicia Gamboa, Institute of Tropical Medicine "Alexander Von Humboldt", Lima, Peru for fruitful discussions and suggestions.
The authors declare that they have no competing interests.
PG and JM equally contributed to the present study and share first authorship. PG JM and JJ designed the study protocol, RR made substantial contributions to the concept and design of the study. PG, JM and VH carried out the test evaluations. PG, JM, CB and JJ analysed and interpret the results and drafted the manuscript. All authors critically reviewed the manuscript and approved the final manuscript.