RESEARCH
External quality assessment of national public health laboratories in Africa, 2002-2009
Évaluation externe de la qualité des laboratoires de santé publique nationaux en Afrique, 2002-2009
Evaluación externa de la calidad de los laboratorios nacionales de la sanidad pública en África, 2002-2009
John FreanI,*; Olga PerovicI; Vivian FenshamI; Kerrigan McCarthyI; Anne von GottbergI; Linda de GouveiaI; Bhavani PoonsamyI; Leigh DiniI; Jenny RossouwI; Karen KeddyI; Wondimagegnehu AlemuII; Ali YahayaII; Antoine PiersonIII; Virginie DolmazonIII; Sébastien CognatIII; Jean Bosco NdihokubwayoII
INational Institute for Communicable Diseases, National Health Laboratory Service, P/Bag X4, Sandringham, Johannesburg, 2131, South Africa
IIRegional Office for Africa, World Health Organization, Brazzaville, Congo
IIIGlobal Capacity, Alert and Response, World Health Organization, Lyon, France
ABSTRACT
OBJECTIVE: To describe findings from an external quality assessment programme involving laboratories in Africa that routinely investigate epidemic-prone diseases.
METHODS: Beginning in 2002, the Regional Office for Africa of the World Health Organization (WHO) invited national public health laboratories and related facilities in Africa to participate in the programme. Three surveys comprising specimens and questionnaires associated with bacterial enteric diseases, bacterial meningitis, plague, tuberculosis and malaria were sent annually to test participants' diagnostic proficiency. Identical surveys were sent to referee laboratories for quality control. Materials were prepared, packaged and shipped in accordance with standard protocols. Findings and reports were due within 30 days. Key methodological decisions and test results were categorized as acceptable or unacceptable on the basis of consensus feedback from referees, using established grading schemes.
FINDINGS: Between 2002 and 2009, participation increased from 30 to 48 Member States of the WHO and from 39 to 78 laboratories. Each survey was returned by 64-93% of participants. Mean turnaround time was 25.9 days. For bacterial enteric diseases and meningitis components, bacterial identification was acceptable in 65% and 69% of challenges, respectively, but serotyping and antibiotic susceptibility testing and reporting were frequently unacceptable. Microscopy was acceptable for 73% of plague challenges. Tuberculosis microscopy was satisfactorily performed, with 87% of responses receiving acceptable scores. In the malaria component, 82% of responses received acceptable scores for species identification but only 51% of parasite quantitation scores were acceptable.
CONCLUSION: The external quality assessment programme consistently identified certain functional deficiencies requiring strengthening that were present in African public health microbiology laboratories.
RÉSUMÉ
OBJECTIF: Décrire les résultats d'un programme d'évaluation externe de la qualité impliquant des laboratoires africains étudiant de matière systématique les maladies à potentiel épidémique.
MÉTHODES: Dès 2002, le Bureau régional pour l'Afrique de l'Organisation mondiale de la Santé (OMS) a invité les laboratoires de santé publique nationaux et des installations connexes en Afrique à participer au programme. Trois études comprenant des échantillons et des questionnaires associés aux maladies entériques bactériennes, à la méningite bactérienne, à la peste, à la tuberculose et au paludisme ont été envoyés chaque année pour tester la qualité du diagnostic des participants. Des études identiques ont été envoyées à des laboratoires arbitres pour le contrôle de la qualité. Les matériaux ont été préparés, emballés et expédiés conformément aux protocoles standard. Les résultats et les rapports devaient parvenir dans les 30 jours. Les principales décisions méthodologiques et les résultats des tests ont été qualifiés d'acceptables ou d'inacceptables, sur la base des commentaires consensuels des arbitres, utilisant des schémas de notation établis.
RÉSULTATS: Entre 2002 et 2009, la participation est passée de 30 à 48 États membres de l'OMS, et de 39 à 78 laboratoires. Chaque étude a été renvoyée par 64 à 93% des participants. Le délai moyen était de 25,9 jours. Pour les maladies entériques bactériennes et les composants de la méningite, l'identification bactérienne était acceptable dans 65% et 69% des tests, respectivement, mais le sérotypage et les tests de sensibilité aux antibiotiques, ainsi que les rapports, étaient souvent inacceptables. La microscopie était acceptable dans 73% des tests de peste. La microscopie de la tuberculose était effectuée de manière satisfaisante, 87% des réponses obtenant des scores acceptables. Dans la composante paludisme, 82% des réponses ont obtenu des scores acceptables pour l'identification des espèces, mais seuls 51% des scores de quantification des parasites étaient acceptables.
CONCLUSION: Le programme externe d'évaluation de la qualité a identifié de manière constante certaines déficiences fonctionnelles nécessitant une amélioration, dans les laboratoires de microbiologie du système de santé publique africain.
RESUMEN
OBJETIVO: Describir los resultados de un programa de evaluación externa de la calidad de los laboratorios africanos que investigan habitualmente las enfermedades con tendencia a provocar epidemias.
MÉTODOS: En el año 2002, la Oficina Regional para África de la Organización Mundial de la Salud (OMS) invitó a los laboratorios nacionales de la sanidad pública así como a los centros relacionados de África a participar en el programa. Con el fin de comprobar la capacidad de diagnóstico de los participantes, se les enviaron anualmente tres encuestas con muestras y cuestionarios relacionados con enfermedades enterobacterianas, meningitis bacteriana, peste, tuberculosis y malaria. Con el fin de realizar un control de calidad, se enviaron las mismas encuestas a los laboratorios evaluadores. Los materiales se prepararon, empaquetaron y enviaron de acuerdo con los protocolos estándar. Los resultados y las encuestas debían estar listos en un plazo de 30 días. Las principales decisiones metodológicas y los resultados de las pruebas se clasificaron como aceptables o inaceptables en base a la respuesta consensuada de los evaluadores por medio de los programas de calificación establecidos.
RESULTADOS: La participación aumentó desde 30 hasta 48 Estados Miembros de la OMS y desde 39 hasta 78 laboratorios entre los años 2002 y 2009. Entre un 64 y un 93% de los participantes remitieron cada una de las encuestas y el plazo medio de entrega fue de 25,9 días. En los casos de enfermedades enterobacterianas y de meningitis, la identificación bacteriana fue aceptable en un 65% y un 69% de los casos, respectivamente, si bien las pruebas y los resultados de serotipado y de sensibilidad a los antibióticos fueron, con frecuencia, inaceptables. La microscopía resultó aceptable en el 73% de las pruebas de peste. La microscopía de tuberculosis se realizó de manera satisfactoria: un 87% de las respuestas obtuvo resultados aceptables. En el componente de la malaria, un 82% de las respuestas obtuvieron puntuaciones aceptables en cuanto a la identificación de la especie, pero solo el 51% de las puntuaciones para la cuantificación de parásitos resultó aceptable.
CONCLUSIÓN: El programa de evaluación externa de la calidad identificó de manera sistemática algunas deficiencias funcionales que
Introduction
In Africa, communicable diseases constitute a very important public health problem and outbreaks pose serious threats to health.1 These diseases exert enormous influence on the economy and society and should be targeted through aggressive disease prevention, control and surveillance activities. However, successful performance of these activities requires reliable epidemiologic data and generating such data is one of the roles of national public health laboratories.2 This awareness led the Regional Committee for Africa of the World Health Organization (WHO) to recommend strengthening the ability of national public health laboratories to accurately, reliably and promptly confirm epidemics in an effort to improve the public health response and enhance national and global health security. This was the intention behind its adoption of the strategy for integrated disease surveillance and response in 19983 and of Resolution AFR/RC58/R2 in 2008. 3-5 The integrated disease surveillance and response technical guidelines recommend the use of standard laboratory diagnostic methods for priority diseases in the WHO African Region.6
In many African countries, reliable confirmation of suspected infectious diseases is hampered by a lack of standardized diagnostic methods and by a shortage of funds, staff and laboratory supplies for national public health laboratories, despite the critical role played by these laboratories as part of a functional infrastructure for disease surveillance.7 Development and maintenance of high-quality national public health laboratory services require financial and managerial commitment to provide qualified staff, training, equipment, consumables, reagents and physical facilities. Periodic quality assessment of performance is also essential because it can help ensure the reliability of findings and is an important component of laboratory accreditation, towards which national public health laboratories should, ideally, strive.
In July 2002, WHO launched an external quality assessment programme (EQAP) to test the proficiency of microbiological testing for epidemic-prone diseases by laboratories in the African Region.8 The EQAP was extended to laboratories in three African Member States in the Region of the Eastern Mediterranean during 2005. The National Institute for Communicable Diseases (NICD), a division of the National Health Laboratory Service of South Africa, provided technical coordination following an agreement with the co-funders, the WHO Regional Office for Africa and the WHO office in Lyon, France. 8-10 Accordingly, the NICD undertook to provide specimens for laboratory identification of selected agents of bacterial enteric diseases, bacterial meningitis and plague; to advise WHO about the needs of participating laboratories, to correct deficiencies and maintain proficiency; and to further extend the EQAP to include general bacteriological analyses and tuberculosis and malaria microscopy.11 This article reviews findings from microbiology EQAP surveys conducted by WHO and NICD in Africa during 2002-2009.
Methods
Participating laboratories
The WHO Regional Office for Africa invited national public health laboratories (nominated by their ministries of health) and/or the main hospital or research laboratories functioning as such, as well as laboratories in the Paediatric Bacterial Meningitis Surveillance Network,12 to participate in the bacterial enteric diseases and bacterial meningitis components of the EQAP (Table 1, available at: http://www.who.int/bulwletin/volumes/90/3/11-091876). Some laboratories were both national public health laboratories and in the Paediatric Bacterial Meningitis Surveillance Network.12 Certain laboratories were also recruited to participate in the plague, tuberculosis and malaria components of the EQAP because of their expertise in these areas.
Coordination and technical guidance
The NICD Technical Implementation Group, together with representatives from WHO's African Regional Office and the WHO's Lyon Office, coordinated the EQAP. The EQAP Regional Advisory Group, consisting of the Technical Implementation Group, WHO partners, additional technical consultants and invited experts, met annually to evaluate progress and plan future activities. Technical input provided by the Technical Implementation Group about EQAP specimens included in-house production, quality control and validation of survey materials and evaluation of findings and reports. Eleven international expert laboratories (including some WHO Collaborating Centres) served as external technical advisers or referees.8 These laboratories processed EQAP specimens blindly and provided comments about their quality. Additional technical details of the programme and information about its management are comprehensively described elsewhere.8
Survey contents and distribution
Initial surveys evaluated laboratory proficiency in bacterial enteric diseases, bacterial meningitis and plague components; tuberculosis and malaria components were added in 2005. The range of organisms sent across surveys is shown in Table 2. Current regulations strictly limit or prohibit transfer of Shigella dysenteriae type 1 and Yersinia pestis cultures,13 precluding their use in the EQAP.
Surveys containing bacterial enteric diseases, bacterial meningitis and plague components generally included two challenge specimens per disease component. The NICD used techniques developed by the Clinical Microbiology Proficiency Testing programme at the University of British Columbia 8, 14 in Vancouver, Canada, to produce clinically relevant simulated preparations (e.g. cerebrospinal fluid smears for Gram staining) and simulated biological fluids (e.g. cerebrospinal fluid, stool and pus) inoculated with appropriate bacteria, including some general pathogens and potential contaminants. Trans-isolate medium,15 which is capable of sustaining small numbers of the common bacterial meningitis pathogens in cerebrospinal fluid during transport, was used in the EQAP and generally performed well. When organism viability in simulated specimens proved to be limited (as in the case of Vibrio cholerae), lyophilized cultures were substituted. Specimens for the plague component included culturable bacteria other than Y. pestis, blood smears containing Y. pestis (inactivated by fixing) for staining and detection of typical bipolar-stained bacilli, simulated plague bubo aspirates for detection of fraction 1 (F1) antigen by means of dipstick-format rapid tests (Institut Pasteur, Antananarivo, Madagascar), and serum for antibody assays. F1 assays were distributed to participating laboratories via the EQAP.
The tuberculosis component included 8 challenge specimens - 4 Ziehl-Neelsen-stained slides and 4 unstained slides - for microscopy. Slides contained smears prepared from pooled clinical sputum specimens in accordance with protocols of the Association of Public Health Laboratories, headquartered in the United States of America.16 Participants were required to stain unstained slides for detection and quantitation of mycobacteria. Laboratories reported on the presence of acid-fast bacilli using the International Union for Tuberculosis and Lung Disease smear-grading system. 8, 17
For the malaria component, 10 challenges, each consisting of Giemsa-stained thin and thick blood films for identification of parasite species and quantitation of parasite density, respectively, were included. Films were prepared using standard procedures developed for the EQAP.8 Identical challenges were included within and across surveys to test for intra-and interrater reliability.
Survey materials were fully characterized by the NICD before shipment (for bacteria, this included biochemical identification, serotyping or serogrouping and antimicrobial susceptibility profiles).14 Bacterial strains included those routinely supplied by the NICD as controls to South Africa's national laboratory system, with the intention that EQAP participants would retain them for that purpose. Each survey included written information that described clinically relevant case scenarios for challenges, as well as forms for participants to record activities performed before and after testing. Participants were prompted for their decisions about testing algorithms, interpretation of test results and reporting of findings. A training exercise was sent with the second survey in 2004. It explained the protocols for antimicrobial susceptibility testing of pneumococci of the Clinical Laboratory Standards Institute,18 the Société Française de Microbiologie19 and the British Society for Antimicrobial Agents and Chemotherapy.20
All EQAP documents were prepared in English and French and most were also prepared in Portuguese. They included survey information; report forms; individual results and graded assessment of performance; a commentary on overall performance, including statistical evaluation; assessment criteria, and technical suggestions for improvements.
Three surveys were sent per year. All were packaged according to International Air Transport Association Packing Instruction 6508 and were shipped from Johannesburg, South Africa, to participants and referees simultaneously by express air courier.13 Participating laboratories were requested to return results within 30 days of survey dispatch but some leeway was allowed because of unpredictable delivery times.
Evaluation of responses
Areas of the survey involving critical decision points and tests were assigned grades that accorded with predetermined schemes. The Technical Implementation Group evaluated referee laboratory responses for consensus results before determining evaluation criteria within each grading area.
The grading scheme for the bacterial enteric diseases, bacterial meningitis and plague components was adopted from the Clinical Microbiology Proficiency Testing programme. 8,14 This programme uses a discontinuous scale that assesses the response in terms of its clinical influence on a hypothetical patient. The best answer receives a score of 4, answers that are incorrect but have no or little clinical impact receive a 3, answers that are incorrect and could lead to a minor diagnosis or treatment error receive a 1 and answers that are incorrect and could lead to a major diagnosis or treatment error receive a 0. Responses were graded in several technical areas, such as microscopy, culture and identification (including antigen test results), serotyping, choice of antibiotics for antimicrobial susceptibility testing and reporting of susceptibility. Grading areas depended on the particular challenge. For example, microscopy performance (i.e. Gram staining and examination of smears) was usually assessed for the bacterial meningitis and plague components but not for the bacterial enteric diseases component, whereas rapid antigen test performance was only graded for the bacterial meningitis and plague components.
Assessment of microscopy in the tuberculosis component was adapted from the Association of Public Health Laboratories guidelines for external quality assessment.16 In this grading scheme, correct responses and minor quantitation errors receive 10 points, responses with minor misclassification errors receive 5 points and responses with major misclassification errors (i.e. high false positives or high false negatives) receive 0 points.
For the malaria component, species identification was graded using the scheme developed by the Clinical Microbiology Proficiency Testing programme, as described above. Quantitation of malaria parasite density was assessed with regard to consensus counts, calculated as the median of combined counts from participants, referees and NICD. Counts within the target range, defined as 0.5 to 1.5 times the consensus count, were considered acceptable.21
Data analysis
Responses and scores were entered and stored on a customised Access database (Microsoft, Redmond, USA) and analysed on a Microsoft Excel spreadsheet. For bacterial meningitis, bacterial enteric diseases, plague and malaria microscopy, scores of 3 or 4 were acceptable; scores of 1 or 0 were unacceptable. For tuberculosis microscopy, scores of 5 or 10 were acceptable and a score of 0 was unacceptable. Acceptable quantitation of malaria parasite density was defined as a count within the target range. For all periods (i.e. one year or an entire review period), the proportion of acceptable responses in a component and grading area challenge was expressed as a percentage of the number of eligible responses received. Assessment of overall or cumulative performance in a component was performed by grouping all grading area responses together and calculating the proportion with acceptable scores.
Results
As shown in Table 1, 39 laboratories in 30 of 46 Member States in the African Region were enrolled in 2002. In 2005, laboratories from three Member States in the Eastern Mediterranean Region were added; by the end of the year, 73 laboratories had been recruited, including those with the ability to identify agents of tuberculosis and malaria. By the end of 2009, 48 African Member States were involved, with 76 laboratories enrolled in the bacterial enteric diseases and bacterial meningitis components, 70 in the malaria component, 68 in the tuberculosis component and 17 in the plague component. All laboratories evaluated challenge specimens associated with 2 to 4 disease components per survey. The mean response time across all surveys, which reflects transport and bureaucratic delays and specimen processing time, was 25.9 days (standard deviation: 6.2 days; range: 9 54 days). As is generally the case for proficiency testing schemes, the timeline for returning results was not intended to reflect standards for routine clinical sample turnaround times, given the logistical constraints involved. Overall performance in each disease component, combining all grading areas, is summarized in Fig. 1.
Bacterial enteric diseases
Between 63% and 91% of surveyed laboratories responded. Fig. 2 summarizes the percentages of acceptable responses in four grading areas. Cumulative acceptable performance was 65%. Although 64% of responses involving decisions about antibiotic selection for antimicrobial susceptibility testing were acceptable, 43% involving test reporting were unacceptable. Only a minority of laboratories had the capacity to serotype Salmonella and Shigella species.
Bacterial meningitis
A total of 63-93% of laboratories responded per survey. Cumulative acceptable performance was 52%; performance in five grading areas is presented in Fig. 3. Microscopy findings were reasonable but erratic (62-88% of responses were acceptable). Problems with antimicrobial susceptibility testing of pneumococci were identified early in the programme and subsequent surveys included identified strains of pneumococci as susceptibility testing challenges only. Antibiotic choice for antimicrobial susceptibility testing was poor (54% of responses were acceptable) and reporting of corresponding findings was very poor (only 25% of responses were acceptable). Few laboratories performed minimum inhibitory concentration testing.
Plague
Participation per survey ranged from 64% to 90% of laboratories. Cumulative acceptable performance was 66%. Performance in three grading areas is specified in Fig. 4. Microscopy findings for bipolar Gram-stained bacteria were good (77-100% were acceptable), as were the findings for F1 antigen dipstick tests (76% were acceptable). However, use of biochemical tests, including API® strips (Biomérieux, Durham, USA), for identification of Gram-negative organisms generally yielded unacceptable scores and Gram staining of routinely isolated bacteria was erratic.
Tuberculosis
Responses were returned by 72-90% of laboratories per survey. The proportion of acceptable responses is shown in Fig. 5. Laboratories generally performed well, with 87% of responses considered acceptable. Although evaluation of prestained slides was good, participant-stained slides resulted in a greater proportion of false-negative results.
Malaria
Between 63% and 83% of surveyed laboratories responded. Results in two grading areas are shown in Fig. 6. Acceptable scores for species identification were given to 82% of responses. Whereas proportions of false positive and false-negative results were low, correct identification of blood parasites other than Plasmodium falciparum (the predominant agent of malaria in Africa) was limited (58% of responses were acceptable). Quantitation of malaria parasites was poor, with only 51% of responses considered acceptable.
Discussion
Several inadequacies have been identified in medical laboratory services in Africa.22 Deficiencies in areas such as consumables, basic equipment, skilled personnel, training programmes, logistical support, national standards and quality assessment have been emphasized. While our programme was limited to the assessment of national public health laboratory proficiency in relation to a preselected range of pathogens of public health importance, we found that many of these problems were present in participating laboratories. Our findings suggest that shortage or lack of appropriate reagents was immediately responsible for poor performance in some programme grading areas. For example, a lack of agglutinating antisera was frequently a primary reason for failure to fully characterize bacterial isolates. Failure to correctly perform antimicrobial susceptibility testing, including inappropriate choice of antibiotics for testing, absence of quality control for both media and antibiotic disks and failure to adhere to internationally recognized guidelines,18-20 were also among shortcomings demonstrated by some laboratories. More generally, problems with quality control of media and reagents, including lack of appropriate control organisms, and insufficient training in organism identification and quantitation were evident.
Published external quality assessment activities in Africa have generally been limited in duration, geographic footprint and scope of assessment. Multicountry programmes for quality assessment of automated quantitation of CD4+ T-cells23 and for malaria microscopy in a clinical vaccine trial24 are examples of more extensive efforts. The EQAP is unique in its geographic range (all sub-Saharan and some northern African countries are included), scope of assessed activities and sustainability, demonstrated by 10 years of uninterrupted activity as of 2011. As a direct result of the EQAP, national diagnostic capabilities for the five targeted epidemic-prone diseases include regular proficiency testing and access to training resources in the form of reference material (e.g. characterized bacterial control strains, malaria and tuberculosis control slides and copies of international laboratory standards and guidelines), commentaries, technical advice, bench aids and occasionally scarce reagents, such as rapid plague F1 assays. The multilateral scheme has provided a better understanding of the performance of participating laboratories, identifying their strengths, limitations and continuing challenges. The NICD's capacity to produce, perform quality control of and distribute samples for the EQAP has been robust. This programme has allowed WHO to identify the capacities and resources needed by countries for sustainable improvement in the competency of participating laboratories.
This international EQAP provides a unique opportunity for African national public health laboratories to continuously assess and improve their performance in microbiological analysis. It is a powerful channel for educating laboratory personnel and for standardizing procedures. Objective data yielded by the programme illustrate the strengths and weaknesses of participating laboratories and constitute an evidence-based tool for guiding and improving laboratory-strengthening activities for WHO in the area. Given the considerable challenges of securing resources to procure and maintain stock of the much needed reagents, transport media and other necessary laboratory supplies, EQAP findings should be used to advocate for mobilization of additional resources at the local level to improve laboratory capacity. This should include budgeting for supplies and communication activities to strengthen the linkage between national public health laboratories and other public health services. The results should also be used to advocate for additional investment in all aspects of health laboratory services. The programme has served as an example for national and regional external quality assessment; several countries in Africa and in WHO's Region of the Eastern Mediterranean are planning, or have implemented, national or regional external quality assessment schemes that are based on this model.8
Notwithstanding its value as a tool for advocacy, external quality assessment alone does not invariably improve laboratory performance. The lack of a general upward trend in EQAP performance over time across all disease components and grading areas (Fig. 1) corroborates this. Identified deficiencies need to be actively corrected for there to be sustainable improvement. Although several EQAP-associated training visits were provided by NICD in partnership with WHO's Regional Office for Africa, implementation of corrective actions is primarily the responsibility of laboratory personnel and management and is verified through an accreditation process. Only 340 diagnostic laboratories in Africa are accredited, with 312 (92%) in South Africa.25 The commitment to strengthening African NATIONAL PUBLIC HEALTH LABORATORIEs2-5 led directly to the development of the WHO-African Region stepwise approach to accreditation.25 External quality assessment is among the 12 areas of laboratory quality evaluated by this approach, making participation in a proficiency testing programme such as the EQAP an indispensable part of its implementation.
Acknowledgements
We thank L Arntzen, C Bopp, R van Deventer, P Hance, H Haritos, R Landsberg, T Mahlanga, L Mayer, P Nicolas, L Rahalison, A Sooka and the staff at referee laboratories, for their highly valued contributions to the programme.
References
1. The African regional health report 2006: the health of the people. Brazzaville: World Health Organization Regional Office for Africa; 2006. Available from: http://whqlibdoc.who.int/afro/2006/9290231033_rev_eng.pdf [accessed 1 December 2011] .
2. Koplan JP, Puska P, Jousilahti P, Cahill K, Huttunen J; National Public Health Institute Partners. Improving the world's health through national public health institutes. Bull World Health Organ 2005;83:154-7. PMID:15744409
3. Integrated epidemiological surveillance of diseases: regional strategy for communicable diseases. In: 48th session of the WHO Regional Committee for Africa. Harare: World Health Organization Regional Office for Africa; 1998 (AFR/RC48/15). Available from: http://afrolib.afro.who.int/RC/en/AFR_RC48.pdf [accessed 1 December 2011] .
4. Strengthening public health laboratories in the WHO African region: a critical need for disease control. Yaounde: World Health Organization Regional Office for Africa; 2008. Available from: http://ahm.afro.who.int/issue12/pdf/AHM12Pages47to52.pdf [accessed 1 December 2011] .
5. International Health Regulations. Geneva: World Health Organization; 2005.
6. Technical guidelines for integrated disease surveillance and response in the African region. Harare & Atlanta: World Health Organization Regional Office for Africa & Centers for Disease Control and Prevention; 2001. Available from: http://www.cdc.gov/idsr/focus/surv_sys_strengthening/tech_guidelines-integrated-diseaseENG.pdf [accessed 1 December 2011] .
7. Manual of basic techniques for a health laboratory. 2nd ed. Geneva: World Health Organization; 2003.
8. Policy and procedures of the WHO/NICD Microbiology External Quality Assessment Programme in Africa (WHO/CDS/EPR/LYO/2007.3). Geneva: World Health Organization; 2007.
9. Requirements and guidance for external quality assessment scheme for health laboratories. Geneva: World Health Organization; 1999 (WHO/DIL/LAB/99).
10. Inhorn SL. Quality assurance practices for health laboratories. Washington: American Public Health Association; 1978.
11. Guide for national public health laboratory networking to strengthen integrated disease surveillance and response. Brazzaville & Atlanta: World Health Organization Regional Office for Africa & Centers for Disease Control and Prevention; 2008.
12. Centers for Disease Control and Prevention. Pediatric bacterial meningitis surveillance-African region, 2002-2008. MMWR Morb Mortal Wkly Rep 2009;58:493-7. PMID:19444153
13. Guidance on regulations for the transport of infectious substances, 2007-2008 (WHO/CDS/EPR/2007.2). Geneva: World Health Organization; 2007.
14. Clinical Microbiology Proficiency Testing [Internet]. Vancouver: University of British Columbia. Available from: http://www.cmpt.ca [accessed 1 December 2011] .
15. Ajello GW, Feeley JC, Hayes PS, Reingold AL, Bolan G, Broome CV et al. Trans-isolate medium: a new medium for primary culturing and transport of Neisseria meningitidis, Streptococcus pneumoniae, and Haemophilus influenzae. J Clin Microbiol 1984;20:55-8. PMID:6430956
16. External quality assessment for AFB smear microscopy. Silver Spring: Association of Public Health Laboratories; 2011. Available from: http://www.aphl.org/AboutAPHL/publications/Documents/External_Quality_Assessment_for_AFB_Smear_Microscopy.pdf [accessed 1 December 2011] .
17. Priorities for tuberculosis bacteriology services in low-income countries. 2nd ed. Paris: International Union Against Tuberculosis and Lung Disease; 2007.
18. Performance standards for antimicrobial susceptibility testing. 19th Informational Supplement M100-S19. Wayne: Clinical and Laboratory Standards Institute; 2009.
19. Comité de l'Antibiogramme de la Société Française de Microbiologie. Recommandations 2008. Paris: Société Française de Microbiologie; 2008. Available from: http://www.sfm-microbiologie.org/UserFiles/file/CASFM/casfm_2008.pdf [accessed 6 December 2011] .
20. BSAC methods for antimicrobial susceptibility testing, version 8. Birmingham: British Society for Antimicrobial Chemotherapy; 2008. Available from: http://www.bsac.org.uk [accessed 6 December 2011] .
21. Malaria microscopy quality assurance manual, version 1. Geneva: World Health Organization; 2009.
22. Petti CA, Polage CR, Quinn TC, Ronald AR, Sande MA. Laboratory medicine in Africa: a barrier to effective health care. Clin Infect Dis 2006;42:377-82. doi:10.1086/499363 PMID:16392084
23. Glencross DK, Aggett HM, Stevens WS, Mandy F. African regional external quality assessment for CD4 T-cell enumeration: development, outcomes, and performance of laboratories. Cytometry B Clin Cytom 2008;74(Suppl 1):S69-79. PMID:18228560
24. Swysen C, Vekemans J, Bruls M, Oyakhirome S, Drakeley C, Kremsner P et al. Development of standardized laboratory methods and quality processes for a phase III study of the RTS,S/AS01 candidate malaria vaccine. Malar J 2011;10:223. doi:10.1186/1475-2875-10-223 PMID:21816032
25. Gershy-Damet GM, Rotz P, Cross D, Belabbes el H, Cham F, Ndihokubwayo JB et al. The World Health Organization African region laboratory accreditation process: improving the quality of laboratory systems in the African region. Am J Clin Pathol 2010;134:393-400. doi:10.1309/AJCPTUUC2V1WJQBM PMID:20716795
Submitted: 10 June 2011
Revised version received: 28 October 2011
Accepted: 9 November 2011
Published online: 20 January 2012
Funding: This study was funded by the United States Agency for International Development; the Global Alliance for Vaccine Initiative, through the WHO Paediatric Bacterial Meningitis Surveillance Programme; and the Government of the Netherlands.
Competing interests: None declared.
* Correspondence to John Frean (e-mail: johnf@nicd.ac.za).