ABSTRACT

Objective

To assess a LED-fluorescence microscopy (LED-FM) capacitation program for the training of laboratory technicians without previous experience in FM.

Methods

We evaluated a teaching program that consists of a three-day course followed by an "in situ" two-month phase in which technicians acquired skills without the help of a FM expert; in order to gain confidence to recognize auramine-stained bacillus, during this phase, technicians examined duplicate slides stained by Ziehl Neelsen (ZN) and FM in a unblinded way. Technicians with acceptable performance, continued with a blinded-training period. Testing panels and rechecking process were used to evaluate proficiency after different length of experience.

Results

Post-course panel results showed that 70% of trainees made Low False Positive errors (LFPs). Analysis of two other panels showed that LFPs significantly decreased (Chi-squared test, p<0.05) as the "in situ" training phase progressed. Processing at least three slides/day was associated with acceptable performance. During the blinded-training period, results of the rechecking process showed that sensitivity (96.8%) and specificity (99.8%) levels were satisfactory.

Conclusion

Moderate training (a three-day course) is not enough to make technicians proficient in LED-FM; however, great ability can be reached after a short "in situ" training phase even without the presence of experienced staff available in field to review doubtful results. Training was more effective in services with a minimum workload of 750 slides/year.

Key Words:
Tuberculosis; teaching; fluorescence (source: MeSH, NLM)

RESUMEN

Objetivo

Evaluar un programa de capacitación en microscopía de fluorescencia LED (MF-LED) para el entrenamiento de técnicos de laboratorio sin experiencia en MF.

Métodos

Se evaluó un programa de capacitación que consiste en un curso de tres días seguido de dos meses de entrenamiento «in situ», en donde los técnicos adquirieron habilidades sin presencia de un experto en la práctica diaria; para alcanzar confianza en el reconocimiento del bacilo, los técnicos, durante estos meses, examinaron en forma «no cegada» extendidos duplicados teñidos por Ziehl Neelsen (ZN) y MF. Aquellos laboratoristas que lograron rendimiento aceptable continuaron su entrenamiento «a ciegas». Su desempeño fue evaluado en distintos períodos del entrenamiento mediante paneles de láminas y relectura de extendidos.

Resultados

Los resultados de un panel posterior al curso mostraron que 70% de los participantes cometieron errores falsos positivos bajos (FPB). Dos paneles posteriores evidenciaron que los FPB disminuían significativamente (prueba de Chi cuadrado, p<0.05) a medida que el entrenamiento avanzaba. El procesamiento de al menos tres extendidos/ día se asoció con desempeño aceptable. Durante el período a ciegas, la relectura de láminas evidenció que la sensibilidad (96,8%) y especificidad (99,8%) fueron satisfactorias.

Conclusiones

Una capacitación moderada (curso de tres días) no es suficiente para adquirir competencia en MF-LED; sin embargo, se puede alcanzar habilidad después de una corta capacitación «in situ», incluso si no hay personal con experiencia disponible en el servicio para revisar los resultados dudosos diariamente. El entrenamiento fue más efectivo en servicios con carga de trabajo mínima de 750 extendidos/año.

Palabras Clave:
Tuberculosis; capacitación; fluorescencia (fuente DeCS, BIREME)

In comparison to Ziehl Neelsen microscopy (ZN), conventional fluorescence microscopy (FM), using auramine-O staining, can detect approximately 5-10% more Acid Fast Bacilli (AFB) positive smears ¹1. Steingart KR, Henry M, Ng V, Hopewell PC. Fluorescence versus conventional sputum smear microscopy for tuberculosis: a systematic review. Lancet Infect Dis. 2006; 6(10): 570-81., but its use has been limited to the requirement for expensive mercury vapor lamps and dark room facilities. The good performance of the new light emitting diode (LED) technology, which is inexpensive and employs a long lifespan lamp, has led the World Health Organization (WHO) to recommend the use of LED-FM as an alternative of ZN in a phased manner ²2. WHO. Fluorescent Light-Emitting Diode (LED) Microscopy for Diagnosis of Tuberculosis: Policy Statement. Geneva: World Health Organization; 2011.WHO/HTM/TB/2011.8..

Nevertheless, care must be taken because it has been observed that less experienced technicians are likely to commit false positive errors (FP) ³3. Toman K. What are the advantages and disadvantages of fluorescence microscopy? In: Frieden T, ed. Toman's tuberculosis: case detection, treatment and monitoring-questions and answers. 2nd Edition. Geneva: WHO. 2004; 31-4. as impurities and artifacts may be mistaken with AFB. These observations highlight the importance of the initial training before LED-FM implementation. Although there is an impression that the acquisition of skills is harder than that for ZN, there is poor knowledge about proper training in LED-FM. It has been suggested that an ideal initial training should be provided by staff experienced in using LED-FM in everyday practice ⁴4. Cuevas LE, Al-Sonboli N, Lawson L, Yassin MA, Arbide I, Al-Aghbari N, et al. LED fluorescence microscopy for the diagnosis of pulmonary tuberculosis: a multi-country cross-sectional evaluation. PLoS Med. 2011; 8(7): e1001057.^,55. Van-Deun A, Cattamanchi A, Davis JL, Ridderhof J. In reply to "Can LED fluorescence microscopy replace Ziehl-Neelsen microscopy in tuberculosis detection?". Int J Tuberc Lung Dis. 2012; 16(11): 1558-9.. Nevertheless, in many countries, with almost no laboratory with experience in FM and limited resources for training, this prerequisite may be difficult to carry out. We therefore designed a training program that consists of a training course followed by a stand-alone phase in which each technician, in their own laboratory, examined the same specimens by ZN and FM in a unblinded conducive way to acquire confidence in its ability to recognize the bacillus. The introduction of appropriate quality control and monitoring during the entire training process allowed the measurement of the trainees' proficiency after different length of experience. In this work, we report the results of the evaluation of this training program and the performance of the trainees in the use of LED-FM.

MATERIALS AND METHODS

Participating laboratories and technicians

The present study was conducted as part of a multicenter study developed to assess the feasibility of using LED-FM in laboratories of Argentina. Six laboratories were selected based on previous experience on FM and their interest in the project. Besides, to be eligible, the laboratory had to have acceptable performance in ZN (no FP and at most one Low False Negative error (LFN)), demonstrated by their annual rechecking results obtained in the last three years. All the laboratories depended on one coordination center that was in charge of training, monitoring, slides rechecking and data management.

For the purpose of the multicenter study, ZN slides were examined with the bright field microscopy routinely used in each laboratory at 1000x magnification, whereas FM slides were examined with Olympus CX31 microscopes with a TK-LED illumination (Tolket, Argentina), using 200x magnification for screening and 400x magnification for confirming and quantifying the slides. LED-FM grading of smears was done according to the WHO/IUTLD guidelines ⁶6. Rieder HL, Van Deun A, Kam KM, Kim SJ, Chonde TM, Trébucq A, et al. Priorities for tuberculosis bacteriology services in low-income countries. 2nd Edition. Paris: International Union Against Tuberculosis and Lung Disease; 2007. p.26. whereas ZN grading was made according to the national guidelines ⁷7. Sequeira MD, Barrera L, Imaz MS. Manual para el Diagnóstico Bacteriológico de Tuberculosis. Santa Fe: Instituto Nacional de Enfermedades Respiratorias «E. Coni». 2012; p.24. (Table 1). After examination, all smears were kept in the dark at ambient temperature. All specimens were further cultured according to the technique routinely established in each laboratory.

Out of the six centers, one had 40 years of experience on conventional FM, whereas the other five had no prior FM skills.

Training program

The training was organized in three phases: i. a training course, ii. an "in situ" training phase, in which ZN and FM were read in a unblinded way and iii. a "continuation phase", in which FM staining slides were observed independently of the ZN result.

Training course

A total of six workers, one of each laboratory participating in the multicenter study, were trained in a standardized three-day course to use the fluorescence microscope and to prepare auramine stains at the coordination laboratory. The training consisted of interactive lectures, discussions and practical hands-on training. It included two days of supervised microscopy routine during from which an expert helped to clarify any doubtful result in order to instill confidence.

All the teaching materials developed for the training process were given to the attendees at the end of the course. As soon as these trainees arrived to their laboratories, they were in charge of guiding other personnel; providing the materials employed at the course carried out at the coordination laboratory. Of the six course attendees, one could not participate in the study due to a sudden serious health problem. Overall, four of the five trained workers, without previous experience in FM, prepared nine technicians, whereas the sixth worker attending the course, with previous FM experience, guided another skilled microscopist in the objective and methodology of the strategy; and both were included in the project to act as a control group. The objective of including this control group was to compare its FM performance with that obtained by the technicians recently trained after different length of experience.

After training, each reader analysed a panel of 18 unstained slides by fm. The panel, generated from homogenized sputum, was composed by nine negative smear slides and nine positive smears; five which were "low positive smears" -three Scanty slides and two Positive (1+) slides-. The slides were stained and independently examined and reported in a standard form by the technicians. After reading, the panels were sent to the coordination laboratory along with recording and reporting forms. The information was decoded and analysed. For this evaluation, errors were classified as previously described ⁸8. APHL, CDC, IUATLD, KNCV, RIT & WHO. External Quality Assessment for AFB Smear Microscopy. Washington DC: World Health Organization; 2002..

"In situ" two-month training phase

A subset of the sputum specimens processed daily in the respective laboratories was selected to be included in the study; slides were prepared in duplicate for staining by ZN technique (0,3% carbolfuchsin and 0,1% methylene blue) or with 0,1% auramine-O, counterstained with 0.5% potassium permanganate, for LED-FM. Each technician read the slides both by ZN and FM in a unblinded way; reading first each ZN-stained slide and then the corresponding FM-stained slide. According to the coordination instructions, the sample of smears processed daily was enriched with positive smears by selecting second diagnostic sputum specimens from known TB patients or follow-up patients on anti-TB treatment. The mean number of slides read daily by each technician was assigned by the coordination under the following scheme: four trainees (30.8%) examined less than three smears per day, other four participants (30.8%) read three smears per day, and the remaining trainees ⁵5. Van-Deun A, Cattamanchi A, Davis JL, Ridderhof J. In reply to "Can LED fluorescence microscopy replace Ziehl-Neelsen microscopy in tuberculosis detection?". Int J Tuberc Lung Dis. 2012; 16(11): 1558-9. examined more than three smears per day. The decision of included technicians that read such low number of slides per day was based on the observation of the proportion - about 75%- of laboratories that processed less than 500 smear per year in Argentina.

Testing panels with similar composition of those employed for the evaluation of the trainees after the three-day course were used to assess the reader performance at the end of the first and second month of this "in situ" phase.

Continuation phase

Readers that showed acceptable performance in FM at the end of the unblinded phase, continued with a training period in which the ZN and FM slides were read in a blinded way. Acceptable performance was defined as at most one LFN without any high false negative error (HFN) nor FP in the last testing panel.

During this phase, FM slides were examined without being aware of the ZN results. ZN and FM slides were read by different microscopists. They all read similar proportion of slides stained by each method. In one laboratory, in which only one technician was responsible of reading both slides, blinding was assessed by overlabelling the FM and ZN slides with different identification numbers. Moreover, different registers were used to record ZN and FM results. Performance of FM during this continuation phase was done by rechecking all slides at the coordination center.

During all the training phases, patient care was based on the results of ZN; therefore, as none of the laboratory comparisons affected routine patient management or involved collection of additional specimens, informed consent was not deemed necessary.

Analysis of data

For the analysis of proficiency testing panels, each centre was requested to have the slides read independently by the readers, and then the results were sent back along with the panel to the coordination laboratory. On receipt of the panel results, the information was decoded and analysed, and in case of disagreements, the slides were re-stained and checked by a technician on receipt to rule out labelling errors. The report containing results and recommendations were sent back to the trainees with a delay of no more than seven days since the reception of the panel results.

For the rechecking process, the slides were re-stained and read by the first rechecker, and results were compared with results of laboratories. Divergent results were verified by a second rechecker, which were considered as final ⁸8. APHL, CDC, IUATLD, KNCV, RIT & WHO. External Quality Assessment for AFB Smear Microscopy. Washington DC: World Health Organization; 2002., except in cases of divergent results from diagnostic smears reported as "scanty" by the trainees. In these cases, AFB may be wash off fixed smears during re-staining, which may result in a report of FP by the rechecker. For this reason, both results (the "scanty" result from the trainees and the negative result from the recheckers) were stratified by culture to assign true-/false-positive results. It is however acknowledged that this partial verification could result in an inaccurate estimate of reader's performance.

The main outcomes for comparison of trainees" performance were the percentage of errors and the sensitivity and specificity of the readers compared with the recheckers.

The percentage of technicians with acceptable performance after different length of training was determined. The effect of trainees" characteristics (sex, qualification, service year and workload) on their practical performances was statistically tested using Chi-squared test or Fisher exact tests for bivariate analysis.

RESULTS

Characteristics of the trainees

Out of the 13 participants without previous experience in FM, four (30.8%) were initially trained at the coordinator center's course, 11 (85.6%) were women, seven (63.9%) had more than 10 years of service, 10 (76.9%) had a professional qualification whereas the rest (three) were laboratory technicians.

The two participants, with more than 10 years of experience in FM, were women and biochemistries.

Performance of trainees by panel testing analysis before and during the "in situ" two-month training phase

Analysis of the effect of being initially trained at the coordination center or at their own laboratories showed no significant differences in the percentage of errors found in the initial panel of slides (Chi-squared test, p>0.05) (Table 2).

The results of the panel examined after the initial course showed that out of the 13 trainees, nine (69.2%) made minor errors (LFP and/or LFN); all of them with LFPS, reached a specificity of only 70.9%. During the "in situ" training phase, the analysis of the second and third panels showed that the percentage of minor errors and specially the frequency of LFPS significantly decreased (p<0.05) (Table 2). Overall, before the "in situ" training phase, only four of the 13 trainees (30.8%) obtained an acceptable performance (Table 3), but after one and two months of experience, the percentage of acceptable performance increased to eight (61.5%) and 10 (76.9%), respectively. The two microscopists, with previous experience in FM, read all the three panels without any error.

Acceptable performance by trainee's characteristics

At the end of the second month of the "in situ" phase, 10 out of the 13 technicians without FM experience obtained an acceptable performance.

Analysis of effect of trainees' characteristics on acceptable performance showed that the only feature associated with their performances was the number of slides read per day during the training phase. Trainees who had read three or more slides per day were found to obtain better results than those who had read fewer slides per day (Fisher' exact test, p=0.0) (Table 4).

Performance of trainees by rechecking during the blinded-training phase

Only those trainees with an acceptable performance at the end of the "in situ" phase could continue with a period in which each technician stained and read 200 slides by FM without being aware of the ZN results. Sensitivity and specificity levels compared to the recheckers were satisfactory during the entire phase (Table 5); nevertheless; although good, analysis of the sensitivity by 100 slides showed that sensitivity significantly improved with the increasing of experience. On the other hand, the two FM-experienced microscopists obtained 100% specificity and 98.9% sensitivity, values that were not significant different of those obtained by the microscopists without FM-experience during their evaluation of the second 100 slides (99.9% specificity and 99.3% sensitivity) (Chi-squared test, p>0.05).

DISCUSSION

Immediately following WHO recommendation of using LED-FM as an alternative of ZN, various authors ⁴4. Cuevas LE, Al-Sonboli N, Lawson L, Yassin MA, Arbide I, Al-Aghbari N, et al. LED fluorescence microscopy for the diagnosis of pulmonary tuberculosis: a multi-country cross-sectional evaluation. PLoS Med. 2011; 8(7): e1001057.^,99. Albert H, Manabe Y, Lukyamuzi G, Ademun P, Mukkada S, Nyesiga B, et al. Performance of three LED-based fluorescence microscopy systems for detection of tuberculosis in Uganda. PLoS One. 2010; 5(12): e15206.

10. Bonnet M, Gagnidze L, Githui W, Guérin PJ, Bonte L, Varaine F, et al. Performance of LED-based fluorescence microscopy to diagnose tuberculosis in a peripheral health centre in Nairobi. PLoS One. 2011; 6(2): e17214.^-1111. Cattamanchi A, Huang L, Worodria W, den Boon S, Kalema N, Katagira W, et al. Integrated strategies to optimize sputum smear microscopy: a prospective observational study. Am J Respir Crit Care Med. 2011; 183(4): 547-51. couldn't demonstrate the superior performance of LED-FM over ZN, pointing out that their observations might be explained by a limited proficiency of the technicians associated to a less-than-ideal initial training. Here we report the evaluation of a LED-FM training program developed as part of a study to assess the feasibility of using LED-FM in laboratories in Argentina. After a three-days training course, there was a large difference in the specificity of the trainees (70.9%) compared with the FM-experienced technicians (100.0%), supporting the observation of others that, unlike ZN, moderate training (a three-days course) is not enough to make technicians proficient in LED-FM ⁴4. Cuevas LE, Al-Sonboli N, Lawson L, Yassin MA, Arbide I, Al-Aghbari N, et al. LED fluorescence microscopy for the diagnosis of pulmonary tuberculosis: a multi-country cross-sectional evaluation. PLoS Med. 2011; 8(7): e1001057.^,55. Van-Deun A, Cattamanchi A, Davis JL, Ridderhof J. In reply to "Can LED fluorescence microscopy replace Ziehl-Neelsen microscopy in tuberculosis detection?". Int J Tuberc Lung Dis. 2012; 16(11): 1558-9.. The percentages of FPS significantly decreased after the first month of "in situ" training and this reduction was observed in all trainees, independently of where they have received the three-days initial course (at the coordination center by FM experienced technicians or at their own laboratories by those technicians formerly trained at the site of coordination), suggesting that the primary training received by technicians at the coordination center appeared to be enough to guide their partners at their own laboratories, previous the "in situ" training phase. Overall, the method was picked up by mostly technicians after a standalone practice of about two months; this time does not seem very extensive, but was longer that the period reported by Van Deun, who found that their technicians did not commit FPS after only two weeks-training under supervision of daily practice ¹²12. Van Deun A, Aung KJ, Khan MH, de Jong BC, Gumusboga M, Hossain MA. An operational study comparing microscopes and staining variations for tuberculosis LED FM. Int J Tuberc Lung Dis. 2014; 18(8): 964-71..

In routine laboratories, technicians have different backgrounds in terms of qualification, experience and others. When examining the trainee's characteristics associated with poor performance, we found that participants who had read less than three slides/day were found to commit higher errors than those who had examined more slides/ day. Other factors could not be identified possible due to the limited power of the study associated with the low number of participants included.

During the continuation of the blinded-training phase, the specificity obtained by the trainees was high, although some LFP were indeed identified. There are several documented practical problems that may adversely affect the repeatability of the AFB smears results during external quality assessment (EQA) ¹³13. Van Deun A, Roorda FA, Chambugonj N, Hye A, Hossain A. Reproducibility of sputum smear examination for acid-fast bacilli: practical problems met during cross-checking. Int J Tuberc Lung Dis.1999; 3(9): 823-9.. Besides, in a smear read as scanty by the trainees, AFB might be wash off fixed smears during re-staining process, which may result in a report of a false positive by the rechecker. For this reason, we decided the use culture as a standard to consider a LFP result in those diagnostic smears read as scanty by the trainees and negative by the recheckers. Neither technique is a perfect reference standard, and the assumption that either culture system has higher precision than microscopy in all cases might be controversial; this is because, cases with false-negative result in culture medium may appear due to different methodological factors that affect the viability of the bacilli.

Our training program proposed the unblinded reading of duplicate slides stained by ZN and auramine-O, as a methodology of obtaining confidence in the bacilli recognition. The use of this parallel reading of slides stained by ZN -a technique known to be less sensitive than LED-FM- may have some risks of introducing bias toward missed scanty slides as, based on his trust in ZN, when a FM-unexperienced technician read a ZN slide as negative, he could declare a scanty auramine-stained smear as a false positive slide. This could reduce his ability of identifying slides with few bacilli. Nevertheless, technicians' sensitivity resulted high during the blinded phase, reaching levels of sensitivity like those obtained by technicians with FM experience of more than 10 years.

In conclusion, our study shows that training technicians belonging to laboratories where no experienced staff in LED-FM is available in everyday practice is feasible with high degrees of trainees" reading proficiency reached after only a short standalone training phase. Nevertheless, considering the necessity of closed monitoring of the technicians' performance, it seems that a well-established EQA program is a prerequisite to assure the implementation of this type of training. Moreover, even WHO updated the scale up of the automated nucleic acid amplification test, Xpert MTB-RIF, to replace microscopy as the initial diagnostic test, the current recommendation of WHO is to continue using microscopy for treatment monitoring, proposing that existing microscopy facilities may be upgraded with LED-FM to improve their effectiveness and the efficiency of testing ¹⁴14. WHO. Framework of indicators and targets for laboratory strengthening under the End TB Strategy. Geneva: World Health Organization; 2016. WHO/HTM/TB/2016.18.. The 2015 Global TB Report ¹⁵15. WHO. Global tuberculosis report. France: Word Health Organization; 2015. WHO/HTM/TB/2015.22. showed that the adoption of LED technology remains low; up to 2014 only 2% of the laboratories in the Region of the Americas had switched ZN to LED-FM. We hope that the learning of this intervention, developed for extensive countries with limited resources, could be of help to improve the implementation of this technology

Acknowledgement:

This study was supported by Agencia Nacional de Promoción Científica y Tecnológica. PAE-PID-2007-00127.

REFERENCES

Publication Dates

History