Evaluation of the Haemoglobin Colour Scale and comparison with the HemoCue haemoglobin assay

J.J. Paddle¹

OBJECTIVE: To evaluate the Haemoglobin Colour Scale developed by WHO for estimating haemoglobin concentration and to compare the results obtained using it and the HemoCue assay with those determined using a reference method, the Technicon H3 analyser.
METHODS: The Colour Scale and HemoCue assay were used to test 408 blood samples. Subsequently, Bland–Altman plots were determined and the proximity of the test results to those obtained using the reference method was determined.
FINDINGS: The mean difference between the Haemoglobin Colour Scale and the reference method was 0.19 g/dl (95% confidence interval: 3.50 g/dl below to 3.11 g/dl above); the corresponding value for the HemoCue assay was 0.50 g/dl (1.16 g/dl below to 0.16 g/dl above). Only 46.08% of the results obtained by means of the Colour Scale were within 1.0 g/dl of the reference method, whereas 95.34% of the HemoCue results fell within this limit; 22.79% of the Colour Scale results but none of the HemoCue results lay more than 2.0 g/dl from the reference method.
CONCLUSION: The Haemoglobin Colour Scale test is too inaccurate for general use, particularly if devices such as the HemoCue are available.

Keywords Hemoglobinometry/methods; Reference standards; Comparative study; Developing countries (source: MeSH, NLM).

Mots clés Hémoglobinométrie/méthodes; Norme; Etude comparative; Pays en développement (source: MeSH, INSERM).

Palabras clave Hemoglobinometría/métodos; Estándares de referencia; Estudio comparativo; Países en desarrollo (fuente: DeCS, BIREME).

Introduction

WHO has developed a simple method of assaying the concentration of haemoglobin in a drop of blood, the Haemoglobin Colour Scale. To use the Colour Scale, one drop of blood is spotted on to a strip of a particular type of white absorbent paper and its colour is compared with a chart (1) comprising six reference shades of red matched to blood with haemoglobin concentrations of 4, 6, 8, 10, 12 and 14 g/dl. The test is easy to perform, cheap and portable, and is thus potentially suitable for use in developing countries and in remote situations.

Initial laboratory trials (1) and field studies (2, 3) gave results that were promising compared with those of a routine laboratory method. Correlation coefficients of 0.9386 were reported for the laboratory trial, and 0.863–0.962 for one of the field studies (2), which also reported a sensitivity of 91% and a specificity of 86% for the detection of anaemia, i.e. haemoglobin concentrations 3) also claimed good results, reporting agreement within ±1 g/dl of the reference (Coulter Counter; Onyx, Coulter Counters, Johannesburg, South Africa) in 40% of values, and 1–2 g/dl in a further 27% of values.

Regression analysis and the calculation of correlation coefficients may, however, be inappropriate for assessing agreement between different methods of clinical measurement (4). In particular, it should be borne in mind that high correlation does not necessarily imply good agreement. A more valid assessment of agreement between tests is provided by a Bland–Altman plot of the differences between the results of assays against the mean value. This plot has not previously been applied to any of the published data on the Haemoglobin Colour Scale. The present paper reports the results obtained with the Colour Scale on blood samples in the laboratory, together with appropriate Bland–Altman plots. The samples were also tested with the HemoCue analyser (HemoCue AB, Ängelholm, Sweden), which has been validated in other studies (5, 6).

Methods

The haemoglobin concentrations of 408 randomly and anonymously collected venous blood specimens, taken from routine laboratory samples, were estimated by a single operator using three assays, as described below. The operator practised using the assays on a selection of samples before commencing the study, following the instructions and the published guidelines (2). The reference haemoglobin concentration was obtained for each sample by means of the Technicon H3 automated laboratory analyser (Bayer AG, Leverkusen, Germany). The Haemoglobin Colour Scale, which potentially provides the most subjective test since it depends on the observer's interpretation of the colour of a drop of blood, was performed first on each sample, i.e. before the operator knew the results of the HemoCue and reference assays.

The samples were obtained over a period of two months by venepuncture and were collected in ethylenediaminetetraacetic acid vacuum tubes on each day of testing. The samples, taken at random from racks in the laboratory of a district general hospital in order to ensure anonymity, originated from adult and child inpatients with various medical and surgical conditions.

The Haemoglobin Colour Scale and HemoCue results were compared separately with the laboratory reference readings. Bland–Altman plots were obtained and the proximities of the test values to the reference value were noted.

The analyses were performed using Microsoft Excel and SPSS software. The haemoglobin concentrations were measured in g/dl, to one decimal place with the HemoCue and reference method, and to values of 4, 6, 8, 10, 12 and 14 g/dl for the Haemoglobin Colour Scale.

Results

Fig.1 shows the Bland–Altman plots for the Haemoglobin Colour Scale and HemoCue tests compared with the reference method. The mean differences ± 1.96 standard deviations are indicated. The limits of agreement, i.e. the ranges between which 95% of the results lie, were 3.50 g/dl below and 3.11 g/dl above the reference value for the Colour Scale, and 1.16 g/dl below and 0.16 g/dl above this value for the HemoCue method.

Table 1 shows the results for the two assays that lay within 1 g/dl, 1.1–2.0 g/dl, 2.1–3.0 g/dl, and >3.0 g/dl of the reference.

Discussion

Because most established tests were unsuitable for reasons of complexity, cost or poor results, the Haemoglobin Colour Scale was introduced for use in the field by public health workers as a simple means of screening for anaemia (7). The simplest haemoglobin test, the Tallqvist method (8), in which the colour of blood on blotting paper was compared with a colour scale, had been criticized because of its inaccuracy (9, 10). The Colour Scale was designed with a view to overcoming problems associated with this method, specifically by using a particular type of absorbent paper, developing an accurate colour chart using computerized spectrophotometry, and modifying the method of viewing the blood against the chart (1).

Good results have been reported with the Haemoglobin Colour Scale in studies conducted in the laboratory (1) and in the field (2, 3). However, use of correlation coefficients and the other statistical tests employed may not have been appropriate for analysing the data.

The present study indicated poor agreement between the results obtained using the Haemoglobin Colour Scale and those obtained using the reference method or HemoCue. Bland–Altman plots show limits of agreement (95% confidence intervals) of 3.50 g/dl below and 3.11 g/dl above the reference value for the Colour Scale, and of 1.16 g/dl below and 0.16 g/dl above the reference for the HemoCue assay.

It may be useful to assess the proximity of the measured haemoglobin to the reference result. Since the Haemoglobin Colour Scale can, at best, estimate haemoglobin to the nearest even integer unit, a measured concentration to within 1 g/dl of the reference could be considered correct, while results outside this range would be progressively more inaccurate. For example, a haemoglobin concentration of 10 g/dl as determined by the Colour Scale would be considered correct if the reference were 9.1 g/dl; if, however, the reference were 8.9 g/dl, the result to be considered correct would be 8 g/dl. Even by these criteria the Colour Scale performed poorly. Only 46.08% of readings could be considered correct. Over half were inaccurate by more than 1 g/dl, and over one-fifth differed by more than 2 g/dl from the reference value. In contrast, the results of the HemoCue test were within 1 g/dl of the reference in 95.34% of readings, and none were wrong by more than 2 g/dl. The worst readings with the HemoCue assay, obtained in three instances, i.e. 0.7%, were 1.5 g/dl higher than the reference. The worst "under read" was 0.6 g/dl, which occurred once (0.2%).

It should be noted that the use of a single operator in the present study might have introduced bias. On the other hand, this approach eliminated the possibility of inter-observer error noted by other authors (2, 3), and simplified comparison with other assays. Furthermore, the operator practised using the assays on a selection of blood samples before commencing the study, carefully following the instructions supplied with the kits, and observing published advice (2) such as the use of natural light from over the shoulder when reading the Colour Scale. Consequently, this operator probably achieved greater accuracy than would have been the case with a larger number of less meticulous ones.

It could be argued that using laboratory samples from general hospital inpatients might reduce the value of this assessment with respect to the intended use of the Haemoglobin Colour Scale in developing countries. However, the samples covered a diversity of adult and child patients with both medical and surgical conditions and a range of degrees of anaemia. There is no reason to suspect that the blood of a patient with anaemia attributable to a particular condition differs in appearance from that of another patient with a different condition but which contains the same concentration of haemoglobin. The anonymous and random method of sampling made it impossible to be certain that bias in respect of age, sex, or medical condition did not occur. However, the large sample size made this extremely unlikely.

Conclusion

While considerations such as cost and simplicity of use make the Haemoglobin Colour Scale an attractive proposition, its poor accuracy renders its use questionable. The wide limits of agreement on the Bland–Altman plot highlight the shortcomings of the Colour Scale, particularly in view of the good agreement demonstrated in the present study and elsewhere for devices such as the HemoCue (5, 6).

Acknowledgements

Gratitude is expressed to the staff of the Haematology Laboratory, Royal Cornwall Hospital, for assistance with data collection.

Conflicts of interest: none declared.

Résumé

Evaluation de l'échelle colorée pour le dosage de l'hémoglobine et comparaison avec le test HemoCue

OBJECTIF: Evaluer l'échelle colorée développée par l'OMS pour le dosage de l'hémoglobine (Haemoglobin Colour Scale) et comparer les résultats ainsi que ceux du test HemoCue avec les concentrations d'hémoglobine déterminées par une méthode de référence avec l'analyseur Technicon H3.
MÉTHODES: L'échelle colorée et le test HemoCue ont été utilisés pour tester 408 échantillons de sang. Des courbes de Bland-Altman ont ensuite été établies et la proximité des résultats des tests avec ceux obtenus par la méthode de référence a été déterminée.
RÉSULTATS: La différence moyenne entre les valeurs obtenues avec l'échelle colorée et avec la méthode de référence était de 0,19 g/ dl (limites de confiance à 95 % : 3,50 g/dl au-dessous et 3,11 g/ dl au-dessus) ; les valeurs correspondantes pour le test HemoCue étaient de 0,50 g/dl (1,16 g/dl au-dessous et 0,16 g/dl au-dessus). Seuls 46,08 % des résultats obtenus avec l'échelle colorée se trouvaient à moins de 1,0 g/dl de ceux obtenus avec la méthode de référence, tandis que 95,34 % des résultats de l'HemoCue se situaient dans ces limites ; 22,79 % des résultats de l'échelle colorée, mais aucun de ceux de l'HemoCue, se situaient à plus de 2,0 g/dl des résultats de la méthode de référence.
CONCLUSION: L'échelle colorée pour le dosage de l'hémoglobine est trop inexacte pour l'usage général, surtout si des dispositifs tels que l'HemoCue sont disponibles.

Resumen

Evaluación de la Escala Colorimétrica para Hemoglobina y comparación con la prueba HemoCue

OBJETIVO: Evaluar la Escala Colorimétrica para Hemoglobina desarrollada por la OMS para calcular la concentración de hemoglobina y comparar los resultados arrojados por dicha escala y por la prueba HemoCue con los obtenidos mediante una técnica de referencia basada en el analizador Technicon H3 .
MÉTODOS: Se analizaron 408 muestras de sangre mediante la escala colorimétrica y la prueba HemoCue. Seguidamente se representaron gráficamente los datos conforme al método de Bland-Altman, y se determinó el grado de similitud de los resultados de las pruebas con los del método de referencia.
RESULTADOS: La diferencia media entre la Escala Colorimétrica para Hemoglobina y el método de referencia fue de 0,19 g/dl (intervalo de confianza de 95%: 3,50 g/dl por debajo y 3,11 g/dl por arriba); el valor correspondiente para la prueba HemoCue fue de 0,50 g/dl (1,16 g/dl por debajo y 0,16 g/dl por arriba). Sólo el 46,08% de los resultados obtenidos por medio de la escala colorimétrica difirieron en menos de 1,0 g/dl de los conseguidos con el método de referencia, mientras que el 95,34% de los resultados de HemoCue no superaron ese grado de diferencia; el 22,79% de los resultados de la escala colorimétrica, frente a ninguno de los resultados de HemoCue, se desviaron más de 2,0 g/dl de los del método de referencia.
CONCLUSIÓN: La Escala Colorimétrica para Hemoglobina es demasiado imprecisa como alternativa de uso general, sobre todo si se dispone de dispositivos como HemoCue.

References

1. Stott GJ, Lewis SM. A simple and reliable method for estimating haemoglobin. Bulletin of the World Health Organization 1995;73:369-73.        

2. Lewis SM, Stott GJ, Wynn KJ. An inexpensive and reliable new haemoglobin colour scale for assessing anaemia. Journal of Clinical Pathology 1998;51:21-4.        

3. van den Broek NR, Ntonya C, Mhango E, White SA. Diagnosing anaemia in pregnancy in rural clinics: assessing the potential of the Haemoglobin Colour Scale. Bulletin of the World Health Organization 1999;77:15-21.        

4. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307-10.        

5. Be WKM, Kerkhamp HEM, Booij LHDH. HemoCue — a new haemoglobinometer in the clinic. European Journal of Anaesthesiology 1991;8:55-8.        

6. Jaeger M, Ashbury T, Adams M, Duncan P. Perioperative on-site haemoglobin determination: as accurate as laboratory values? Canadian Journal of Anaesthesia 1996;43:795-8.        

7. Control of nutritional anaemia with special reference to iron deficiency. Report of an IAEF/USAID/WHO Joint Meeting. Geneva: World Health Organization; 1975. WHO Technical Report Series, No. 580.        

8. Tallqvist TW. (Practical method for the direct evaluation of the amount of haemoglobin in the blood.) Archives of General Medicine 1900;3:421-5.          In French.

9. van Lerberghe W, Keegels G, Cornelis G, Ancona C, Mangelschots E, van Balen H. Haemoglobin measurement: the reliability of some simple techniques for use in a primary health care setting. Bulletin of the World Health Organization 1983;61:957-65.        

10. Stone JE, Simmons WK, Jutsum PJ, Gurney JM. An evaluation of methods of screening for anaemia. Bulletin of the World Health Organization 1984;62:115-20.        

¹Anaesthetic Specialist Registrar, Department of Anaesthesia, Royal Cornwall Hospital, Truro TR1 3LJ, England (email: jpaddle@doctors.org.uk).

Ref. No. 01-1351