POLICY AND PRACTICE
Overlooked opportunities for investing in health research and development
Investir dans la recherche et le développement en santé : les occasions manquées
Inversión en investigaciones y desarrollo sanitarios: oportunidades desaprovechadas
David W. FraserI
IExecutive Director, International Clinical Epidemiology Network (INCLEN) Inc., 3600 Market Street, Suite 380, Philadelphia, PA 191042644, USA
ABSTRACT
In 1996, an Ad Hoc Committee on Health Research Relating to Future Intervention Options (formed under the auspices of the World Health Organization) described a model for setting priorities in research funding. This model, however, as presented in the Ad Hoc Committees report entitled Investing in health research and development, fails in the following important situations: (i) when there is a health problem about which little is known; (ii) when current control measures are unsustainable; (iii) when there are complex risk factors, like social factors, which affect many different diseases; and (iv) when the disease burden and resources for control vary greatly from one place to another. In situations of uncertainty or complexity, a method of priority-setting that emphasizes certainty and simplicity may actually mislead. A transparent, matrix-based process illustrated with an example of priority-setting for malaria may permit such uncertainty and complexity to be better taken into account in setting health research priorities.
Keywords: burden of illness; disability-adjusted life year; health priority; research priority; research support.
RÉSUMÉ
En 1996, le Comité ad hoc sur la recherche en santé concernant les options dinterventions futures (créé sous les auspices de lOrganisation mondiale de la Santé) a publié un modèle détude des investissements potentiels dans la recherche et le développement en santé. Dans ce schéma, la charge de morbidité des maladies était représentée par une zone rectangulaire divisée en quatre compartiments : a) la charge de morbidité actuellement évitée; b) la charge que lon pourrait éviter grâce à des interventions ayant un bon rapport coût/efficacité si lon utilisait plus efficacement les méthodes de prévention disponibles; c) la charge de morbidité que lon pourrait éviter grâce à des interventions nayant pas un bon rapport coût/efficacité, à laide des méthodes actuelles; et d) la charge de morbidité pour laquelle nous ne disposons pas encore de méthodes de prévention. Ce Comité ad hoc indiquait que la taille des compartiments b), c) et d) était grossièrement proportionnelle aux priorités de la recherche dans chacun de ces domaines et faisait valoir que le compartiment b) était du ressort de la recherche sur des systèmes de santé et les politiques, le c) de la compétence de la recherche biomédicale portant sur des solutions de rechange moins onéreuses et que le d) relevait de la recherche biomédicale portant sur de nouvelles interventions. Toutefois, ce modèle ne peut être appliqué dans les situations suivantes : i) lorsquil sagit dun problème sanitaire dont on sait peu de chose ; ii) lorsquon ne peut maintenir durablement les mesures de lutte actuelles; iii) lorsquil y a des facteurs de risque complexes comme les « facteurs sociaux », qui interviennent dans de nombreuses maladies ; et iv) lorsque la charge de morbidité et les ressources consacrées à la lutte sont extrêmement variables dun endroit à lautre. Dans toutes ces situations, la charge de morbidité que la recherche pourrait permettre de réduire ne peut être représentée par un diagramme simple. Dans les situations incertaines ou complexes, une méthode détablissement des priorités qui mettrait laccent sur labsence de doute et la simplicité risquerait dinduire en erreur. En revanche, une méthode transparente, basée sur un tableau à double entrée, pourrait permettre de mieux tenir compte de ces incertitudes et de cette complexité lorsquon fixe les priorités de la recherche en santé. Par exemple, on a tenu compte des priorités de la recherche au niveau international (dans le cas présent, des priorités du programme OMS Faire reculer le paludisme), des avantages comparés que présentait lorganisation qui sest chargée de la recherche (dans le cas présent lINCLEN) et de la faisabilité de cette dernière, lorsquon a fixé les critères précis permettant de classer les projets de recherche candidats selon cette méthode. Les participants à létablissement des priorités ont discuté ouvertement des points à attribuer aux projets de recherche candidats pour chacun des huit critères retenus. Lorsque les participants ont donné des notes différentes à une proposition de recherche pour un critère donné, lensemble des notes a été enregistré et lon sest servi de leur moyenne pour calculer la somme des points obtenus pour les huit critères. On a ensuite cherché à parvenir à un consensus sur la série de propositions dont le rang de priorité était le plus élevé, à partir du total des points obtenus. Lorsquon utilise une méthode matricielle de ce type, on peut retenir comme critère de jugement des recherches à entreprendre la charge de morbidité 1) qui pourrait être évitée avec un bon rapport coût/efficacité, 2) qui pourrait être évitée, mais sans un bon rapport coût/ efficacité, et 2) quil est impossible déviter actuellement (voir plus haut b), c) et d)); on pourra retenir par ailleurs les sources dincertitude et de complexité majeures suivantes : risque de nouveaux problèmes de santé, importance du maintien des efforts de lutte contre les maladies, facteurs transdisciplinaires intervenant dans de nombreuses maladies et variations observées localement quant à la survenue des maladies et aux méthodes de lutte. Il est très probable que les résultats de cette méthode conduisent à des prises de décision plus nuancées dans les situations incertaines et complexes.
RESUMEN
En 1996, un Comité Especial de la OMS sobre Investigaciones Sanitarias relativas a Opciones de Intervención Futuras (creado bajo los auspicios de la Organización Mundial de la Salud) publicó un modelo para estudiar las posibilidades de inversión en las investigaciones y el desarrollo sanitarios. La carga de morbilidad se representó en dicho modelo mediante un diagrama consistente en una zona rectangular con cuatro compartimentos correspondientes a (a) la carga actualmente evitada, (b) la carga que podría evitarse de manera eficaz en relación con el costo haciendo un uso más eficiente de los métodos preventivos ya disponibles, (c) la carga que podría evitarse, pero no de manera eficaz en relación con el costo, con métodos ya disponibles, y (d) la carga para cuya prevención aún no disponemos de métodos. El Comité Especial concluyó que las áreas (b), (c) y (d) del diagrama eran aproximadamente proporcionales a la prioridad que había que conceder a las investigaciones en cada uno de esos ámbitos, y expuso las razones por las que a su juicio la carga (b) debíaabordarse mediante investigaciones de los sistemas y las políticas de salud, la carga (c) mediante la investigación biomédica de alternativas de bajo costo, y la carga (d) mediante investigaciones biomédicas encaminadas a idear nuevas intervenciones. Sin embargo, el modelo falla en las siguientes situaciones: (i) cuando interviene un problema de salud sobre el que se tiene poca información; (ii) cuando las medidas de control en vigor son insostenibles; (iii) cuando concurren factores de riesgo complejos, como «factores sociales», que afectan a muchas enfermedades; (iv) cuando la carga de morbilidad y los recursos destinados a las actividades de control varían considerablemente de un lugar a otro. En todas esas circunstancias, la carga de morbilidad que pueda aliviarse mediante las investigaciones no se presta a quedar reflejada claramente en un simple diagrama. En situaciones de incertidumbre o complejidad, un método de establecimiento de prioridades que haga hincapié en la certeza y la simplicidad puede inducir a error. Por el contrario, un método transparente basado en el uso de matrices puede integrar mejor la incertidumbre y la complejidad en el proceso de establecimiento de las prioridades de investigación sanitaria. Un ejemplo de ese método consistió en fijar criterios explícitos para ordenar distintas alternativas de proyectos de investigación teniendo en cuenta las prioridades internacionales en ese ámbito (en este caso, las prioridades del programa de la OMS Hacer Retroceder el Paludismo), las ventajas comparativas de la organización que emprendió esas investigaciones (en este caso la INCLEN), y la viabilidad de éstas. Los participantes en el proceso de establecimiento de prioridades discutieron abiertamente la puntuación que debía asignarse a los proyectos de investigación en liza para cada uno de los ocho criterios explicitados. Cuando los participantes asignaban distinta puntuación a una determinada propuesta de investigación según uno de los criterios, se registraba el intervalo de esas puntuaciones y se utilizaba el punto medio del intervalo como sumando de la puntuación total correspondiente a los ocho criterios establecidos. A continuación se intentaba alcanzar un consenso para determinar las propuestas de mayor prioridad a partir de las puntuaciones totales. Usando un método de ese tipo basado en el uso de matrices, se podrían incluir entre los criterios de valoración de las distintas líneas de investigación no sólo la magnitud de la carga de morbilidad que podría evitarse de manera eficaz en relación con el costo, de la que podría evitarse pero no eficientemente, y de la que por ahora no puede evitarse (designadas antes como (b), (c) y (d)), sino también fuentes importantes de incertidumbre y complejidad tales como el riesgo de problemas de salud emergentes, la necesidad de sostener las actividades de lucha contra las enfermedades, los factores interdisciplinarios que inciden en varias enfermedades, y las diferencias locales en la aparición y el control de las enfermedades. Los resultados de este método llevarían muy probablemente a adoptar decisiones más matizadas en situaciones de incertidumbre y complejidad.
Introduction
In 1996, WHO published the report of an Ad Hoc Committee on Health Research Relating to Future Intervention Options, which presented a schema for considering potential investments in health research and development (1). Disease burden was represented in that schema as a rectangular area with four compartments (Fig. 1), which indicated (a) the burden currently averted, (b) the burden that could be averted cost-effectively with more efficient use of currently available preventive methods, (c) the burden that could be averted, but not cost-effectively, with currently available methods, and (d) the burden that we do not yet have methods to avert. The Ad Hoc Committee argued that the sizes of (b), (c) and (d), i.e. the areas occupied by each of them within the rectangle, were roughly proportionate to the priorities that should be placed on research in each one. The Committee also made a case for (b) to be addressed by health systems and policy research, (c) to be addressed by biomedical research on lower-cost alternatives, and (d) to be addressed by biomedical research on new interventions. In a 5-step priority-setting process suggested by the Ad Hoc Committee, the first 3 steps relate to the size of the four compartments; the remaining 1½ steps involve consideration of the probability of successful development of a new intervention and the adequacy of the current level of research and development effort in the area.
This approach nicely structures consideration of a wide range of potential health research, but has serious shortcomings. This paper illustrates where the approach fails and suggests a way to improve it, so that important areas of research are not overlooked just because they do not fit into the Ad Hoc Committees schema.
Critical examination of the AHCs approach
First, as an illustration of the strength of the Ad Hoc Committees approach, Fig. 2a shows a plausible illustration of the current situation for pneumonia, which has been estimated to cause a burden of 172 million DALYs (disability-adjusted life years) (1). Given the existence and demonstrated efficacy but relatively small degree of use of unconjugated polysaccharide vaccines against Streptococcus pneumoniae, one of the most common causes of pneumonia morbidity and mortality, research to make these vaccines more readily available may be expected to increase the amount of pneumonia averted (Fig. 2b), although it may not change the amount avertable. However, if current research on conjugated pneumococcal polysaccharide vaccines proves that they are effective, it will be possible to prevent many cases of pneumonia in very young children, although at a cost that may not soon be widely cost-effective thus producing the situation in Fig. 2c.
When the burden of a health problem is unknown, this diagrammatic approach does not work well. Fig. 3 illustrates the uncertainty about the size of the disease burden of some conditions which we suspect are important. Without reasonably accurate information about the magnitude of the disease burden, one cannot even draw the outer rectangle which is the starting point for the Ad Hoc Committees setting of research priorities. This inability is illustrated in Fig. 3 with boundary lines that fade out, and hence never enclose a space that represents the burden of disease. Violence within the family, for example, is only now being systematically studied in many countries, so the burden of spousal abuse or child abuse is poorly understood (2, 3). Similarly, nosocomial infections have been studied systematically in only a few countries, and where they have not been studied particularly in low-income countries the burden is likely to be proportionately greater, so that the global estimates are unreliable. Priority-setting that excludes diseases which are likely to be common, but about which little is known, will not have the desired outcome.
Placing emphasis on the unaverted portions of disease burden makes good sense in setting priorities for research, except in situations where current control measures may not be sustainable. In these situations, careful attention to the disease already averted represented by (a) in Fig. 1 may be important. Such attention may lead to recognition that although the concept of disease averted which underlies rectangle (a) may seem intuitively clear, a diagram that suggests a relationship between the size of (a) and that of (b), (c) and (d) may be quite misleading. Is there a unique pre-intervention baseline that can be used for measuring the disease burden that has been averted? As shown in Fig. 4, the amount of disease currently averted may be unknown, as is the case with the effect of current political structures on health problems caused by war and civil unrest. What is the health risk associated with disintegration of current political structures? Events in the former Soviet Union over the last decade suggest that it can be very large.
On a smaller scale, what is at risk when vertical disease control programmes divert health personnel from their ordinary tasks to attend to an extraordinary one, as with pulse polio immunization? At the request of the Indian government and with the support of the United States Agency for International Development, INCLENs units in India conducted a series of major, multi-site surveys of health professionals, government officials and the public before and after the Pulse Polio Immunization campaigns to assess this potential risk. Happily, public perceptions of the effectiveness of the government health system actually improved as a result of the polio campaigns (4).
The rectangle representing the burden of disease in Fig. 1 applies most obviously to specific disease entities, like pneumonia. Murray & Lopez, however, have emphasized that an exclusive focus on individual diseases would miss the critical role of risk factors that may affect the burden of a whole family of diseases (5). To meet that need, they reported the result of studies that examined 10 risk factors which, according to their estimates, account for 38% of the total burden of disease. Tobacco-related disease was one of these, as illustrated in Fig. 5. However, their roster of risk factors is decidedly incomplete. Omitted entirely are what they call social factors, which, though complicated to study, must surely be as important for improving health as biomedical interventions such as use of pneumococcal vaccine. The importance of social context was emphasized by Zimicki and colleagues in one of the Annexes of the Ad Hoc Committees report, but it is not formally worked into the model (6). Perhaps poverty is too large an issue for health researchers to take on alone, but Fig. 6 illustrates with dotted lines the beneficial effects that could be expected in a variety of health problems from research on such matters as the status of women or care-seeking behaviour. The point of this illustration is not to argue, for example, that the increased status of women would reduce the unavertable burden of suicide but not of acute diarrhoea (which may or may not be the case), but rather to indicate that a cross-cutting determinant can affect the burden of many different health problems.
The three foregoing examples of research priorities are too easily overlooked, in part because they cross disciplinary lines. Health problems that do not fit neatly within one discipline are likely to be studied last, not because they are less important but because they challenge current conceptions of causation and intervention. Adjustments must therefore be made so that the selected priorities reflect the health needs of the people rather than the health concepts of the professionals.
A fourth example of research that may be undervalued by the global view taken by the Ad Hoc Committee relates to health problems that vary greatly across geographical boundaries. Local variations in disease burden and available resources for intervention, as well as obstacles to application of those interventions, may greatly influence the need and opportunity for interventions. The distinctions may be blurred, however, when the local situations across the entire globe are aggregated. Consider, for example, the situation of Haemophilus influenzae type b (Hib), which used to be the most common cause of bacterial meningitis in the USA. Prior to the development of conjugated Hib vaccine, Hib meningitis affected 0.2% of under-five-year-olds in the 48 contiguous states of the USA, but ten times that percentage in the indigenous population of Alaska. With the introduction and widespread use of Hib vaccine, the rates dropped precipitously, as illustrated schematically in Fig. 7. However, little is known about the incidence of Hib disease in Asia and little more is known about it in Africa; the vaccine is barely used in both regions. A high priority could therefore be given to gathering local information about Hib to guide the use of this highly effective intervention, where needed.
Research priority setting
Priority must be given to research that is most likely to improve health. In judging the most promising areas for research, problems that cause the greatest burden and interventions that can be applied cost-effectively should clearly be favoured, as emphasized by the Ad Hoc Committee. However, in situations of uncertainty or complexity, a method of priority-setting that emphasizes certainty and simplicity is misleading. Cooper and colleagues (7) questioned the applicability of the Global Burden of Disease exercise to the continent of Africa, given the limitations of vital statistics there. I would go further and suggest a range of situations in which the needed researchwould be overlooked if we relied solely on the Ad Hoc Committees priority-setting process. In research priority-setting, greater weight should be placed on emerging health problems, ways to sustain disease control efforts, cross-disciplinary factors that affect multiple diseases, and local variations in disease occurrence and control.
How then might one take a more complex range of factors into account in setting research priorities (without reverting to the unsatisfactory practice of wise people just sitting around a table and offering their favourite research topic)? Attempts to build consensus such as the Delphi technique among experts may well have value, although one would like the criteria for assessing candidate research projects to be explicit enough for an outside critic to assess how changes in the criteria used might affect the priorities reached by the consensus-building process. One of the annexes of the Ad Hoc Committees report, in fact, discusses two examples of the use of a modified Delphi technique, and shows whether the decisions reached are influenced by inclusion or exclusion of criteria related to feasibility, impact, and availability of expertise (6). The details of those inputs, however, are not published, so it may be useful to examine in some detail another example of research priority-setting that was designed both to accommodate the complexities of real-life circumstances and to be fully transparent in describing the criteria used.
Example of priority-setting for malaria
INCLEN recently completed a year-long, highly democratic strategic planning process which concluded, among other things, that it should focus in the coming years on research and capacity-building activities that centred on any of nine particular health challenges (called best prospects). One of these nine was malaria. As part of the effort to implement its strategic plan, INCLEN set out to develop its own malaria strategy, with a particular focus on Africa. A group of INCLEN faculty members with a particular interest in malaria was convened for a two-day workshop in Nairobi, Kenya, in July 1999. They discussed international efforts in malaria control such as Roll Back Malaria, the Multilateral Initiative in Malaria, Medicines in Malaria Venture, and the African Initiative in Malaria; national malaria control activities in their own countries; and ways in which the INCLEN faculty might best contribute to these efforts through research and capacity-building. In this effort the group was helped by an outside consultant, Professor Donald Krogstad of Tulane University (USA), while the author of this paper acted as facilitator.
The workshop participants made a menu of potential malaria research and training activities, which they thought deserved careful consideration (Table 1). These 25 activities were divided into the following categories: research on the epidemiology of malaria (8 activities), research on disease management (9), research on malaria prevention (5), training and capacity-building (2), and the shaping of policy (1). They also agreed on eight criteria that might reasonably be used to judge the candidate activities (Table 2). These criteria fell into three groups: the four criteria that had been used in INCLENs strategic planning process to select the best prospects; the three criteria which the workshop participants judged were underlying WHOs programmes in Roll Back Malaria; and a global criterion of feasibility. All the candidate activities were scored according to the eight criteria, ranging from 5 (most favourable) to 1 (least favourable).
Each participant was given full freedom to offer during the group discussions his or her score on a given candidate criterion combination. Discussion was encouraged (and the debate was often spirited), but consensus on each candidate criterion combination was not required. When differences of opinion persisted after discussion, a range of scores was indicated in the scoring matrix (Table 3), where that range captured the lowest and highest scores offered by individual workshop participants. The preliminary ranking of candidate activities reflected the simple sum of individual scores across the eight criteria (using the arithmetical mean of lowand high scores in instances where there were different opinions). The group was then asked to reflect on the resulting ranking to determine whether the various criteria should be weighted differentially. Once the group had agreed that equal weighting seemed reasonable, the ranked activities were grouped in tiers which the workshop participants judged were based on the right cut-off points between projects of first priority, projects of second priority, and projects that should not be pursued further at the present time. Individual workshop participants were then assigned to write concept papers on projects ranked as first priority, under the guidance of Dr James Hakim of the University of Zimbabwe and Dr Dorothy Mbori-Ngacha of the University of Nairobi, who were chosen as Coordinator and Deputy Coordinator, respectively, of INCLENs malaria initiative.
This priority-setting process worked well, insofar as it enabled people with different views to reach satisfactory decisions quickly. Given the wide selection of criteria by which to judge the candidate activities, the process is well suited for adaptation to a wide range of situations, depending on local health conditions and resources. However, this adaptability could be a weakness as well as a strength, because if the process is so adaptable to local situations, each application might be idiosyncratic and no general lessons could be drawn. However that may be, it is worth noting that, in subsequent discussions, many international health officials from various organizations considered the projects given first priority in the Nairobi workshop were fully in line with international thinking about malaria priorities.
The 5-step priority-setting process outlined by the Ad Hoc Committee could, therefore, usefully be adapted to situations where it now does not work well by considering additional criteria in an explicit and transparent way. Using a matrix-based method of the sort used in the INCLEN Nairobi workshop, one might include the following as criteria for judging candidate research areas:
the amounts of disease burden that could be averted cost-effectively, that could be averted but not cost-effectively, and that cannot now be averted (illustrated diagrammatically by (b), (c) and (d), respectively, in Fig. 1 and in the Ad Hoc Committees report);
the last 1½ steps of the 5-step process (representing the probability of success and adequacy of current level of effort);
such important sources of uncertainty and complexity as the risk of emerging health problems, the importance of sustaining disease control efforts, cross-disciplinary factors that affect multiple diseases, and local variations in disease occurrence and control.
Use of this method would be very likely to lead to more nuanced decision-making in situations of uncertainty and complexity. 
References
1. Ad Hoc Committee on Health Research Relating to Future Intervention Options. Investing in health research and development. Geneva, World Health Organization, 1996 (unpublished document TDR/Gen/96.1).
2. Hassan FA et al. Child physical maltreatment in the school and at home. Journal of Clinical Epidemiology, 1997, 50 (Suppl.): 40S.
3. Ramiro LS. Conceptual, methodological and ethical issues in community studies of child abuse and neglect in the Philippine setting. Journal of Clinical Epidemiology, 1998, 50 (Suppl.): 38S.
4. Central Coordinating Team. AIIMS-INCLEN pulse polio immunization program evaluation 199798. New Delhi, Bhumica, 1999.
5. Murray CJL, Lopez AD. Quantifying the burden of disease and injury attributable to ten major risk factors. In: Murray CJL, Lopez AD, eds. The Global Burden of Disease: a comprehensive assessment of mortality and disability from diseases, injuries and risk factors in 1990 and projected to 2020. Cambridge, MA,Harvard School of Public Health on behalf of the World Health Organization and the World Bank, 1996 (Global Burden of Disease and Injury Series, Vol. I).
6. Zimicki S, Duran-Gonzalez L, Becerra JA, Evans DB. The development of priorities for research relating to households and populations. In: Ad Hoc Committee on Health Research Relating to Future Intervention Options. Investing in health research and development. Geneva, World Health Organization, 1996 (unpublished document TDR/Gen/96.1): 259269.
7. Cooper RS et al. Disease burden in sub-Saharan Africa: what should we conclude in the absence of data? Lancet, 1998, 351: 208221.