POLICY AND PRACTICE
Assessing country-level efforts to link research to action
Évaluation à l'échelle d'un pays des efforts pour lier recherche et action
Evaluar los esfuerzos realizados en los países para vincular las investigaciones a la acción
John N LavisI,1; Jonathan LomasII; Maimunah HamidIII; Nelson K SewankamboIV
IMcMaster University, HSC-2D3, 1200 Main Street West, Hamilton, Ontario, Canada L8N 3Z5
IICanadian Health Services Research Foundation, Ottawa, Ontario, Canada
IIIInstitute for Health Systems Research, Ministry of Health, 59000 Kuala Lumpur, Malaysia
IVFaculty of Medicine, Makerere University, Kampala, Uganda
ABSTRACT
We developed a framework for assessing country-level efforts to link research to action. The framework has four elements. The first element assesses the general climate (how those who fund research, universities, researchers and users of research support or place value on efforts to link research to action). The second element addresses the production of research (how priority setting ensures that users' needs are identified and how scoping reviews, systematic reviews and single studies are undertaken to address these needs). The third element addresses the mix of four clusters of activities used to link research to action. These include push efforts (how strategies are used to support action based on the messages arising from research), efforts to facilitate "user pull" (how "one-stop shopping" is provided for optimally packaged high-quality reviews either alone or as part of a national electronic library for health, how these reviews are profiled during "teachable moments" such as intense media coverage, and how rapid-response units meet users' needs for the best research), "user pull" efforts undertaken by those who use research (how users assess their capacity to use research and how structures and processes are changed to support the use of research) and exchange efforts (how meaningful partnerships between researchers and users help them to jointly ask and answer relevant questions). The fourth element addresses approaches to evaluation (how support is provided for rigorous evaluations of efforts to link research to action).
RÉSUMÉ
Un cadre permettant d'évaluer à l'échelle d'un pays les efforts pour lier recherche et action a été mis au point. Ce cadre comporte quatre composantes. La première évalue le contexte général (quel appui ou quelle valeur les acteurs qui financent la recherche, les universités, les chercheurs et les utilisateurs des données de recherche accordent-ils aux efforts pour relier recherche et action). La deuxième composante s'intéresse à la production de la recherche (dans quelle mesure la définition des priorités garantit-elle l'identification des besoins des utilisateurs et dans quelle mesure également des analyses de portée, des revues systématiques et des études simples sont-elles entreprises pour répondre à ces besoins). La troisième composante traite de l'utilisation d'une combinaison d'activités relevant de quatre groupes pour établir un lien entre recherche et action. Ces activités comprennent les efforts de type «push» (comment mettre en uvre des stratégies pour étayer les interventions par des messages en provenance de la recherche), ceux visant à faciliter l'extraction d'informations par les utilisateurs («user pull») [comment s'opère l'approvisionnement à une source unique à partir de revues de haute qualité, présentées de manière optimale, isolément ou dans le cadre d'une librairie électronique nationale en faveur de la santé, comment l'attention est attirée sur ces revues pendant les «fenêtres d'enseignement» (couverture médiatique intense, par exemple) et comment les unités de réponse rapide répondent aux besoins des utilisateurs en matière de recherche de qualité optimale], les efforts de type «user pull» entrepris par les utilisateurs de la recherche (comment ces utilisateurs évaluent-ils leur capacité à exploiter les données de recherche et comment modifier les structures et les processus pour appuyer l'utilisation de la recherche), ainsi que les efforts d'échange (comment des partenariats judicieusement organisés entre chercheurs et utilisateurs aident les uns et les autres à poser des questions pertinentes et à y répondre). La quatrième composante concerne les méthodes d'évaluation (comment apporter un soutien en faveur d'évaluations rigoureuses des efforts pour lier recherche et intervention).
RESUMEN
Elaboramos un marco para evaluar los esfuerzos desplegados a nivel de país para vincular las investigaciones a la acción. El marco consta de cuatro elementos. El primero evalúa el clima general (de qué manera quienes financian las investigaciones, las universidades, los investigadores y los usuarios de las investigaciones respaldan o valoran los esfuerzos realizados para vincular las investigaciones a la acción). El segundo aborda la producción de investigaciones (cómo mediante el establecimiento de prioridades se asegura que se identifiquen las necesidades de los usuarios, y cómo llevar a cabo revisiones sintéticas, revisiones sistemáticas y estudios independientes para responder a esas necesidades). El tercer elemento aborda la combinación de cuatro tipos de actividades orientadas a vincular las investigaciones a la acción: actividades impulsoras (formas de aplicación de estrategias para apoyar las medidas basadas en los mensajes derivados de las investigaciones), actividades de fomento de la «atracción de usuarios» (por ejemplo una «ventanilla única» para conseguir combinaciones óptimas de revisiones de calidad, ya sea de forma independiente o como parte de una biblioteca electrónica nacional de salud; la reseña de esas revisiones durante las «oportunidades didácticas», como una intensa cobertura mediática; y la actuación de unidades de respuesta rápida en respuesta a las necesidades de los usuarios para optimizar las investigaciones), las actividades de «atracción de usuarios» emprendidas por quienes utilizan las investigaciones (análisis de cómo los usuarios evalúan su capacidad para usar las investigaciones, y de la manera de modificar las estructuras y los procedimientos para fomentar el uso de las mismas), y las actividades de intercambio (determinación de cómo las fórmulas de colaboración más eficaces entre investigadores y usuarios les ayudan a dar forma y responder conjuntamente a los interrogantes pertinentes). El cuarto elemento aborda los métodos de evaluación (la prestación de apoyo para la realización de evaluaciones rigurosas de las actividades de vinculación de las investigaciones a la acción).
Introduction
The idea of linking research to action in the health sector has captured a great deal of international attention. In late 2004 WHO issued the World report on knowledge for better health, with a chapter devoted to linking research to action.1 Shortly thereafter WHO convened the Ministerial Summit on Health Research in Mexico City, and the resulting Mexico statement on health research called on "all major stakeholders to strengthen or to establish activities to communicate, improve access to, and promote the use of reliable, relevant, unbiased and timely health information."2 In May 2005 the World Health Assembly called on WHO Member States to "establish or strengthen mechanisms to transfer knowledge in support of evidence-based public health and healthcare delivery systems, and evidence-based health-related policies"; it also called on WHO's Director-General to "assist in the development of more effective mechanisms to bridge the divide between ways in which knowledge is generated and ways in which it is used, including the transformation of health-research findings into policy and practice."3
But statements and resolutions are easier made than acted on. Those who want to take meaningful steps to link research to action would ideally be able to draw on high quality, locally applicable research to inform their efforts. Those interested in linking research to action in clinical environments can draw on an overview of systematic reviews of randomized trials of interventions designed to better align health-care professionals' practise with research4 as well as a systematic review of randomized controlled trials of strategies for guideline dissemination and implementation.5 Most of the studies included in these reviews were conducted in high-income countries. However, these interventions should also be evaluated in low- and middle-income countries.6 Those interested in linking research to action in the areas of health management and policy-making have to deduce the attributes of potential interventions from systematic reviews of observational studies that examine the factors that influence the use of research.7,8 For example, interactions between researchers and health-care policy-makers and the timing or timeliness of research being made available appear to increase the likelihood that research will be used by policy-makers. Hence, interventions such as interactive workshops that bring together researchers and health-care policy-makers and web sites that provide "one-stop shopping" for systematic reviews have been promoted (but not yet evaluated).
Health-care professionals, managers and policy-makers are not the only people who may use research. The full range of potential users (outside the research community itself ) includes the general public, patients, health-care professionals, health managers, executives of biomedical companies and public policy-makers. (While being someone who uses research may be a small role for members of these groups, we use the term "research user" throughout as shorthand for these groups.) Some intermediary groups by which we mean the media, civil society groups, professional associations and other groups that work at the interface between researchers and users of research may also have critically important roles in linking research to action.
In this paper we develop a framework for assessing country-level efforts to link research to action. The main purpose of the framework is to inform country-level dialogues about the domains to which attention could be directed in order to link research to action. Countries provide a natural unit for assessment given that there may be a division of labour within a country (for example, among research funders). A country's capacities and constraints will affect the initial focus of their efforts (for example, creating a demand for research may be one of the first steps for some countries whereas integrating and building on existing efforts may be among the first steps for others). Capacities will also affect a country's requirements for partnerships beyond its borders (for example, regional initiatives may provide economies of scale and global initiatives may provide support). When discussing efforts to link research to action within a country, the term "action" includes maintaining the status quo, which can sometimes take as much effort or more than bringing about change. The framework does not include action in the form of commercialization, however, because the different context brought about by the profit motive warrants separate treatment.
Given the state of the research in this field, many elements of the framework are based only indirectly on research. If countries select and implement one or more options from the framework among particular groups and evaluate them rigorously then 510 years from now we should be in a much better position to refine the framework and ensure that its elements are solidly supported by research. Had such evaluations been undertaken for the frameworks that have already been developed for policy-makers,913 we would have a more rigorous research base from which to draw now. The main purpose of the examples offered here is to highlight how the framework is grounded in promising innovations that are being implemented for one or more user groups. These innovations warrant wider consideration and rigorous evaluation.
Key concepts
Talk of linking research to action often puts fear into those who prefer research to be focused on discovery rather than application. But these two goals need not be in conflict. Discoveries made today are essential to the applications of tomorrow, and making efforts to forge better links between discovery and application is essential to reaping the benefits of investment in discovery.14,15 Many researchers develop innovations in basic science, theory and methodology. Researchers draw on these innovations to conduct studies and write articles and reports. And then later, researchers draw on individual contributions to produce systematic reviews. A fraction of these systematic reviews (and, occasionally, single studies) will yield actionable messages for one or more categories of potential research users; at other times the reviews will yield calls for more or different types of research. Talk of linking research to action is then, in part, simply talk about focusing our efforts to link research to action at the apex of the knowledge pyramid while continuing to build a solid base for the pyramid (Fig. 1). Discovery and application are interdependent points along a continuum; they are not competing objectives for the role of research in society.
The funders and producers of discovery-oriented research, who function in an environment where others have taken the lead on linking research to action outside the research community, can presumably remain focused on discovery and, when appropriate, on linking research to action by researchers.16 These researchers will then be better positioned to explore applications of basic science or create derivative theoretical and methodological innovations. The funders and producers of discovery-oriented research, who function in an environment where others have not taken the lead on linking research to action, face a greater challenge. Those who disburse public funds to these funders and researchers may well ask them why at least some of the disbursements are not allocated to linking research to action outside the research community.
Systematic reviews
For those interested in applying research, the challenge is different: there is a need to collectively create music, not noise, for the select ears of research users. Faced with a desire to link research to action, the first impulse of many funders and researchers is to confuse marketing with targeted assistance by promoting all research, regardless of its contribution to the global pool of knowledge. But funders and researchers who market single studies, articles and reports can do harm. Take, for example, the issue of whether to allow private for-profit hospitals to compete with private not-for-profit hospitals. A systematic review of studies comparing mortality rates in private for-profit hospitals with those in private not-for-profit hospitals found 15 studies that met the eligibility criteria; all but one study found a survival advantage for being treated in not-for-profit hospitals.17 Funders and researchers who promoted the introduction of for-profit hospitals on the basis of the single study with a different result were potentially doing harm.
For potential research users, systematic reviews offer four advantages over single studies; the first two advantages apply primarily to meta-analyses that address the question "what works?" First, the likelihood of being misled by research is lower with a systematic review than with an individual study (that is, bias is reduced).18 Second, confidence in what can be expected from an intervention is higher with a systematic review than with an individual study (that is, precision is increased).18 Third, drawing on an existing systematic review constitutes a more efficient use of time because the research literature has already been identified, selected, appraised and synthesized in a systematic and transparent way;8 potential research users can thus focus on assessing the local applicability of a review and on collecting and synthesizing other types of information, such as routine health information. Fourth, a systematic review can be more constructively contested than an individual study because debate will focus on appraisal and synthesis rather than on why one study was identified and selected over others.8
Systematic review methodologies increasingly lend themselves to providing responses to the many different types of questions asked by different potential research users. For example, health managers and policy-makers are interested in finding the most effective solutions to the most burdensome health problems, the most effective ways to fit these solutions into complex health systems (or the most effective ways to organize health systems) and the most effective ways to bring about desired changes in health systems.19 But in addition to asking questions about effectiveness (does changing X change Y?), they also ask questions about cost effectiveness (is X1 more cost effective than X2 in achieving a one-unit change in Y?), relationships (is X associated with Y?), mechanisms (how are X and Y linked or why does changing X change Y?), and meaning (how have individuals viewed or experienced X or Y?). A variety of new approaches have been developed to conduct and update syntheses of research to address this broad array of questions.2023 Although not all developers or users of these new approaches agree with labelling them systematic reviews, for convenience we use that label here as a generic term for all these forms. However, there remain legitimate and important differences in perspectives about a number of these approaches and these warrant further debate.24,25
Approaches to link research to action
While actionable messages arising from systematic reviews may be the natural unit of research to consider when attempting to link research to action, people are still needed to make these links. Four approaches can be employed, either singly or in combination, to link research to action (Fig. 2).1,26 "Push" efforts are led by researchers, intermediary groups and other purveyors of research (such as communications staff).27 Such efforts are well suited to situations where the potential research users are unaware that they should be considering a particular message (or in some cases would prefer to continue to disregard it). "User-pull" efforts involve patients, health-care professionals, civil servants and others "reaching in" to the research world to extract information for a decision that they face.28 Researchers and intermediary groups can facilitate these efforts by improving access to optimally packaged research that is of high quality and relevance. User-pull efforts are well suited to situations where potential users have identified an information gap and want to address the gap in a timely way. Exchange (or "linkage and exchange") efforts occur when the producers or purveyors of research develop a partnership with a group who uses research.29 Such partnerships are well suited to situations where the two groups can establish a shared understanding about the questions to ask, how to answer them through a systematic review or as part of a research project or programme, and how to weight the research and other types of information that each group brings to the table.
A fourth approach, which integrates efforts through large-scale knowledge-translation platforms, includes elements of the push, pull and exchange approaches. For example, a proposal for the Regional East African Community Health (REACH)-Policy initiative includes:
· a governing board comprising representatives from groups of producers, purveyors and users (that is, an exchange approach);
· a clear goal (that is, improving people's health and health equity in east Africa through the more effective use and application of knowledge to strengthen health policy and practice);
· regular priority-setting processes to ensure that systematic reviews and efforts to link research to action are highly relevant to the needs of potential research users;
· push efforts in areas where actionable messages have been identified; and
· a range of efforts to facilitate user pull (such as one-stop shopping for optimally packaged systematic reviews of high quality and relevance, and a rapid-response unit that provides written summaries, telephone consultations or in-person consultations about the best research in a timely way).
There is more than one approach to linking research to action, and not every approach will work in all situations. For example, a health-care professional with a patient in her office or a senior civil servant who has to brief the health minister in 5 minutes cannot wait for a push effort or a partnership. Similarly, the five researchers who study a particular issue in a country cannot respond to all phone calls from, or develop partnerships with, every clinic or hospital. A community health centre that wants to undertake community-based research to inform its strategic direction while developing local capacity to produce and use research may not be well served by research from other communities that have similar socioeconomic and ethnocultural profiles. Moreover, a single knowledge-translation platform will still leave many actionable messages unused and many user groups without service. Given the failings of any one approach operating in isolation, there is great value in using all four approaches simultaneously.
Framework for assessing country-level efforts
The proposed framework has four elements for assessing country-level efforts to link research to action: the general climate for research use, the production of research that is both highly relevant to and appropriately synthesized for research users, the mix of clusters of activities used to link research to action, and the evaluation of efforts to link research to action (Table 1). When discussing the mix of activity clusters, elements of the integrated approach are discussed in the context of the other three approaches. Additionally, the user-pull approach is separated into those activities that can be undertaken by the producers or purveyors of research (for example, efforts to facilitate pull by appropriately packaging key messages of the research) and those activities that can be undertaken only by research users (for example, revising decision-making processes to include explicit consideration of research).
In the first element of the framework we posit that the general climate is conducive to linking research to action when the following conditions are met:
· at least some funders have a mandate to support efforts to link research to action and they support these efforts in several ways;
· universities and other research institutions consider such efforts to be in their tenure and promotion processes and work to remove disincentives to link research to action;
· some researchers place value both on promoting the use of research and if they are to establish mutually respectful partnerships with research users on the other types of information on which research users regularly draw (for example, public policy-makers draw on legal evidence about institutional constraints, civil servants' assessments of stakeholders' interests and public opinion polls); and
· intermediary groups and research usep ers place value on the use of research.
The funding environment poses particular challenges in many low- and middle-income countries because funding may come not only from national bodies but also from international development aid, research institutes in high-income countries or international foundations and agencies, all of which may have broad development, health or international goals rather than nationally focused health research goals.
The second element of the framework focuses on the production of research. We posit that the research enterprise is conducive to linking research to action when the following conditions are met:
· some funders periodically engage potential research users in priority-setting processes,30 commission or fund scoping reviews to fill information gaps in areas identified as priorities by users, support the production and regular updating of systematic reviews and, when appropriate, fund single studies;
· funders and ethics review boards place value on systematic reviews to justify additional research on a topic; and
· some researchers participate in continuing education programmes to develop their capacity to conduct systematic reviews and respond to the calls for research in priority areas as well as committing to updating regularly systematic reviews they are funded to produce.
The emphasis on production is particularly important in low- and middle-income countries where, apart from a few fields and especially in applied fields,31 there is a dearth of high quality research that can be linked to action.32,33 Our emphasis on "some" funders and researchers, rather than on "all", is to ensure that gains can be achieved without jeopardizing the innovations in basic science, theory and methodology that form the base of the knowledge pyramid.
The third element in the framework addresses the mix of activity clusters used to link research to action. Push efforts are also likely to be conducive to linking research to action when some funders, researchers or intermediary groups, or some combination of these, engage in the following components of a systematic push effort:
· periodically identify actionable messages arising from systematic reviews (or occasionally from single studies when a strong case can be made for their unique contributions);
· fine-tune messages and related resources for different user group;
· work with and through the most credible messengers for each user group;
· use research-informed strategies to encourage and support action based on the messages; and
· evaluate their impact against achievable objectives.27
As emphasized in the Introduction, the degree to which strategies to encourage and support action can be based on research varies by user group. Push efforts are also likely to be conducive to linking research to action when some funders, researchers or intermediary groups, or some combination of these, develop media releases for systematic reviews (rather than only for articles and reports based on single studies) as is now being done by the United States-based Center for the Advancement of Health, when some researchers employ self-assessment tools to evaluate their capacity to develop and execute research-informed push efforts, and when some researchers participate in continuing education programmes to develop these capacities.
We posit that efforts to facilitate user pull are likely to be more conducive to linking research to action when some funders, researchers or intermediary groups, or some combination of these:
· provide one-stop shopping for optimally packaged reviews that are of high quality and relevance (either in stand-alone format as is done through initiatives like the Health Evidence Network of WHO's Regional Office for Europe and WHO's Reproductive Health Library or as part of a national electronic library for health as has been done in the United Kingdom);
· profile these reviews during "teachable moments" (for example, as is done for physicians in the United Kingdom through the "Hitting the Headlines" service of the National Library for Health);
· administer rapid-response units of the type described above (as is done for policy-makers through the European Observatory on Health Systems and Policies); and
· participate in programmes to enhance their capacity to develop and execute efforts to facilitate user pull.
Because studies conducted within particular health systems or particular populations in countries may have limited applicability to other health systems or populations, we posit that efforts both to push and to facilitate user pull will need to highlight the factors that influence the local applicability of systematic reviews.19,34
User-pull efforts are likely to be conducive to linking research to action when some research users:
· employ self-assessment tools to evaluate their capacity to acquire, assess, adapt and apply research (such as the tool developed by the Canadian Health Services Research Foundation);28
· develop structures and processes to help them use and promote research (for example, policy-makers may require that submissions make explicit the research and routine health information underlying any assessments of options); and
· participate in skill-development programmes to enhance their capacity to use and promote research (such as the Executive Training for Research Application programme developed by the Canadian Health Services Research Foundation).
Exchange efforts are likely to be more conducive to linking research to action when they are:
· personal and ongoing, thus creating a window onto the research world for the potential research user (beyond the specific study, research programme or systematic review around which the exchange is occurring) and a window onto the health system for the researcher (facilitating an understanding of the cultural and other differences between the contexts of researchers and research users);
· based on a meaningful partnership where the relative roles and expertise of researchers and research users are recognized; and
· supported by skill-development programmes that allow participating researchers and research users to develop their capacity to engage in mutually beneficial partnerships.
These exchange relationships may be developed and nurtured by trusted individuals or organizations acting in the role of what have been called "knowledge brokers".35 The US-based Agency for Healthcare Research and Quality has paid particular attention to developing partnerships focused on systematic reviews,36 whereas other funders have been more focused on building partnerships around single studies or research programmes.29
For the last element in the framework we posit that evaluations are likely to support future efforts to link research to action when:
· funding is available for evaluations of large-scale natural or planned innovations to link research to action; and
· funders, researchers, intermediary groups and user groups participate in rigorous evaluations of efforts to link research to action.
Weak evaluation designs that do not examine both the intended and unintended consequences of these innovations will not provide the research needed to inform future efforts to link research to action.
Applying the framework
No country can or should undertake every component of the four elements in the framework, especially in the domains of producing research and facilitating user pull. The production of systematic reviews, for example, is a global responsibility even if their interpretation and translation into actionable messages is best done at the local level. For example, a systematic review conducted in Malaysia may need little adaptation beyond translation before it can be used in deliberative forums and decision-making venues in Thailand. Some forms of facilitating user pull are also a global responsibility. The Cochrane Library, for example, provides one-stop shopping for high quality systematic reviews addressing the question of "what works?" National electronic libraries for health can provide links to this global resource, which can often be accessed for free by individuals in low- and middle-income countries through, for example, the Latin American and Caribbean Center on Health Sciences Information (BIREME), the Health InterNetwork Access to Research Initiative (HINARI), and the International Network for the Availability of Scientific Publications (INASP).
A number of initiatives have attempted to address one or two elements of the framework. For example, the Applied Diarrhoeal Disease Project and the Joint Health Research Systems Project for southern Africa were initiatives that focused primarily on producing highly relevant research and undertaking push efforts. Both emphasized the creation of studies relevant to health policy and systems through the inclusion of policy-makers, managers or health-care professionals in the research-planning process and the dissemination of results, which was achieved through targeting presentations of findings towards those who could act on them.37 The International Clinical Epidemiology Network's Knowledge "Plus" Program focuses primarily on push efforts involving clinical practise guidelines. The International Health Policy Program focused primarily on exchange efforts by pairing young researchers with policy-makers in order to strengthen linkages and inform health policy. The programme relied less on universities than on stand-alone organizations with research capacity.38 As part of its new strategic plan, the Alliance for Health Policy and Systems Research has articulated its intent to focus on all elements of the framework.39
Three regional initiatives that offer great promise are in development. The Regional East African Community Health-Policy initiative proposes to establish a unit located within the East African Health Research Council to address all elements of the framework for Kenya, Uganda and the United Republic of Tanzania. The Western Pacific Region of WHO has launched the planning stages of a project known as Evidence Informed Policy Networks (EVIPNet Asia) to address many elements of the framework for one municipality in China (Beijing), two Chinese provinces (Shandong and Sichuan), the Lao People's Democratic Republic, Malaysia, and Viet Nam.40 The African Region of WHO has launched a similar programme, known as EVIPNet Africa. The speed with which these regional initiatives are moving suggests they will pioneer new integrated knowledge-translation platforms well before other parts of the world.
Conclusions
The framework for assessing country-level efforts was designed to inform country-level dialogues about the options for linking research to action for different groups of users. Many elements of the framework are based only indirectly on research. To push forward our understanding of these elements and their interrelationships, we should rigorously evaluate innovative country-level efforts targeted at particular user groups. An important first step in any country will be to identify for specific user groups the supportive elements that are already in place (and that warrant evaluation) as well as the unsupportive elements that should be addressed and the as-yet-unassessed elements that should be examined. Initiatives that are under development in sub-Saharan Africa and the western Pacific may provide opportunities to test the framework in a comprehensive way. 
Funding: John Lavis receives salary support as the Canada Research Chair in Knowledge Transfer and Exchange.
Competing interests: none declared.
References
1. World Health Organization. World report on knowledge for better health. Geneva: WHO; 2004.
2. World Health Organization. The Mexico statement on health research, 2004. Available from: http://www.who.int/rpc/summit/agenda/Mexico_Statement-English.pdf
3. World Health Assembly. Resolution on health research, 2005. Available from: http://www.who.int/rpc/meetings/58th_wha_resolution.pdf
4. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero LA, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care 2001;39 Suppl 2:S2-45.
5. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:1-72.
6. Haines A, Kuruvilla S, Borchert M. Bridging the implementation gap between knowledge and action for health. Bull World Health Organ 2004;82:724-33.
7. Innvaer S, Vist GE, Trommald M, Oxman AD. Health policy-makers' perceptions of their use of evidence: a systematic review. J Health Serv Res Policy 2002;7:239-44.
8. Lavis JN, Davies HTO, Oxman AD, Denis J-L, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy 2005;10 Suppl 1:S35-48.
9. Frenk J. Balancing relevance and excellence: organizational responses to link research with decision making. Soc Sci Med 1992;35:1397-404.
10. Sauerborn R, Nitayarumphong S, Gerhardus A. Strategies to enhance the use of health systems research for health sector reform. Trop Med Int Health 1999;4:827-35.
11. Chunharas S. Linking research to policy and action. In: Neufeld V, Johnson N, editors. Forging links for health research: perspectives from the Council on Health Research for Development. Ottawa: International Development Research Centre; 2001. p. 109-39.
12. Alliance for Health Policy and Systems Research. Getting research into policy and practice. In: Strengthening health systems: the role and promise of policy and systems research. Geneva: Alliance for Health Policy and Systems Research; 2004. p. 51-70.
13. Court J, Young J. Bridging research and policy in international development: context, evidence and links. In: Stone D, Maxwell S, editors. Global knowledge networks and international development. Routledge; 2004. p. 1-24.
14. International Organizing Committee. International conference on health research for development. Bangkok: International Organizing Committee; 2000.
15. International Working Party to Promote and Revitalise Academic Medicine. ICRAM (the International Campaign to Revitalize Academic Medicine): agenda setting. BMJ 2004;329:787-9.
16. Ravensbergen J, Lomas J. Creating a culture of research implementation: ZonMw in the Netherlands. Global Forum Update on Research for Health 2005;2:64-6.
17. Devereaux PJ, Choi PTL, Lacchetti C, Weaver B, Schunemann HJ, Haines T, et al. A systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals. CMAJ 2002;166:1399-406.
18. Egger M, Smith GD, O'Rourke K. Rationale, potentials, and promise of systematic reviews. In: Egger M, Smith GD, Altman DG, editors. Systematic reviews in health care: meta-analysis in context. 2nd ed. London: BMJ Books; 2001. p. 3-19.
19. Lavis JN, Posada FB, Haines A, Osei E. Use of research to inform public policymaking. Lancet 2004;364:1615-21.
20. Bravata DM, McDonald KM, Shojania KG, Sundaram V, Owens DK. Challenges in systematic reviews: synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005;142 (Part 2):1056-65.
21. Pignone M, Saha S, Hoerger T, Lohr KN, Teutsch S, Mandelblatt J. Challenges in systematic reviews of economic analyses. Ann Intern Med 2005;142 (Part 2):1073-9.
22. Chou R, Helfand M. Challenges in systematic reviews that assess treatment harms. Ann Intern Med 2005;142 (Part 2):1090-9.
23. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005;10:45-53.
24. Lomas J. Using research to inform healthcare managers' and policy makers' questions: from summative to interpretive synthesis. Healthcare Policy 2005;1:55-71.
25. Lavis JN, Davies HTO, Gruen RL, Walshe K, Farquhar CM. Working within and beyond the Cochrane Collaboration to make systematic reviews more useful to healthcare managers and policymakers. Healthcare Policy 2006;1:21-33.
26. Lavis JN. Bridging the know-do gap. Hamilton: McMaster University Centre for Health Economics and Policy Analysis; 2004.
27. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003;81:221-48.
28. Canadian Health Services Research Foundation. Is research working for you? A self-assessment tool and discussion guide for health services and policy organizations. Ottawa: Canadian Health Services Research Foundation; 2005.
29. Lomas J. Using 'linkage and exchange' to move research into policy at a Canadian foundation: encouraging partnerships between researchers and policymakers is the goal of a promising new Canadian initiative. Health Aff 2000;19:236-40.
30. Lomas J, Fulop N, Gagnon D, Allen P. On being a good listener: setting priorities for applied health services research. Milbank Q 2003;81:363-88.
31. Haines A, Victora C, Horton R. The Lancet's series on health-systems research: a call for papers. Lancet 2004;363:261-2.
32. Horton R. Medical journals: evidence of bias against the diseases of poverty. Lancet 2003;361:712-3.
33. Rochon PA, Mashari A, Cohen A, Misra R, Laxer D, Streiner DL, et al. Relation between randomized controlled trials published in leading general medical journals and the global burden of disease. CMAJ 2004;170:1673-7.
34. Gruen RL, Morris PS, McDonald EL, Bailie RS. Making systematic reviews more useful for policy-makers. Bul World Health Organ 2005;83:480-1.
35. Canadian Health Services Research Foundation. The theory and practice of knowledge brokering in Canada's health system. Ottawa: Canadian Health Services Research Foundation; 2003.
36. Atkins D, Fink K, Slutsky J. Better information for better health care: the evidence-based practice center program and the Agency for Healthcare Research and Quality. Ann Intern Med 2005;142 (Part 2):1035-41.
37. Varkevisser CM, Mwaluko GMP, Le Grand A. Research in action: the training approach of the Joint Health Systems Research Project for the southern african region. Health Policy Plan 2001;16:281-91.
38. Andreano R. The international health policy program: an internal assessment. Madison (WI): University of Wisconsin Press; 2001.
39. Alliance for Health Policy and Systems Research. Knowledge for better health systems and better health. The alliance strategic plan: ten year outlook and 20062008 plan. Geneva: Alliance for Health Policy and Systems Research; 2006.
40. Hamid M, Bustamante-Manaog T, Viet Dung T, Akkhavong K, Fu H, Ma Y, et al. EVIPNet: translating the spirit of Mexico. Lancet 2005;366:1758-60.
(Submitted: 22 January 2006 Final revised version received: 17 April 2006 Accepted: 11 May 2006)
1 Correspondence to this author (email: lavisj@mcmaster.ca).