Development of the International Alcohol Policy and Injury Index

Formulación de un índice internacional de políticas en materia de alcohol y traumatismos

Elaboração da Política Internacional de Álcool e do índice de lesões

Rachael A. Korcha Jane Witbrodt Cheryl J. Cherpitel Yu Ye Gabriel Andreuccetti Jaewook Kang Maristela Monteiro About the authors

ABSTRACT

Objective

To develop a new index to measure the effectiveness of alcohol control policies on selected indicators of alcohol-related injuries.

Methods

We used the World Health Organization Global Information System on Alcohol and Health (GISAH) for cross-sectional data from 156 countries for this analysis. Five policy domains were selected: physical availability, drinking context, pricing, advertising, and vehicular. Injury mortality and alcohol-attributable fractions (AAFs) for vehicular deaths were also used for the same countries. We created a new composite indicator, the International Alcohol Policy Injury Index (IAPII), in order to assess the association between policy and deaths due to alcohol-related injury.

Results

After we controlled for per-capita alcohol consumption, we found that injury deaths and AAF deaths were inversely associated with four of the five policy domains. The domains were weighted according to effectiveness and used to construct the IAPII, which produced acceptable sensitivity and specificity. Regression results, controlling for consumption, demonstrated that the IAPII was significantly associated with AAF vehicular injury death for males, AAF vehicular injury death for females, and overall injury death at p < 0.01.

Conclusions

Our findings support the IAPII as a reliable indicator of the relationship between alcohol policies and injury deaths: the stronger the policy, the less the likelihood of both overall and vehicular injury death. Future work should test the effectiveness of the IAPII in reducing alcohol-related injury morbidity, which accounts for a larger share of the global burden of disease than alcohol-related injury mortality does.

Keywords
Ethanol; policy; wounds and injuries; death; evaluation studies

RESUMEN

Objetivo

Elaborar un nuevo índice para medir la eficacia de las políticas de control del consumo de alcohol sobre la base de indicadores seleccionados de traumatismos relacionados con el alcohol.

Métodos

En este análisis, utilizamos el Sistema Mundial de Información sobre el Alcohol y la Salud de la Organización Mundial de la Salud (GISAH) a fin de obtener datos transversales de 156 países. Se seleccionaron cinco ámbitos normativos: disponibilidad de bebidas alcohólicas, contexto del consumo de alcohol, precios, publicidad y conducción de vehículos. También se usaron datos de mortalidad por traumatismos y de “fracciones atribuibles al alcohol” de las defunciones causadas por el tránsito de los mismos países. Creamos un nuevo indicador compuesto, el índice internacional de políticas en materia de alcohol y traumatismos (IIPAL), a fin de evaluar la asociación entre la política y las muertes por traumatismos relacionados con el consumo de alcohol.

Resultados

Después de controlar la variable de consumo de alcohol per cápita, observamos que las muertes por traumatismos y las “fracciones atribuibles al alcohol” de las muertes se asociaban inversamente con cuatro de los cinco ámbitos normativos. Los ámbitos se ponderaron según la eficacia y se usaron para construir el nuevo índice, con una sensibilidad y especificidad aceptables. Los resultados de la regresión, con control del consumo de alcohol, mostraron que el índice se asociaba significativamente con la “fracción atribuible al alcohol” de muertes de hombres por traumatismos provocados por el tránsito, la “fracción atribuible al alcohol” de muertes de mujeres por traumatismos provocados por el tránsito y las muertes de personas de ambos sexos por traumatismos (p < 0,01).

Conclusiones

Nuestros resultados indican que el IIPAL es un indicador fiable de la relación entre las políticas en materia de alcohol y las muertes por traumatismos: cuanto más restrictiva la política, menor probabilidad de muertes por traumatismos en general y de muertes por traumatismos debidas al tránsito. Los futuros trabajos deberían verificar la eficacia de este índice para reducir la morbilidad por traumatismos relacionados con el alcohol, que constituyen una proporción mayor de la carga de enfermedad mundial que la mortalidad por traumatismos relacionados con el alcohol.

Palabras clave
Etanol; políticas; heridas y lesiones; muerte; estudios de evaluación

RESUMO

Objetivo

Desenvolver um novo índice para medir a efetividade das políticas de controle do uso de álcool em indicadores selecionados de lesões relacionadas ao uso de álcool.

Métodos

O Sistema Global de Informação sobre Álcool e Saúde (GISAH) da Organização Mundial da Saúde (OMS) foi usado para obter dados transversais de 156 países para esta análise. Foram selecionados cinco domínios de políticas: disponibilidade física, contexto relacionado ao uso de álcool, determinação de preços, publicidade e acidentes de trânsito. A mortalidade por lesões e a fração atribuível ao álcool (FAA) para mortes por acidentes de trânsito também foram usadas para os mesmos países. Foi criado um novo indicador composto, o índice de lesões da Política Internacional de Álcool (IAPII), para avaliar a associação entre política e mortes decorrentes de lesões relacionadas ao uso de álcool.

Resultados

Após o controle do consumo de álcool per capita, foi verificado que as mortes por lesões e a mortalidade atribuível ao consumo de álcool apresentavam associação inversa com quatro dos cinco domínios de políticas. Os domínios foram ponderados segundo efetividade e usados para construir o IAPII, que demonstrou ter sensibilidade e especificidade aceitáveis. Os resultados da análise de regressão, após controlado o consumo de álcool, revelaram uma associação significativa do IAPII com mortalidade por acidentes de trânsito atribuível ao álcool no sexo masculino, mortalidade por acidentes de trânsito atribuível ao álcool no sexo feminino e morte decorrentes de lesões em geral (p < 0,01).

Conclusões

Os achados desta análise respaldam o IAPII como um indicador confiável da relação entre as políticas de álcool e mortes decorrentes de lesões: quanto mais sólida a política, menor a probabilidade de mortes por acidentes de trânsito ou em geral. Outros estudos devem avaliar a efetividade do IAPII em reduzir a morbidade por lesões relacionadas ao uso de álcool, que representa uma parcela maior da carga global da doença que a mortalidade por lesões relacionadas ao álcool.

Palavras-chave
Etanol; políticas; ferimentos e lesões; morte; estudos de avaliação

Alcohol use is associated with a wide array of societal problems, including disease and injury. It is estimated that alcohol is the fifth leading risk factor in the Global Burden of Disease, and is first among those 15-35 years old (11. Institute for Health Metrics and Evaluation. Global Burden of Diseases, Injuries, and Risk Factors Study 2010 (GBD 2010). Seattle: University of Washington; 2013. Archived by WebCite at http://www.webcitation.org/6GRmKzeL5 Accessed 7 May 2013.
http://www.webcitation.org/6GRmKzeL5...
), as well as first in some rapidly developing countries (22. Ezzati M, Lopez AD, Rodgers A, Vander Hoorn S, Murray CJL. Selected major risk factors and global and regional burden of disease. Lancet. 2002 Nov 2;360(9343):1347-60.). Countries around the world have responded to alcohol harms in the form of laws or programs collectively known as “alcohol policies.” Broadly defined, alcohol policies are statutory measures designed to prevent or reduce alcohol-related harm (33. Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity: research and public policy. 2nd ed. New York: Oxford University Press; 2010.).

Establishing which policies are most effective at minimizing harms due to alcohol has been a topic of great attention and debate (33. Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity: research and public policy. 2nd ed. New York: Oxford University Press; 2010.55. Room R, Graham K, Rehm J, Jernigan DH, Monteiro M. Drinking and its burden in a global perspective: policy considerations and options. Eur Addict Res. 2003 Oct;9(4):165-75.). The World Health Organization (WHO) “best buys” include tax increases, restricted access to retailed alcohol, and bans on alcohol advertising as the policies that provide the most impact and cost-effectiveness (66. Bloom DE, Chisholm D, Jané-Llopis E, Prettner K, Stein A, Feigl A. From burden to “best buys”: reducing the economic impact of non-communicable diseases in low- and middle-income countries. Geneva: World Health Organization; 2011. Archived by WebCite at http://www.webcitation.org/6eShAYrZb Accessed on 11 Jan 2016.
http://www.webcitation.org/6eShAYrZb...
). For its part, the measurement of policy effectiveness has largely focused on decreases in consumption or improvement in drinking pattern as measures of policy success (77. Cook WK, Bond J, Greenfield TK. Are alcohol policies associated with alcohol consumption in low- and middle-income countries? Addiction. 2014 Jul;109(7):1081-90., 88. Naimi TS, Blanchette J, Nelson TF, Nguyen T, Oussayef N, Heeren TC, et al. A new scale of the U.S. alcohol policy environment and its relationship to binge drinking. Am J Prev Med. 2014 Jan;46(1):10-6.). The relatively few studies that have examined outcomes other than consumption and drinking pattern, such as injury morbidity, medical morbidity, and mortality, have generally limited the work to a specific country or region (99. Holmes J, Meier PS, Booth A, Guo Y, Brennan A. The temporal relationship between per capita alcohol consumption and harm: a systematic review of time lag specifications in aggregate time series analyses. Drug Alcohol Depend. 2012 Jun 1;123(1–3):7-14.1111. Stockwell T, Zhao J, Martin G, Macdonald S, Vallance K, Treno A, et al. Minimum alcohol prices and outlet densities in British Columbia, Canada: estimated impacts on alcohol-attributable hospital admissions. Am J Public Health. 2013 Nov;103(11):2014-20.).

The development of policy scales began in the 1970s and 1980s as a means of understanding how regulatory measures collectively worked to reduce alcohol-related harms. Many scales have been developed but, similar to individual policy research, the scope has been geographically limited. In addition, the indicators of their performance have been limited to per capita alcohol consumption or the pattern of consumption of the population (88. Naimi TS, Blanchette J, Nelson TF, Nguyen T, Oussayef N, Heeren TC, et al. A new scale of the U.S. alcohol policy environment and its relationship to binge drinking. Am J Prev Med. 2014 Jan;46(1):10-6., 1212. Karlsson T, Österberg E. A scale of formal alcohol control policy in 15 European countries. Nordisk Alkohol Nark. 2001;18(English suppl.):117-28.1515. Anderson P, Braddick F, Reynolds J, Gual A, eds. Alcohol policy in Europe: evidence from AMPHORA. Barcelona: The AMPHORA Project; 2012. Archived at http://webcache.googleusercontent.com/search?q=cache:l3Dhy7nc6YIJ:amphoraproject.net/w2box/data/AM_E-BOOK_8-10-12_corrected.pdf+&cd=1&hl=en&ct=clnk&gl=us Accessed on 26 Mar 2015.
http://webcache.googleusercontent.com/se...
). While several scales have shown inverse associations between restrictive policies and alcohol consumption, the associations with negative health outcomes have not been analyzed.

The WHO estimates that injuries constitute nearly 26% of all alcohol-attributable deaths (1616. World Health Organization. Global status report on alcohol and health 2014. Geneva: WHO; 2014. Archived by WebCite at http://www.webcitation.org/6hvWyyErO Accessed on 31 May 2016.
http://www.webcitation.org/6hvWyyErO...
). Injury can be categorized as unintentional or intentional. The first category is attributable to those actions that occurred without intending to cause damage or injury, while the second category is attributable to harmful actions caused by self or others. International morbidity estimates of alcohol-related injury presenting to the emergency department are at rates ranging from 22% to upwards of 45% (1717. Cherpitel CJ, Borges G, Giesbrecht N, Hungerford D, Peden M, Poznyak V, et al., eds. Alcohol and injuries: emergency department studies in an international perspective. Geneva: World Health Organization; 2009.). Death due to injury accounts for 9% of global mortality, according to the WHO, which has stated: “For every death, it is estimated that there are dozens of hospitalizations, hundreds of emergency department visits and thousands of doctors' appointments. A large proportion of people surviving their injuries incur temporary or permanent disabilities” (1818. World Health Organization. Health topics: injuries. Geneva: WHO; 2016. Archived at http://webcache.googleusercontent.com/search?q=cache:_xexYF3zLvoJ:www.who.int/topics/injuries/en/+&cd=1&hl=en&ct=clnk&gl=us Accessed on 7 May 2013.
http://webcache.googleusercontent.com/se...
). Reducing or preventing injury would not only improve public health but also reduce the tremendous economic burden that injury imposes on society.

This work explains the development of the International Alcohol Policy Injury Index (IAPII), which aims to measure the effectiveness of control policies on selected indicators of alcohol-related injuries. Our hypothesis is that the IAPII will be effective at linking alcohol control policies to alcohol-related injury deaths, while controlling for alcohol consumption.

METHODS

Measures

Alcohol policy

Alcohol policy data were extracted from the WHO's 2012 Global Information System on Alcohol and Health (GISAH) for 156 member countries for which these data were available. GISAH is a primary point of reference for monitoring health and trends in alcohol consumption, alcohol-related harm, and policy responses (1919. World Health Organization. Global Information System on Alcohol and Health (GISAH). Geneva: WHO; 2016. Archived by WebCite at http://www.webcitation.org/6HKT0zJGQ Accessed on 31 May 2016.
http://www.webcitation.org/6HKT0zJGQ...
). Of the policies considered for inclusion, several lacked sufficient country data (e.g., brief intervention), so they could not be used to reliably identify association with injury.

Injury outcome measures

Two measures of injury death were extracted from the 2012 GISAH data and used as outcomes. The first was the alcohol-attributable fraction (AAF) for road traffic fatalities for males and females. AAFs represent the proportion of deaths that would be eliminated in the absence of alcohol. Second, age-standardized all-injury fatality data were used for both intentional (violence related to homicide or suicide) and unintentional (traffic collisions, drowning, poisoning, falls, or burns) deaths, regardless of alcohol involvement (2020. World Health Organization. Global Health Observatory data repository: age-standardized death rate by three major cause groups, both sexes. Data by country. Geneva: WHO; 2016. Archived at http://webcache.googleusercontent.com/search?q=cache:Os9z2jWeL4YJ:apps.who.int/gho/data/node.main-searo.12%3Flang%3Den+&cd=2&hl=en&ct=clnk&gl=us]. Accessed on 31 May 2016.
http://webcache.googleusercontent.com/se...
). All injury measures reflect the number of deaths per 100 000 in the population for a given country. All injury incidence rates were logged to normalize the distributions.

Alcohol consumption as a covariate

Because this work seeks to understand how alcohol policies are associated with injury death and a strong body of literature supports the association between alcohol consumption and policy, all statistical analyses control for country-level alcohol consumption. The measure of alcohol use is the three-year average recorded and unrecorded alcohol per capita (APC) consumption for ages 15+, for the years 2008-2010.

Data Analysis

Correlations between policy items, consumption, and injury measures were explored as a preliminary step in investigating potential relationships. Random, split-half samples of the 156 WHO member countries were then generated to construct and validate the IAPII. Linear regressions were conducted on the first split-half sample to test the direction and strength of relationships between potential (efficacy-weighted) policy items and injury outcomes, controlling for three-year average per capita alcohol consumption, and then validated on the second half sample. The three injury outcomes (AAF for traffic injury death for males, AAF traffic injury death for females, and overall injury deaths) were all log transformed for the analysis.

DEVELOPMENT OF THE IAPII AND VALIDATION RESULTS

Conceptualization of the IAPII

William J. Haddon, long considered the father of modern injury epidemiology, argued for a scientifically driven approach to injury prevention with practical application, and provided a 10-item conceptual framework for addressing hazards due to injury (2121. Haddon W, Jr. Advances in the epidemiology of injuries as a basis for public policy. Public Health Rep. 1980 Sept-Oct;95(5):411-21.). That framework was then modified by Runyan (2222. Runyan CW. Introduction: back to the future–revisiting Haddon's conceptualization of injury epidemiology and prevention. Epidemiol Rev. 2003;25:60-4.). Table 1 presents our adaptation of Runyan (2222. Runyan CW. Introduction: back to the future–revisiting Haddon's conceptualization of injury epidemiology and prevention. Epidemiol Rev. 2003;25:60-4.), to show the practical application of each countermeasure to alcohol related harms, along with the associated alcohol policy domain that could potentially be applied to decrease alcohol-injury harms.

TABLE 1
Adaptation of Haddon's model of 10 countermeasures to reduce or prevent alcohol-related injury, the practical application of the countermeasure, and the associated alcohol policy domain(s) and subdomain(s)

In addition to this work by Haddon, we developed a conceptual model (Figure 1) of alcohol and injury, based on a modified Delphi method using an estimate-talk-estimate (2323. Gustafson DH, Shukla RK, Delbecq A, Walster GW. Comparative study of differences in subjective likelihood estimates: individuals, interacting, Delphi, and nominal groups. Organ Behav Hum Perform. 1973 Apr;9(2):280-91.) method of decision-making among experts in the field of alcohol and injury. The model emphasizes the influence of alcohol policy (right side of model, in red) as it relates to the alcohol/injury death nexus, specifically to drinking and drinking context.

FIGURE 1
Conceptual model of the relationships among alcohol use, injury, the effect of societal drinking, and alcohol policies

These two conceptual models provided the underpinnings for development of an alcohol and injury policy index. We also conducted a thorough review of the alcohol control policy literature (1212. Karlsson T, Österberg E. A scale of formal alcohol control policy in 15 European countries. Nordisk Alkohol Nark. 2001;18(English suppl.):117-28., 1515. Anderson P, Braddick F, Reynolds J, Gual A, eds. Alcohol policy in Europe: evidence from AMPHORA. Barcelona: The AMPHORA Project; 2012. Archived at http://webcache.googleusercontent.com/search?q=cache:l3Dhy7nc6YIJ:amphoraproject.net/w2box/data/AM_E-BOOK_8-10-12_corrected.pdf+&cd=1&hl=en&ct=clnk&gl=us Accessed on 26 Mar 2015.
http://webcache.googleusercontent.com/se...
), including the WHO alcohol policy study (2424. Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity: research and public policy. New York: Oxford University Press; 2003.), the Alcohol Policy Index (API) (2525. Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB. Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007 Apr;4(4):e151.), and the Toolkit for Evaluating Alcohol policy Stringency and Enforcement (TEASE-16) (1414. Carragher N, Byrnes J, Doran CM, Shakeshaft A. Developing an alcohol policy assessment toolkit: application in the western Pacific. Bull World Health Organ. 2014 Oct;92(10):726-33.). Based on adaptation of the Haddon/Runyan model (Table 1) and our conceptual model (Figure 1), and taking into account that other alcohol policy indices failed to comprehensively assess policy enforcement, we developed the International Alcohol Policy Injury Index (IAPII) as a measure to link alcohol control policies to alcohol-related injury deaths.

Development of the IAPII

Based on theoretical and conceptual considerations, the selected alcohol policies were extracted from the GISAH, merged into a single data file, and coded according to the relative stringency of a given policy, using a point system from previous alcohol indices (1414. Carragher N, Byrnes J, Doran CM, Shakeshaft A. Developing an alcohol policy assessment toolkit: application in the western Pacific. Bull World Health Organ. 2014 Oct;92(10):726-33., 2525. Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB. Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007 Apr;4(4):e151.). There were no points for the most lenient policy option, full points for the most restrictive option, and partial points for intermediate options. The maximum points (1.0) were for the highest level of stringency.

The policies were grouped into five broad regulatory domains (33. Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity: research and public policy. 2nd ed. New York: Oxford University Press; 2010.): 1) physical availability (four items: legal minimum drinking age, government monopoly on retail sales, restrictions on density of outlets, and restrictions on hours and days of operation); 2) drinking context (two items: community mobilization programs and mandatory server training); 3) advertising/promotion (one item: a composite measure of restrictions on the majority of media ads); 4) vehicular (three items: random breath testing, legal blood alcohol concentration (BAC) limits, and penalties for exceeding the maximum BAC); and 5) pricing (three items: wine index prices, beer index prices, and spirit index prices).

Most of the individual policy items were created from several measures (composites). For example, age limits were created from 6 variables (3 beverage types each for on- and off-premise sales); government monopoly from 3 variables (3 beverage types); outlet density restrictions from 3 variables (3 beverage types); time restrictions from 6 variables (3 beverage types each for hours and days); community programs from 4 variables; BAC penalty from 7 variables (various punitive actions); advertising/promotion from 24 variables (3 beverages across 8 venues); and pricing from 3 variables (3 beverage types).

Policy Enforcement

Policy enforcement data had been lacking in most other alcohol policy indices, but is an important component of policy success. Successful enforcement of alcohol policy (2626. Sager F. Habermas' models of decisionism, technocracy and pragmatism in times of governance: the relationship of public administration, politics and science in the alcohol prevention policies of the Swiss member states. Public Adm. 2007 Jun;85(2):429-47.), as well as policies affecting other industries (2727. Slemrod J. What do cross-country studies teach about government involvement, prosperity, and economic growth? Brookings Pap Econ Act. 1995;No. 2:373-431.), have been found to be highly correlated with the economic prosperity of a country. Therefore, the IAPII incorporated the per capita gross national income (GNI) of each country as a proxy for implementation and enforcement (2828. Waters E, Thom B. Alcohol, policy and politics in Kazakhstan. Eur Asia Stud. 2007 Sept;59(6):999-1023., 2929. Peer N. There has been little progress in implementing comprehensive alcohol control strategies in Africa. Am J Drug Alcohol Abuse. 2017 Nov;43(6):631-5.), given that enforcement data were not available for WHO member countries. The World Bank's four income classifications (in U.S. dollars) were used: low income (≤ US$ 1 025), low middle-income (US$ 1 026 to 4 035), upper middle-income (US$ 4 036 to 12 475), and high income (≥ US$ 12 476). Appendix A, which is supplementary material, lists the 156 countries included in the analysis according to WHO geographic region. An in-depth description of the composite measures and scoring method can be obtained by contacting the first author (RAK).

In correlational analysis, with the exception of pricing, all policy items produced negative coefficients, that is, with higher stringency being associated with fewer injury deaths. Because the three pricing policy items were not associated with injury, they were dropped from index inclusion. The resulting 10 policy items, in four regulatory domains, were retained for inclusion in the IAPII (Table 2).

TABLE 2
Policy domains in developing the International Alcohol Policy Injury Index (IAPII)

The 10 (stringency-weighted) policy items and the respective (stringency-weighted) enforcement levels were assigned an effectiveness weight. These weights were similar to the ones described in the WHO 2003 publication (2424. Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity: research and public policy. New York: Oxford University Press; 2003.) that used a “star” rating (range of 1 to 3) to indicate how effective an array of international alcohol policies were in reducing alcohol consumption, alcohol-related problems, and societal costs.

Efficacy weights for the IAPII were assigned based on current research findings, especially those pertaining to breadth of support and cross-cultural testing. Scores in the overall index were designed to range between 0 and 100. Briefly, each of our 10 policy items was assigned a weight of 1, 2, or 3 to indicate a limited, moderate, or high effectiveness rating, respectively. We then determined that the respective proportionate (rounded here) point values 3.8, 7.7, and 11.5 would yield a total of 100 points when summed across 10 policies. More precisely, the IAPII includes six items given an effectiveness rating of 3 and four items given an effectiveness rating of 2, totaling 26 across 10 items (each effectiveness rating represents 3.8 points). Therefore, each policy item within a domain was assigned a score based on a country's level of stringency points and level of enforcement points, adjusted for its effectiveness rating.

Following methodology used by the TEASE-16 analyses (1414. Carragher N, Byrnes J, Doran CM, Shakeshaft A. Developing an alcohol policy assessment toolkit: application in the western Pacific. Bull World Health Organ. 2014 Oct;92(10):726-33.), a 50:50 combination effectiveness weight (equivalent points) was allocated to the stringency and enforcement values. Summing the policy scores yielded a domain score. Summing all the domain scores yielded an overall score. The final index (domains and overall score) was then constructed and tested against injury data on the first split-half sample, and then validated on the second half sample.

Replication of regressions using the second random split-half sample confirmed the findings from the first random split-half. Figure 2, Figure 3, and Figure 4 plot each country's IAPII score and, respectively, the incidence of AAF traffic deaths for males, AAF traffic deaths for females, and overall injury death for both genders. Regression results, each controlling for alcohol consumption, for logged AAF traffic injury death for males (B = -.02; R2 = 0.43), logged AAF traffic injury death for females (B = -.02; R2 = 0.49), and logged overall injury death (B = -.2; R2 = 0.43) were all significant at the p < 0.001 level and were also all inversely associated with injury deaths.

FIGURE 2
Plot of alcohol-attributable fraction (AAF) for road traffic deaths in males and International Alcohol Policy Injury Index (IAPII) score
FIGURE 3
Plot of alcohol-attributable fraction (AAF) for road traffic deaths in females and International Alcohol Policy Injury Index (IAPII) score
FIGURE 4
Plot of country-level number of injury deaths per 100 000 and the International Alcohol Policy Injury Index (IAPII) score

To examine if the IAPII was specifically associated with injury deaths, Brand's Alcohol Policy Index (API) (2525. Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB. Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007 Apr;4(4):e151.) effectiveness scoring method (used to predict per capita alcohol consumption in 30 member countries of the Organization for Economic Cooperation and Development) was calculated for each country, and the regression analyses were repeated. The API was associated with overall injury death, but was not significantly associated for AAF traffic injury deaths for males or females.

DISCUSSION

Our findings support the IAPII as an index associated with injury death and AAF traffic deaths. Information on policies in the four hypothesized domains was collected and weighted using theoretical constructs, and was empirically tested. Using concepts of prevention and context, the domains of availability, context, advertising/promotion, and vehicular policies were associated with injury in the expected inverse direction; however, pricing was not associated with injury mortality. Generally, pricing has been used to predict consumption, and our models controlled for consumption, which may possibly explain the lack of association. Though pricing policies have been deemed a best buy by the WHO (3030. Anderson P, Chisholm D, Fuhr DC. Effectiveness and cost-effectiveness of policies and programmes to reduce the harm caused by alcohol. Lancet. 2009 Jun;373(9682):2234-46.), they may be better regional indicators of effectiveness. For example, several authors (1010. Wagenaar AC, Tobler AL, Komro KA. Effects of alcohol tax and price policies on morbidity and mortality: a systematic review. Am J Public Health. 2010 Nov;100(11):2270-8., 1111. Stockwell T, Zhao J, Martin G, Macdonald S, Vallance K, Treno A, et al. Minimum alcohol prices and outlet densities in British Columbia, Canada: estimated impacts on alcohol-attributable hospital admissions. Am J Public Health. 2013 Nov;103(11):2014-20.) have shown an association between pricing and injury morbidity and mortality, although these studies were conducted in specific regions with greater attention to pricing variation. Moreover, other regional studies have found pricing to be differentially effective, depending on such characteristics as socioeconomic status and gender (3131. Mäkelä P, Herttua K, Martikainen P. The socioeconomic differences in alcohol-related harm and the effects of alcohol prices on them: a summary of evidence from Finland. Alcohol Alcohol. 2015 Nov;50(6):661-9., 3232. Meier PS, Holmes J, Angus C, Ally AK, Meng Y, Brennan A. Estimated effects of different alcohol taxation and price policies on health inequalities: a mathematical modelling study. PLoS Med. 2016 Feb 23;13(2):e1001963.), thereby making these policies less generalizable for larger geographic areas. Indeed, larger countries with less federal control and more regional authority show tremendous variation in pricing and are not included in many of the WHO pricing policies (e.g., Canada, the United States) because generalization is neither possible nor ideal. Additionally, other international alcohol indices (1414. Carragher N, Byrnes J, Doran CM, Shakeshaft A. Developing an alcohol policy assessment toolkit: application in the western Pacific. Bull World Health Organ. 2014 Oct;92(10):726-33., 2525. Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB. Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007 Apr;4(4):e151.) have found that the exclusion of pricing from the index had little impact on the association between the index and outcome. In sum, pricing policies may not be sensitive enough to account for injury deaths within an international context. In addition, because cost of living, taxation, and purchasing power varies between countries, the use of cross-sectional data can be problematic. It is possible that change in pricing over time may demonstrate an association with alcohol-related mortality and morbidity, but that issue is beyond the scope of this study.

While the AAF for road traffic deaths reflects alcohol use, overall injury death is not only alcohol related. However, injury mortality is a viable proxy of alcohol consumption because there is strong evidence from the scientific literature on the role of alcohol use on injury risk (1717. Cherpitel CJ, Borges G, Giesbrecht N, Hungerford D, Peden M, Poznyak V, et al., eds. Alcohol and injuries: emergency department studies in an international perspective. Geneva: World Health Organization; 2009., 3333. Taylor B, Irving HM, Kanteres F, Room R, Borges G, Cherpitel CJ, et al. The more you drink, the harder you fall: a systematic review and meta-analysis of how acute alcohol consumption and injury or collision risk increase together. Drug Alcohol Depend. 2010 Jul;110(1–2):108-16.) and it is estimated that over a quarter of all injury deaths are alcohol related (1616. World Health Organization. Global status report on alcohol and health 2014. Geneva: WHO; 2014. Archived by WebCite at http://www.webcitation.org/6hvWyyErO Accessed on 31 May 2016.
http://www.webcitation.org/6hvWyyErO...
).

The index and its policy domains were similar to those of other policy indices. However, strengths of this work include our use of a large sample (156 WHO member countries) and the incorporation of GNI as a measure of enforcement. Overall injury death was significant for both the API and the IAPII, but the API did not significantly predict vehicular fatality AAFs for men or women. This signifies that the weighting and domains in the IAPII may better predict an association with injury death due to alcohol after accounting for alcohol consumption.

Injury prevention and policy

The IAPII uses domains and scoring that are similar to those of other recent international indices (1414. Carragher N, Byrnes J, Doran CM, Shakeshaft A. Developing an alcohol policy assessment toolkit: application in the western Pacific. Bull World Health Organ. 2014 Oct;92(10):726-33., 2525. Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB. Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007 Apr;4(4):e151.). Unfortunately, screening, brief intervention, and referral to treatment (SBIRT); treatment provision; and other context-related policies outlined in Table 1 and Figure 1 could not be added to the index because too few countries provided such information.

Alcohol-related injury deaths are only exceeded by alcohol-related cardiovascular diseases (at 26% and 33%, respectively) as the main causes of premature deaths worldwide. In addition, injuries account for nearly one-third of alcohol-attributable disability-adjusted life years (DALYs) (1616. World Health Organization. Global status report on alcohol and health 2014. Geneva: WHO; 2014. Archived by WebCite at http://www.webcitation.org/6hvWyyErO Accessed on 31 May 2016.
http://www.webcitation.org/6hvWyyErO...
). Given those facts, why is there not more surveillance of injury due to alcohol? International morbidity data that specifically addresses alcohol-related injury (such as context of the injury, injury due to others, and alcohol-related cause of injury) are sorely lacking. That is despite the fact that the WHO has provided guidelines to enact injury surveillance (3434. World Health Organization, Western Pacific Region. Addressing the harmful use of alcohol: a guide to developing effective alcohol legislation. Archived at http://webcache.googleusercontent.com/search?q=cache:Hul7Q9daHBEJ:www.wpro.who.int/publications/docs/Addressingtheharmfuluseofalcoholforupload.pdf+&cd=1&hl=en&ct=clnk&gl=us Accessed on 31 May 2016.
http://webcache.googleusercontent.com/se...
).

To date, injury prevention has been an issue primarily addressed by developed nations. However, it is the lower- and middle-income countries (LMICs) that would most benefit from such information. Alcohol has been identified as a factor underlying higher mortality risk in disadvantaged populations (3535. Probst C, Roerecke M, Behrendt S, Rehm J. Socioeconomic differences in alcohol-attributable mortality compared with all-cause mortality: a systematic review and meta-analysis. Int J Epidemiol. 2014 Aug;43(4):1314-27.). The Global Alcohol Policy Alliance (3636. Global Alcohol Policy Alliance. Global Alcohol Policy Alliance. Available at http://globalgapa.org Accessed on 31 May 2016.
http://globalgapa.org...
) proposes policy guidelines for LMICs, but it does not suggest any surveillance of alcohol-related morbidity as a means of information to create policies. Schmitz (3737. Schmitz HP. The global health network on alcohol control: successes and limits of evidence-based advocacy. Health Policy Plan. 2016 Apr;31(Suppl.1):i87-i97.) believes that more consideration should be given to key regional stakeholders, both within the government and international organizations, to promote policies culturally relevant for LMIC populations. While we agree, we believe that morbidity data on alcohol-related injuries would assist policymakers from the LMICs to implement laws that are both practical and culturally relevant, and, therefore, more likely to be enforced. However, conducting such surveillance may not be economically viable in many of the LMICs. In principle, all alcohol-attributable mortality is avoidable, and development of alcohol policies must take into consideration the differential effect that alcohol-related harms may cause to specific populations.

Further examination of the IAPII could offer insight to key stakeholders on its utility. Cost analysis on injury reduction would provide and possibly incentivize stakeholders in the medical and political arenas to find viable solutions for preventing injury death due to alcohol use. Injury morbidity, which reflects a considerably larger burden to society than injury death, should be examined, with the goal of preventing all injury due to alcohol use.

Limitations

Though the IAPII reflects measurement of the alcohol policies considered, it cannot assume absolute accuracy in the association between alcohol policies and injury. The IAPII currently cannot capture the importance of alcohol to society, which may play a pivotal role in the stringency of policies and method by which research is supported (1212. Karlsson T, Österberg E. A scale of formal alcohol control policy in 15 European countries. Nordisk Alkohol Nark. 2001;18(English suppl.):117-28., 3838. Savic M, Room R. Differences in alcohol-related research publication output between countries: a manifestation of societal concern? Eur Addict Res. 2014;20(6):319-23.). Additionally, we chose to include countries with all policy information and forgo those that did not have that, which may have resulted in bias. Our conceptual model (Figure 1) was based on a modified Delphi method using estimate-talk-estimate or nominal group technique (3939. Graefe A, Armstrong JS. Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. Int J Forecast. 2011 Jan-Mar;27(1):183-95.). This process has shown validity (4040. Eubanks BH, Mohtadi NG, Lafave MR, Wiley JP, Bois AJ, Boorman RS, et al. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Med Res Methodol. 2016 May;16:56., 4141. Sundberg LR, Garvare R, Nyström ME. Reaching beyond the review of research evidence: a qualitative study of decision making during the development of clinical practice guidelines for disease prevention in healthcare. BMC Health Serv Res. 2017 May;17:344.), although group consensus using a full Delphi method (4242. van Beeck E. The Delphi method: a tool to support injury control and trauma care policy. Injury. 2017 Jan;48(1):3-4.) may have resulted in different conceptual findings (1515. Anderson P, Braddick F, Reynolds J, Gual A, eds. Alcohol policy in Europe: evidence from AMPHORA. Barcelona: The AMPHORA Project; 2012. Archived at http://webcache.googleusercontent.com/search?q=cache:l3Dhy7nc6YIJ:amphoraproject.net/w2box/data/AM_E-BOOK_8-10-12_corrected.pdf+&cd=1&hl=en&ct=clnk&gl=us Accessed on 26 Mar 2015.
http://webcache.googleusercontent.com/se...
). Because of the strong association between enforcement of policies and economic prosperity, the IAPII incorporates per capita GNI (4343. World Bank. Data Bank. Washington, D.C.: World Bank; 2017. Archived by WebCite at http://www.webcitation.org/6sS6GWetZ Accessed on 3 Aug 2017.
http://www.webcitation.org/6sS6GWetZ...
) as a proxy for enforcement (2828. Waters E, Thom B. Alcohol, policy and politics in Kazakhstan. Eur Asia Stud. 2007 Sept;59(6):999-1023., 2929. Peer N. There has been little progress in implementing comprehensive alcohol control strategies in Africa. Am J Drug Alcohol Abuse. 2017 Nov;43(6):631-5.), but this could differ from other measures of alcohol policy enforcement. Finally, the index was created in a manner similar to other indices, and important confounders beyond that of alcohol consumption (e.g., religion, percentage of women using alcohol) need more comprehensive investigation.

Whether an international index on alcohol policy provides utility has also been brought into question (4444. Ritter A. Comparing alcohol policies between countries: science or silliness? PLoS Med. 2007 Apr;4(4):e153.). Specifically, does the creation of an index provide any utility as a comparative measure, and what utility does it provide? We assert that indices have the power to provide a numerical assessment for disseminating information to the public and key stakeholders, to advocate, revise, or expand legislation on alcohol policy to reduce injuries based on the evidence from the IAPII. It makes it possible to examine what policies exist and in which countries, in order to allow for an evaluative framework for assimilating or structuring similar policies.

Conclusions

As hypothesized, the IAPII shows clear association with injury deaths and AAF for traffic deaths after controlling for alcohol consumption. The combination of individual policy domains, policy effectiveness, stringency weighting, and enforcement provides the needed components for this index.

Recommendations

Future work should test the effectiveness of the IAPII in reducing alcohol-related injury morbidity, which accounts for a larger share of the global burden of disease than alcohol-related injury mortality does.

  • Suggested citation Korcha RA, Witbrodt J, Cherpitel CJ, Ye Y, Andreuccetti G, Kang J, et al. Development of the International Alcohol Policy and Injury Index. Rev Panam Salud Publica. 2018;42:e6. https://doi.org/10.26633/RPSP.2018.6
  • Funding. U.S. National Institute on Alcohol Abuse and Alcoholism (NIAAA), grant #NIAAA R01 AA13750 (principal investigator: Cheryl J. Cherpitel).
  • Disclaimer. The authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the RPSP/PAJPH, PAHO, the NIAAA, WHO, or other author affiliations.

REFERENCES

  • 1
    Institute for Health Metrics and Evaluation. Global Burden of Diseases, Injuries, and Risk Factors Study 2010 (GBD 2010). Seattle: University of Washington; 2013. Archived by WebCite at http://www.webcitation.org/6GRmKzeL5 Accessed 7 May 2013.
    » http://www.webcitation.org/6GRmKzeL5
  • 2
    Ezzati M, Lopez AD, Rodgers A, Vander Hoorn S, Murray CJL. Selected major risk factors and global and regional burden of disease. Lancet. 2002 Nov 2;360(9343):1347-60.
  • 3
    Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity: research and public policy. 2nd ed. New York: Oxford University Press; 2010.
  • 4
    Stockwell T, Giesbrecht N. Strategies to prevent alcohol-related injury targeted to high risk products, settings and populations. In: Cherpitel CJ, Borges G, Giesbrecht N, Monteiro M, Stockwell T, eds. Prevention of alcohol-related injuries in the Americas: from evidence to policy action. Washington, D.C.: Pan American Health Organization; 2013:149-58.
  • 5
    Room R, Graham K, Rehm J, Jernigan DH, Monteiro M. Drinking and its burden in a global perspective: policy considerations and options. Eur Addict Res. 2003 Oct;9(4):165-75.
  • 6
    Bloom DE, Chisholm D, Jané-Llopis E, Prettner K, Stein A, Feigl A. From burden to “best buys”: reducing the economic impact of non-communicable diseases in low- and middle-income countries. Geneva: World Health Organization; 2011. Archived by WebCite at http://www.webcitation.org/6eShAYrZb Accessed on 11 Jan 2016.
    » http://www.webcitation.org/6eShAYrZb
  • 7
    Cook WK, Bond J, Greenfield TK. Are alcohol policies associated with alcohol consumption in low- and middle-income countries? Addiction. 2014 Jul;109(7):1081-90.
  • 8
    Naimi TS, Blanchette J, Nelson TF, Nguyen T, Oussayef N, Heeren TC, et al. A new scale of the U.S. alcohol policy environment and its relationship to binge drinking. Am J Prev Med. 2014 Jan;46(1):10-6.
  • 9
    Holmes J, Meier PS, Booth A, Guo Y, Brennan A. The temporal relationship between per capita alcohol consumption and harm: a systematic review of time lag specifications in aggregate time series analyses. Drug Alcohol Depend. 2012 Jun 1;123(1–3):7-14.
  • 10
    Wagenaar AC, Tobler AL, Komro KA. Effects of alcohol tax and price policies on morbidity and mortality: a systematic review. Am J Public Health. 2010 Nov;100(11):2270-8.
  • 11
    Stockwell T, Zhao J, Martin G, Macdonald S, Vallance K, Treno A, et al. Minimum alcohol prices and outlet densities in British Columbia, Canada: estimated impacts on alcohol-attributable hospital admissions. Am J Public Health. 2013 Nov;103(11):2014-20.
  • 12
    Karlsson T, Österberg E. A scale of formal alcohol control policy in 15 European countries. Nordisk Alkohol Nark. 2001;18(English suppl.):117-28.
  • 13
    Casswell S, Meier P, MacKintosh AM, Brown A, Hastings G, Thamarangsi T, et al. The International Alcohol Control (IAC) study-evaluating the impact of alcohol policies. Alcohol Clin Exp Res. 2012 Aug;36(8):1462-7.
  • 14
    Carragher N, Byrnes J, Doran CM, Shakeshaft A. Developing an alcohol policy assessment toolkit: application in the western Pacific. Bull World Health Organ. 2014 Oct;92(10):726-33.
  • 15
    Anderson P, Braddick F, Reynolds J, Gual A, eds. Alcohol policy in Europe: evidence from AMPHORA. Barcelona: The AMPHORA Project; 2012. Archived at http://webcache.googleusercontent.com/search?q=cache:l3Dhy7nc6YIJ:amphoraproject.net/w2box/data/AM_E-BOOK_8-10-12_corrected.pdf+&cd=1&hl=en&ct=clnk&gl=us Accessed on 26 Mar 2015.
    » http://webcache.googleusercontent.com/search?q=cache:l3Dhy7nc6YIJ:amphoraproject.net/w2box/data/AM_E-BOOK_8-10-12_corrected.pdf+&cd=1&hl=en&ct=clnk&gl=us
  • 16
    World Health Organization. Global status report on alcohol and health 2014. Geneva: WHO; 2014. Archived by WebCite at http://www.webcitation.org/6hvWyyErO Accessed on 31 May 2016.
    » http://www.webcitation.org/6hvWyyErO
  • 17
    Cherpitel CJ, Borges G, Giesbrecht N, Hungerford D, Peden M, Poznyak V, et al., eds. Alcohol and injuries: emergency department studies in an international perspective. Geneva: World Health Organization; 2009.
  • 18
    World Health Organization. Health topics: injuries. Geneva: WHO; 2016. Archived at http://webcache.googleusercontent.com/search?q=cache:_xexYF3zLvoJ:www.who.int/topics/injuries/en/+&cd=1&hl=en&ct=clnk&gl=us Accessed on 7 May 2013.
    » http://webcache.googleusercontent.com/search?q=cache:_xexYF3zLvoJ:www.who.int/topics/injuries/en/+&cd=1&hl=en&ct=clnk&gl=us
  • 19
    World Health Organization. Global Information System on Alcohol and Health (GISAH). Geneva: WHO; 2016. Archived by WebCite at http://www.webcitation.org/6HKT0zJGQ Accessed on 31 May 2016.
    » http://www.webcitation.org/6HKT0zJGQ
  • 20
    World Health Organization. Global Health Observatory data repository: age-standardized death rate by three major cause groups, both sexes. Data by country. Geneva: WHO; 2016. Archived at http://webcache.googleusercontent.com/search?q=cache:Os9z2jWeL4YJ:apps.who.int/gho/data/node.main-searo.12%3Flang%3Den+&cd=2&hl=en&ct=clnk&gl=us] Accessed on 31 May 2016.
    » http://webcache.googleusercontent.com/search?q=cache:Os9z2jWeL4YJ:apps.who.int/gho/data/node.main-searo.12%3Flang%3Den+&cd=2&hl=en&ct=clnk&gl=us]
  • 21
    Haddon W, Jr. Advances in the epidemiology of injuries as a basis for public policy. Public Health Rep. 1980 Sept-Oct;95(5):411-21.
  • 22
    Runyan CW. Introduction: back to the future–revisiting Haddon's conceptualization of injury epidemiology and prevention. Epidemiol Rev. 2003;25:60-4.
  • 23
    Gustafson DH, Shukla RK, Delbecq A, Walster GW. Comparative study of differences in subjective likelihood estimates: individuals, interacting, Delphi, and nominal groups. Organ Behav Hum Perform. 1973 Apr;9(2):280-91.
  • 24
    Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity: research and public policy. New York: Oxford University Press; 2003.
  • 25
    Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB. Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007 Apr;4(4):e151.
  • 26
    Sager F. Habermas' models of decisionism, technocracy and pragmatism in times of governance: the relationship of public administration, politics and science in the alcohol prevention policies of the Swiss member states. Public Adm. 2007 Jun;85(2):429-47.
  • 27
    Slemrod J. What do cross-country studies teach about government involvement, prosperity, and economic growth? Brookings Pap Econ Act. 1995;No. 2:373-431.
  • 28
    Waters E, Thom B. Alcohol, policy and politics in Kazakhstan. Eur Asia Stud. 2007 Sept;59(6):999-1023.
  • 29
    Peer N. There has been little progress in implementing comprehensive alcohol control strategies in Africa. Am J Drug Alcohol Abuse. 2017 Nov;43(6):631-5.
  • 30
    Anderson P, Chisholm D, Fuhr DC. Effectiveness and cost-effectiveness of policies and programmes to reduce the harm caused by alcohol. Lancet. 2009 Jun;373(9682):2234-46.
  • 31
    Mäkelä P, Herttua K, Martikainen P. The socioeconomic differences in alcohol-related harm and the effects of alcohol prices on them: a summary of evidence from Finland. Alcohol Alcohol. 2015 Nov;50(6):661-9.
  • 32
    Meier PS, Holmes J, Angus C, Ally AK, Meng Y, Brennan A. Estimated effects of different alcohol taxation and price policies on health inequalities: a mathematical modelling study. PLoS Med. 2016 Feb 23;13(2):e1001963.
  • 33
    Taylor B, Irving HM, Kanteres F, Room R, Borges G, Cherpitel CJ, et al. The more you drink, the harder you fall: a systematic review and meta-analysis of how acute alcohol consumption and injury or collision risk increase together. Drug Alcohol Depend. 2010 Jul;110(1–2):108-16.
  • 34
    World Health Organization, Western Pacific Region. Addressing the harmful use of alcohol: a guide to developing effective alcohol legislation. Archived at http://webcache.googleusercontent.com/search?q=cache:Hul7Q9daHBEJ:www.wpro.who.int/publications/docs/Addressingtheharmfuluseofalcoholforupload.pdf+&cd=1&hl=en&ct=clnk&gl=us Accessed on 31 May 2016.
    » http://webcache.googleusercontent.com/search?q=cache:Hul7Q9daHBEJ:www.wpro.who.int/publications/docs/Addressingtheharmfuluseofalcoholforupload.pdf+&cd=1&hl=en&ct=clnk&gl=us
  • 35
    Probst C, Roerecke M, Behrendt S, Rehm J. Socioeconomic differences in alcohol-attributable mortality compared with all-cause mortality: a systematic review and meta-analysis. Int J Epidemiol. 2014 Aug;43(4):1314-27.
  • 36
    Global Alcohol Policy Alliance. Global Alcohol Policy Alliance. Available at http://globalgapa.org Accessed on 31 May 2016.
    » http://globalgapa.org
  • 37
    Schmitz HP. The global health network on alcohol control: successes and limits of evidence-based advocacy. Health Policy Plan. 2016 Apr;31(Suppl.1):i87-i97.
  • 38
    Savic M, Room R. Differences in alcohol-related research publication output between countries: a manifestation of societal concern? Eur Addict Res. 2014;20(6):319-23.
  • 39
    Graefe A, Armstrong JS. Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. Int J Forecast. 2011 Jan-Mar;27(1):183-95.
  • 40
    Eubanks BH, Mohtadi NG, Lafave MR, Wiley JP, Bois AJ, Boorman RS, et al. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Med Res Methodol. 2016 May;16:56.
  • 41
    Sundberg LR, Garvare R, Nyström ME. Reaching beyond the review of research evidence: a qualitative study of decision making during the development of clinical practice guidelines for disease prevention in healthcare. BMC Health Serv Res. 2017 May;17:344.
  • 42
    van Beeck E. The Delphi method: a tool to support injury control and trauma care policy. Injury. 2017 Jan;48(1):3-4.
  • 43
    World Bank. Data Bank. Washington, D.C.: World Bank; 2017. Archived by WebCite at http://www.webcitation.org/6sS6GWetZ Accessed on 3 Aug 2017.
    » http://www.webcitation.org/6sS6GWetZ
  • 44
    Ritter A. Comparing alcohol policies between countries: science or silliness? PLoS Med. 2007 Apr;4(4):e153.

History

  • Received
    16 Feb 2017
  • Accepted
    03 Dec 2017
  • Online publication
    07 June 2018
Organización Panamericana de la Salud Washington - Washington - United States
E-mail: contacto_rpsp@paho.org