ABSTRACT

Objective

To determine the level of stability or change in topic areas published by public health journals in Latin America and the Caribbean, using keywords and co-word analysis, in order to support evidence-based research planning.

Methods

Keywords were extracted from papers indexed in Scopus^® that were published by the Revista de Salud Pública (RSP; Colombia), the Salud Pública de México (SPM; Mexico), and the Revista Peruana de Medicina Experimental y Salud Pública (RPMESP; Peru) for three periods: 2005 – 2007, 2008 – 2010, and 2011 – 2013. Co-word analysis was used to examine keywords extracted. Textual information was analyzed using centrality measures (inbetweenness and closeness). The hypothesis of stability/change of thematic coverage was tested using the Spearman's rho correlation coefficient. VOSviewer was used to visualize the co-word maps.

Results

A moderate level of change in thematic coverage was observed in 2005 – 2010, as evidenced by the correlation coefficients for two of the 3-year periods, 2005 – 2007 and 2008 – 2010: 0.545 for RSP and 0.593 for SPM. However, in 2008 – 2013, more keywords remained constant from one period to the next, given the size of the correlation coefficients for the last 3-year periods: 2008 – 2010 and 2011 – 2013: 0.727 for RSP and 0.605 for SPM.

Conclusion

The research hypothesis was partially accepted given that just two consecutive 3-year periods showed a statistically-significant degree of stability in thematic coverage in public health studies. In that sense, this study provides compelling evidence of the effectiveness of using a combined approach for examining the dynamics of thematic coverage: centrality measures for identifying the main keywords and visual inspection for detecting the structure of textual information.

Keywords
Literature based discovery; terminology as topic; public health; Colombia; Mexico; Peru

RESUMEN

Objetivo

Determinar el grado de estabilidad o cambio en los temas que se publican en las revistas de salud pública de América Latina y el Caribe, por medio del análisis de palabras clave y copalabras, a fin de sustentar la planificación de investigaciones basadas en la evidencia.

Métodos

Se extrajeron las palabras clave de los artículos indizados en Scopus® publicados por tres revistas —Revista de la Salud Pública (RSP; Colombia), Salud Pública de México (SPM; México) y Revista Peruana de la Salud y de Medicina Experimental Pública (RPMESP; Perú)— en tres períodos: 2005-2007, 2008-2010 y 2011-2013. Se usó el análisis de copalabras para examinar las palabras clave extraídas. Se analizó la información textual aplicando diversas medidas de centralidad (intermediación y proximidad). Se puso a prueba la hipótesis de estabilidad o cambio de la cobertura temática usando el coeficiente de correlación rho de Spearman. Se usó VOSviewer para visualizar los mapas de copalabras.

Resultados

En el período 2005-2010 se observó un nivel moderado de cambios en la cobertura temática, como lo demuestran los coeficientes de correlación correspondientes a dos períodos trianuales (2005-2007 y 2008-2010): 0,545 en RSP y 0,593 en SPM. En cambio, en el período 2008-2013 un mayor número de palabras clave se mantuvo constante de un período al siguiente, considerando la magnitud de los coeficientes de correlación de los últimos períodos trianuales: 2008-2010 y 2011-2013: 0,727 en RSP y 0,605 en SPM.

Conclusiones

La hipótesis de investigación fue aceptada parcialmente considerando que solo en dos períodos trianuales consecutivos se observó una estabilidad estadísticamente significativa en la cobertura temática de los artículos sobre salud pública. En ese sentido, el presente estudio aporta datos convincentes sobre la eficacia de usar un enfoque combinado para examinar la dinámica de la cobertura temática: medidas de centralidad para determinar las principales palabras clave e inspección visual para determinar la estructura de la información textual.

Palabras clave
Descubrimiento basado en la literatura; terminología como asunto; salud pública; Colombia; México; Perú

RESUMO

Objetivo

Determinar o nível de estabilidade ou mudança em áreas de tópicos publicadas por periódicos de saúde pública na América Latina e no Caribe, com o uso de palavras-chave e da análise da coocorrência das palavras, para apoiar o planejamento de pesquisas com fundamentação científica.

Métodos

As palavras-chave foram extraídas de artigos indexados na Scopus® publicados na Revista de Salud Pública (RSP, Colômbia), Salud Pública de México (SPM, México) e Revista Peruana de Medicina Experimental y Salud Pública (RPMESP, Peru) em três triênios distintos: 2005–2007, 2008–2010 e 2011–2013. Foi usada a análise da coocorrência das palavras para analisar as palavras-chave extraídas. Informação textual foi analisada com o uso de medidas de centralidade (intermediação e proximidade). A hipótese de estabilidade/mudança da cobertura temática foi testada com o uso do coeficiente de correlação (rho) de Spearman e o programa VOSviewer foi usado para visualizar os mapas de coocorrência das palavras.

Resultados

Um nível moderado de mudança na cobertura temática foi observado no período de 2005–2010, como evidenciado pelos coeficientes de correlação em dois dos triênios estudados (2005–2007 e 2008–2010): 0,545 para RSP e 0,593 para SPM. Porém, no triênio de 2008–2013, verificou-se que um número maior de palavras-chave continuou constante de um período ao outro, como demonstrado pelos coeficientes de correlação para os últimos triênios (2008–2010 e 2011–2013): 0,727 para RSP e 0,605 para SPM.

Conclusões

A hipótese de pesquisa foi parcialmente aceita visto que somente dois triênios consecutivos apresentaram um nível estatisticamente significativo de estabilidade na cobertura temática em estudos de saúde pública. O presente estudo fornece evidências convincentes da efetividade de usar um enfoque combinado para examinar a dinâmica da cobertura temática: medidas de centralidade para identificar as principais palavras-chave e inspeção visual para detectar a estrutura da informação textual.

Palavras-chave
Descoberta baseada em literatura; terminologia como assunto; saúde pública; Colômbia; México; Peru

The World Health Organization defines the concept of public health as, “all organized measures (whether public or private) to prevent disease, promote health, and prolong life among the population as a whole…to provide conditions in which people can be healthy, and focus on entire populations, not on individual patients or diseases”(¹1. World Health Organization. Public Health. Available from: www.capitalareaphn.org/about/what-is-public-health Accessed 17 November 2015.
www.capitalareaphn.org/about/what-is-pub... ). Countries of Latin America and the Caribbean have undertaken significant advances in reducing mortality and morbidity rates by applying effective strategies for improving public health (²2. Cardona D, Acosta LD, Bertone CL. Inequidades en salud entre países de América Latina y El Caribe (2005 – 2010). Gac Sanit. 2013;27(4):292–7. – ⁴4. World Health Organization. Global Health Observatory data, 2015. Available from: www.who.int/gho/countries/en Accessed 17 November 2015.
www.who.int/gho/countries/en... ). However, significant challenges remain due to the increasing prevalence of psychological (e.g., stress and depression) and lifestyle-related (e.g., obesity and coronary diseases) concerns. Therefore, current public health policies must enact effective advances against new challenges that are relevant to national needs, rather than to international research agendas with limited relevance for health problems at the national level (⁵5. Bernier NF, Clavier C. Public health policy research: making the case for a political science approach. Health Prom Int. 2011;26(1):109–16. – ⁷7. Jones CM, Clavier C, Potvin L. Adapting public policy theory for public health research: a framework to understand the development of national policies on global health. Soc Sci Med. 2017;177(1):69–77.).

Bibliometric studies on public health

Bibliometric studies on public health traditionally apply a descriptive approach using raw and adjusted indicators, such as total output, citation counts, or output adjusted per population (⁸8. Clarke A, Gatineau M, Grimaud O, Royer-Devaux S, Wyn-Roberts N, Bis I, et al. A bibliometric overview of public health research in Europe. Eur J Public Health. 2007;17(suppl 1):43–9. – ¹²12. Porta M. The bibliographic “impact factor” of the Institute for Scientific Information: how relevant is it really for public health journals. J Epidemiol Community Health. 1996;50(6):606–10.). For example, in two studies, Soteriades and Falagas (¹⁰10. Soteriades E, Falagas ME. Comparison of amount of biomedical research originating from the European Union and the United States. Br Med J. 2005;331:192–4., ¹¹11. Soteriades E, Falagas ME. A bibliometric analysis in the fields of preventive medicine, occupational and environmental medicine, epidemiology, and public health. BMC Public Health. 2006;6(1):301.) examined scientific output from 1994 – 2004 in biomedical areas (including public health and clinical medicine) for European Union countries and the United States. The research team found that even though the United States led scientific production in this area, European countries showed remarkable advances when adjusted indicators were considered, e.g., papers per 1 000 population, citations per billions of US$, and total product/gross domestic product (GDP). Another study (⁸8. Clarke A, Gatineau M, Grimaud O, Royer-Devaux S, Wyn-Roberts N, Bis I, et al. A bibliometric overview of public health research in Europe. Eur J Public Health. 2007;17(suppl 1):43–9.) analyzed papers that were published from 1995 – 2004 and indexed in the Web of Science^® (Clarivate Analytics, Philadelphia, Pennsylvania, United States; WoS). Authors of that study reported comparable results, and according to multiple regression analysis, GDP was a modest predictor (r² = 0.53; P < 0.02) of publications for European countries.

The first bibliometric studies on public health in Latin American and the Caribbean (LAC) also applied a descriptive approach, whereas more recent studies identify factors associated with production and collaboration at the country or institutional level. Macías-Chapula (⁸8. Clarke A, Gatineau M, Grimaud O, Royer-Devaux S, Wyn-Roberts N, Bis I, et al. A bibliometric overview of public health research in Europe. Eur J Public Health. 2007;17(suppl 1):43–9., ¹³13. Macías-Chapula CA. Influence of local and regional publications in the production of Public Health research papers in Latin America. Scientometrics. 2010;84(3):703–16. – ¹⁵15. Macías-Chapula CA. Comparative analysis of Health Public policy research results among Mexico, Chile and Argentina. Scientometrics. 2013;95(2):615–28.), a prolific bibliometric studies researcher, examined visibility and online access to public health research results in Argentina, Chile, and Mexico using data extracted from the LILACS (Latin American and Caribbean Center on Health Sciences Information, PAHO/WHO, São Paulo, Brazil) and MEDLINE^® (U.S. National Library of Medicine, Bethesda, Maryland, United States) databases (¹⁵15. Macías-Chapula CA. Comparative analysis of Health Public policy research results among Mexico, Chile and Argentina. Scientometrics. 2013;95(2):615–28.). He found that only 6 of 30 health science descriptors emerged as relevant for the analyzed countries in both databases: health services accessibility, health care reform, decentralization, health systems, consumer participation, and financing. Other researchers have analyzed Latin American scientific output compared to world production and combined bibliometric analysis with socioeconomic and health indicators (¹⁶16. Chinchilla-Rodríguez Z, Zacca-González G, Vargas-Quesada B, Moya-Anegón F. Latin American scientific output in Public Health: combined analysis using bibliometric, socioeconomic and health indicators. Scientometrics. 2015;102(1):609–28., ¹⁷17. Zacca-González G, Chinchilla-Rodríguez Z, Vargas-Quesada B, Moya-Anegón F. Bibliometric analysis of regional Latin America's scientific output in Public Health through SCImago Journal & Country Rank. BMC Public Health. 2014;14:632.). Other advances in the field include analysis of manuscript quality; for example, those published by journals that are indexed in world-class citation databases, such as MEDLINE,^® WoS,^® and Scopus^® (Elsevier, Paris, France) provide a greater level of certainty concerning validity and reliability.

Most of the studies mentioned thus far have a common feature: bibliometric analysis was carried out at the national, and in some cases, regional level. No similar studies focused on the contents of papers published by public health journals were found. Even when bibliometric studies based on citation analysis used a different approach than this study, a general overview was nevertheless included because they were the only ones that employed bibliometric techniques for analyzing the contents of Latin American public health journals. The following section describes a complementary approach for analyzing the contents of scientific production.

Co-word analytical approach

The co-word analytical approach for textual analysis relies on a central premise: simultaneous occurrences of signal words reflect the main topics of a documental body. This differs from co-citation analysis that provides a distribution of clusters grouping relevant authors, studies, and academic publications. French researchers Rip and Courtial (¹⁸18. Rip A, Courtial JP. Co-word maps of biotechnology: An example of cognitive scientometrics. Scientometrics. 1984;6(6):381–400.) were the first to carry out an analysis based on co-occurrence of significant words and terms. Later, their approach was replicated by various fields, such as polymer chemistry, plant proteins, stem cells, patient adherence, intelligence network, and complementary medicine, among others (¹⁹19. An XY, Wu QQ. Co-word analysis of the trends in stem cells field based on subject heading weighting. Scientometrics. 2011;88(1):133–84. – ²³23. Wang ZY, Li G, Li CY, Li A. Research on the semantic-based co-word analysis. Scientometrics. 2012;90(3):855–75.).

An application of co-word analysis in the field of integrative and complementary medicine identified the level of content similarity in 1 329 articles published from 2007 – 2012 in specialized journals by using cluster techniques and a visualization software (²⁰20. Callon M, Courtial JP, Laville F. Co-word analysis as a tool for describing the network of interactions between basic and technological research: The case of polymer chemistry. Scientometrics. 1991;22(1):155–205.). A previous work with this textual method examined the knowledge structure and evolution of academic literature on patient adherence (²⁴24. Zhang J, Xie J, Hou W, Tu X, Song J, Wang Z, Lu Z. Mapping the knowledge structure of research on patient adherence: Knowledge domain visualization based co-word analysis and social network analysis. PloS One. 2012;7(4):e34497. Available from: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0034497 Accessed 17 November 2015.
http://journals.plos.org/plosone/article... ). In that study, researchers analyzed 2 308 articles (788 for the period 2000 – 2005, and 1 520 for 2006 – 2011). They found that in the first period, research themes were more general; while in the second, some thematic changes were observed as evidenced by the appearance of new terms. Despite its acceptance within the academic community, almost 20 years ago an eminent researcher on advanced methods for co-word analysis warned that because words change from one period to the next—in frequency of relations with other words and in terms of positional meaning—it is risky to assume conceptual stability of terms in texts over time (²⁵25. Leydesdorff L. Why words and co-words cannot map the development of the sciences. J Am Soc Inf Sci Technol. 1997;48(5):418–27.).

To explore a useful tool for analyzing advances in public health research, this study focused on the main topics covered in the academic literature and any changes to these over time. Its objective was to determine the level of stability or change in topic areas—expressed by the keywords of each paper—published by journals in Latin America and the Caribbean in 2005 – 2013 and indexed by Scopus.^® Ultimately, this information may serve as a useful tool for public health research managers and policymakers striving to make evidence-based policies that align the research agenda with country needs.

MATERIALS AND METHODS

This was a bibliometric study using the co-word approach to analyze any change in keywords in public health papers indexed in the Scopus^® database in 2005 – 2013. To have a comparable documental body, public health studies were selected from three LAC countries with similar investments in health care as percentage of the GDP.

Documental body

The Scopus^® database was chosen as the sole information source because of its broad coverage of academic literature produced in developing countries, especially those in Latin America and the Caribbean. In fact, this database recently added 59 Colombian academic publications to its index (²⁶26. Molina-Molina S, Moya-Anegón F. Política nacional y visibilidad internacional. El caso colombiano. Prof Inform. 2013;22(6):529–35.). Moreover, previous studies have documented its advantages over WoS^® for carrying out bibliometric analysis (²⁷27. Kalantari A, Jahanihashemi H, Kamran MKA. The importance of access to the Scopus database in the economic crisis condition. Int J Inf Sci Manage. 2014;12(2):47–56., ²⁸28. Moya-Anegón F, Chinchilla-Rodríguez Z, Vargas-Quesada B, Corera-Álvarez E, Muñoz-Fernández F, González-Molina A, Herrero-Solana V. Coverage analysis of Scopus: A journal metric approach. Scientometrics. 2007;73(1):53–78.).

The countries selected for the study had 4% – 8% of their GDP assigned to health care; established public health research institutes or equivalents; and public health journals (per the journal's scope statement). These three criteria made it possible to select countries with comparable characteristics, and to exclude countries, such as Chile and Cuba, whose percentage of GDP for health care was outside the specified range.

Data collection and matrix generation

After applying the inclusion criteria, three countries were selected: Colombia, Mexico, and Peru. Despite the differences in public health agendas, the three countries share similar characteristics in terms of economic development, public health programs, demographic characteristics, and investment in improving public health coverage. Because of this commonality on contextual issues, it was considered appropriate to make comparisons among them.

The next step was to define the time period to carry out the co-word analysis using textual information from the public health journals of the selected countries: Revista de Salud Pública (RSP), Colombia; Salud Pública de México (SPM), Mexico; and Revista Peruana de Salud Pública y Medicina Experimental (RPMESP), Peru. Even though these journals are published in Spanish—and therefore, have lower citation rates than comparable journals published in English—two are in the second quartile of public health publications (SPM = 0.49 and RPMESP = 0.25), according to SCImago Journal & Country Rank (SCImago Research Group, Consejo Superior de Investigaciones Científicas, University of Granada, Extremadura, Carlos III [Madrid] and Alcalá de Henares, Spain). However, given that the study purpose was to analyze stability and change over time, RPMESP was excluded because the textual information was available only for one period (2011 – 2013) making it impossible to analyze variations from one period to the next.

Studies published from 2005 – 2013 were included in the analysis. The contents of the “keywords” field was extracted for each study (989 papers for RSP and 1 313 for SPM) and were entered into three Microsoft Excel™ (Microsoft Corp., Redmond, Washington, United States) worksheets, one for each 3-year period: 2005 – 2007; 2008 – 2010; and 2011 – 2013. These time periods were used because RSP and SPM are indexed in SciELO (Latin American and Caribbean Center on Health Sciences Information, São Paulo, Brazil), which uses 3-year periods for calculating impact factor.

Visualization of co-word maps

The Microsoft Excel™ Comma-Separated-Values (CSV) files were exported to Table 2 Net tool (SciencesPo Médialab, Paris, France) using the following settings: Normal for “type of network;” Index Keywords/Semicolon-separated for “nodes;” and one expression per cell for the “Links row number” option. The resulting .gefx files were opened in Gephi (Gephi Consortium, Paris, France) visualization software, version 0.8.2, for creating .net (Pajek format) files. To obtain centrality measures for primary keywords (betweenness and closeness), the Force Atlas algorithm was used with appropriate settings (repulsion strength: 10 000; attraction strength: 10; gravity: 10). Centrality measures are indicators that provide an understanding of the identified elements within a text body. “Betweenness” refers to a position property by which a node acts as a bridge or connector between other nodes in a social network, whereas “closeness” represents the average distance a node is from other nodes. Higher values of betweenness and closeness let us identify significant nodes (terms, in this study) within a social network, or a co-words-based map.

To identify words with semantic value, multi-term keywords were considered for obtaining rankings with the highest centrality measures. For visualizing the co-word maps, VOSviewer^® (Nees Jan van Eck and Ludo Waltman, Leiden, Netherlands) software was used, with 50 for the “minimum total link strength” of an item field and the default settings for the remaining options.

At the end of the generation of co-word maps, authors constructed six datasets listing multi-term keywords with highest centrality values.

Data analysis

Given that keywords were ranked according to their centrality values, Spearman's rho was used to order ranks to determine if the rank in one 3-year period was associated with the next one. For each correlation coefficient, in addition to its magnitud, the statistical significance and the 95% Confidence Interval were computed based on Fisher's transformation. These computations were performed using IBM SPSS Statistics software, version 15.0 for Windows (SPSS Inc., an IBM company, Chicago, Illinois, United States) and Stata^® 13.1 (StataCorp LP, College Station, Texas, United States). If two rankings were correlated, there was assumed to be some degree of stability; and if not correlated, the assumption was that there was some degree of change. Since the aim was to measure the degree of stability or change in thematic coverage and because there were no predefined thresholds, three ranges distributed proportionally were used for understanding the coefficient obtained with the correlation analysis:

1. 0.01 – 0.29 = low stability / high change

2. 0.30 – 0.69 = medium stability

3. 0.70 – 1.00 = high stability / low change

It must be noted that though Spearman's rho ranged from −1 to +1, only positive values were considered because of the assumed minimum level of stability between one 3-year period and the following. Pearson's coefficient correlation was not used given that the rank order—of multi-word keywords with higher values of centrality—was measured at the ordinal level, rather than the interval or ratio levels. Regarding bivariate analysis, a low level of stability among periods in the journals was hypothesized.

RESULTS

Textual information from a total of 2 395 papers was analyzed. From the RSP, 689 papers; and from the SPM, 1 313 papers. Mexico's SPM had a higher publication rate than the Colombian RSP.

Centrality measures

As seen in Tables 1 and 2, the second step of the analysis included describing the 20 terms with the highest betweenness values (a complete listing of the 100 multi-term keywords with the highest centrality values for RSP and SPM is available upon request from the corresponding author). The first 20 keywords of Colombian thematic coverage showed some degree of stability during the period 2005 – 2010 given that at least 14 important multi-term keywords remained during 2005 – 2007 and 2008 – 2010. For the period 2008 – 2013, only 12 multi-term keywords remained among the central ones for both 3-year periods analyzed: 2008 – 2010 and 2010 – 2013 (Table 1). Most of the terms with higher centrality values dealt with demographics (middle-aged, preschool child, socioeconomic factors, etc.) and methodological (cross-sectional study or comparative study) issues. The same trend was observed for the Mexican thematic coverage.

Regarding keywords with higher centrality values for the Mexican public health journal (SPM), an inverse pattern was observed compared to the Colombian journal's thematic coverage; that is, the period 2008 – 2013 had a higher level of stability compared to the previous 6-year period. From 2005 – 2010, just 10 multi-term keywords remained among the terms with the highest centrality values, whereas during 2008 – 2013, 12 multi-term keywords appeared in the top 20 for both 2008 – 2010 and 2011 – 2013 (Table 2).

Co-word analysis-based mapping

For a different perspective on the results of the co-word analysis, Figures 1 and 2 show RSP's network sociograms for the periods 2005 – 2007 and 2008 – 2010. The network map for keywords in RSP (2005 – 2007) showed a low level of thematic diversity because they appeared in clearly delimited clusters and with a low degree of overlap. For example, the left-side cluster includes terms related to health studies (e.g., clinical trials, cohort analysis, disease transmission, genetics, and health survey) and health care infrastructure (e.g., hospitals, information, service, intensive care units, and urban health). The cluster located at the center of the network map includes terms related to demographics (e.g., adult, female, male, middle aged) and health service (e.g., health care quality, health promotion, health service, public health).

A different picture emerged from the network map for keywords in RSP (2008 – 2010) that showed a high level of thematic diversity. Clusters were not clearly delimited and there was a high level of overlap among them, especially at the center of the map where five clusters were located. Nevertheless, in the peripheral areas, two clusters were clearly defined. In the upper left area, there was a cluster of keywords associated with health services (e.g., consumer participation, health care delivery, health care quality, public hospital), and in the bottom right, another, but related to heart disease and associated factors (e.g., alcohol drinking, body mass, cardiovascular disease, fitness, hypertension, obesity). For the other areas of the network map, the high level of overlap made it difficult to identify clusters with clearly defined zones.

Bivariate analysis of stability and change

Correlation analysis showed mixed results due to a high level of stability in both journals for the period 2008 – 2013, but a medium level of thematic stability for 2005 – 2010.

The research hypothesis was partially accepted because in the first period (2005 – 2010), the level of thematic change showed moderate variability, as evidenced by the correlation coefficients: 0.545 and 0.593. That means that between the years 2005 – 2007 and 2008 – 2010, the keywords assigned to each paper appeared in a different order, and in some cases, they were different. However, in the second period (2008 – 2013), especially for RSP, thematic coverage was more stable than in the previous, suggesting that the keywords assigned in 2008 – 2010 and 2011 – 2013 appeared in almost the same order and were the same in most cases.

DISCUSSION

Previous co-word analysis of thematic coverage in public health used cluster techniques and social network analysis for examining co-word matrices, and one of them worked with 6-year periods, 2000 – 2005 and 2006 – 2011 (²¹21. Danell JB. Reception of integrative and complementary medicine (ICM) in scientific journals: a citation and co-word analysis. Scientometrics. 2014;98(2):807–21., ²⁴24. Zhang J, Xie J, Hou W, Tu X, Song J, Wang Z, Lu Z. Mapping the knowledge structure of research on patient adherence: Knowledge domain visualization based co-word analysis and social network analysis. PloS One. 2012;7(4):e34497. Available from: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0034497 Accessed 17 November 2015.
http://journals.plos.org/plosone/article... ). For this study, we utilized Spearman's rho correlation coefficient and considered 3-year periods (2005 – 2007, 2008 – 2010, 2011 – 2013) for measuring stability and change of thematic coverage. The research hypothesis was partially accepted given that just one period showed a significant degree of stability in thematic coverage. We found that most of the keywords were visible in both periods, even while the frequency and word relations changed. For example, for the group “diseases and conditions,” the new terms were “chronic disease, clinical practice, cardiovascular disease, apnea syndrome, etc.;” while in the group “research methods and techniques,” the new terms were “RCT, meta-analysis, cohort, double-blind, follow-up, pharmacy records, reliability, and validity, etc.”

The clusters of keywords discussed earlier are based on associations between nodes, which allow identification of significant nodes within the map of co-words. Although those words are compiled manually from each publication, the analysis is carried out by visualization software. Even when we did not analyze the full text of a document and only keywords were considered, the extracted terms provide a general overview of the topic.

This study provides compelling evidence of the effectiveness of using a combined approach for examining the dynamics of thematic coverage: centrality measures for identifying the main keywords and visual inspection for detecting the structure. In that sense, it is a useful approach for monitoring research themes in public health studies, not only in Latin America, but also in other geographic areas. For example, if a health ministry were interested in advances to address the Sustainable Development Goals, it could apply a similar approach to find specific studies focused on prevention programs for malaria or tuberculosis, and compare thematic coverage with prevalence or incidence ratios for those diseases.

We are not suggesting that there is a direct relationship between bibliometric indicators and advances in disease-prevention programs, but we are introducing a complementary perspective that goes beyond the descriptive level and explores the association with other demographic or public health indicators. One example of this is a recent study that examined the relationship between a set of bibliometric indicators and an index for health inequalities in 10 countries of Latin America (¹⁶16. Chinchilla-Rodríguez Z, Zacca-González G, Vargas-Quesada B, Moya-Anegón F. Latin American scientific output in Public Health: combined analysis using bibliometric, socioeconomic and health indicators. Scientometrics. 2015;102(1):609–28.). Even when the authors reported that they did not find a significant correlation among those indicators as measured by Spearman's rho, they offered a plausible explanation: research results do not necessarily influence the decisionmaking process at the ground level. This apparent lack of connection between theory and practice presents another challenge: how to translate research findings in a way that they can be adopted and incorporated at different levels of public health policy.

Limitations. There were three limitations to this study. First, there is a known publication bias against studies in non-English journals. To address this coverage bias, we decided to include all contributions, but focused on the keywords assigned to each paper. Given that the keywords appear in English as well as Spanish, there were no translation issues. In that sense, our findings can be generalized to English and Spanish content collected for the study. Secondly, regarding time periods, we found that some previous studies used 5-year periods for analysis (²⁹29. Mayta-Tristán P, Huamaní C, Montenegro-Idrogo JJ, Samanez-Figari C, Gonzáles-Alcalde G. Scientific production and cancer-related collaboration networks in Peru 2000-2011: A bibliometric study in Scopus and Science Citation Index. Rev Peru Med Exp Salud Publica. 2014;30(1):31–6., ³⁰30. Sanz-Valero J, Casterá VT, Wanden-Berghe C. Bibliometric study of scientific output published by the Revista Panamericana de Salud Pública/Pan American Journal of Public Health from 1997–2012. Rev Panam Salud Publica. 2014;35(2):81–8.). However, because our analysis focused on stability and change, we decided to work with 3-year periods. Lastly, we did not work with the Medical Subject Heading (U.S. National Library of Medicine, Bethesda, United States; MeSH) for co-word analysis because we extracted data from Scopus,^® a database that does not have a standardized list of keywords. Therefore, these results can only be generalized to non-MeSH keywords. Results using keywords from a normalized and structured list of subject headings may have differed from those presented here. In order to provide evidence for convergent validity, a future study could use data from MEDLINE^® to compare results between Scopus and MeSH keywords.

Conclusions

This study shows the effectiveness of a co-word analysis approach for analyzing stability and change in thematic coverage by two public health journals in Latin America. LAC health ministries and public health research institutes can replicate the procedure described in this review to verify whether or not the topics being published by their domestic or regional public health journals reflect their country's priorities, programs, and policies. Also, regarding the output of research funding, policymakers can use this procedure to provide additional indicators of whether resources allocated for public health programs have met their established evaluation criteria.

It is known that the research agenda of developed countries does not always correspond with that of developing countries, especially for matters such as immunization and disease surveillance. Future work should focus on validating this co-word approach by carrying out contextualized studies for public health issues relevant to each country.

Acknowledgements

The authors wish to thank Orlando Gregorio-Chaviano for his comments and Emma Virginia Martínez, Farita Huamán-Delgado, and Mark Neff for their proofreading of the manuscript.

REFERENCES

Publication Dates

History