ABSTRACT

Here, we describe the percentage of non-citation in Brazilian public health journals, a field that, until now, had not been investigated nationally or internationally. We analyzed articles, published between 2008 and 2012, of eight public health journals indexed in the scopus database. The percentage of non-citation differs between journals (from 5.7% to 58.1%). We identified four statistically distinct groups: História, Ciência, Saúde – Manguinhos (58% uncited articles); Physis: Revista de Saúde Coletiva, Interface, and Saúde e Sociedade (32% to 37%); Ciência & Saúde Coletiva and Revista Brasileira de Epidemiologia (16% to 17%); and Cadernos de Saúde Pública and Revista de Saúde Pública (6%). The non-citation in the first three years post-publication also varies according to journal. Four journals have shown a clear decline of non-citation: Cadernos de Saúde Pública, Ciência & Saúde Coletiva, Revista Brasileira de Epidemiologia, and Physis. Another three (Revista de Saúde Pública, Saúde e Sociedade, and Interface) presented an oscillation in non-citation, but the rates of 2008 and 2012 are similar, with different magnitudes. In turn, the journal História, Ciência, Saúde – Manguinhos maintains high rates of non-citation. Multidisciplinary journals attract more citation, but a comprehensive citation model still needs to be formulated and tested.

Journal Article; Scientific and Technical Publications; Citation Databases; Systems for Evaluation of Publications

INTRODUCTION

From the beginning of the citation analysis, Garfield¹1. Garfield E. Uncitedness III – the importance of not being cited. Essays Inf Sci. 1973;1:413-4. Available from: http://www.garfield.library.upenn.edu/essays/V1p413y1962-73.pdf
http://www.garfield.library.upenn.edu/es... already showed concern about articles that were not cited. Hamilton²2. Hamilton DP. Publishing by– and for?– the numbers. Science. 1990;250(4986):1331-2. https://doi.org/10.1126/science.2255902
https://doi.org/10.1126/science.2255902... , in a citation analysis in the Web of Science, found 55% of uncited articles over the first five years post-publication. This proportion, however, varied according to the area of knowledge: 47.4% in the exact and biological sciences, 74.7% in the social sciences, and 98.0% in the arts and humanities³3. Hamilton DP. Research papers: who’s uncited now? Science. 1991;251(4989):25. https://doi.org/10.1126/science.1986409
https://doi.org/10.1126/science.1986409... . A reanalysis of Hamilton’s data, restricted to original articles, estimated lower rates of non-citation: 22.4% (exact and biological sciences); 48% (social sciences); and 93% (arts and humanities)⁴4. Pendlebury DA. Science, citation and funding [letter to the editor]. Science. 1991;251(5000):1410-1. https://doi.org/10.1126/science.251.5000.1410-b
https://doi.org/10.1126/science.251.5000... . Although the non-citation rates estimated by Hamilton have been criticized²2. Hamilton DP. Publishing by– and for?– the numbers. Science. 1990;250(4986):1331-2. https://doi.org/10.1126/science.2255902
https://doi.org/10.1126/science.2255902...

3. Hamilton DP. Research papers: who’s uncited now? Science. 1991;251(4989):25. https://doi.org/10.1126/science.1986409
https://doi.org/10.1126/science.1986409...

4. Pendlebury DA. Science, citation and funding [letter to the editor]. Science. 1991;251(5000):1410-1. https://doi.org/10.1126/science.251.5000.1410-b
https://doi.org/10.1126/science.251.5000...

5. Abt HA. Science, citation, and funding. Science. 1991;251(5000):1408-9. https://doi.org/10.1126/science.251.5000.1408-a
https://doi.org/10.1126/science.251.5000... ^-66. Garfield E. I had a dream... about uncitedness. The Scientist. 1998 [cited 2017 Aug 6];12(14):10. Available from: http://garfield.library.upenn.edu/commentaries/tsv12(14)p10y19980706.pdf
http://garfield.library.upenn.edu/commen... , the idea remains that, in fact, most of the articles were not cited or will not be cited⁷7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82..

However, there are clear indications that non-citation has been continuously declining since the 1980s in the fields of natural sciences, engineering, medicine, and social sciences, but not in the humanities⁷7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82..

The field of public health, multidisciplinary in nature, may be a relevant case for the evaluation of non-citation. However, it has not yet been investigated. Here, we describe the phenomenon of non-citation in Brazilian public health journals.

METHODS

The selected journals had to obey two criteria: a) to be part of the Collective Health Editors Forum of the Brazilian Association of Collective Health (created in 2014, access at https://www.abrasco.org.br/site/2014/11/forum-de-editores-de-saude-coletiva-carta-de-sao-paulo/), which represents the Brazilian public health journals; and b) be indexed in the Scopus database, chosen by the multidisciplinary scope (more than 20 thousand journals indexed), relevance in the scientometric evaluation, and availability of the metadata for analysis. Thus, we selected eight journals (abbreviations by the National Library of Medicine Catalog - NLM): Cadernos de Saúde Pública (Cad Saude Publica); Ciência & Saúde Coletiva (Cien Saude Colet); História, Ciência, Saúde – Manguinhos (Hist Cienc Saude); Interface: Comunicação, Saúde, Educação (Interface); Physis: Revista de Saúde Coletiva (Physis); Revista Brasileira de Epidemiologia (Rev Bras Epidemiol); Revista de Saúde Pública (Rev Saude Publica); and Saúde e Sociedade (Saude Soc).

From these, data were extracted (in November 2015) referring to two types of articles: the original articles and the reviews, published between 2008 and 2012. We chose 2008 as the initial date because all selected journals were indexed in Scopus. The end date (2015) ensured a minimum of three years for the most recent (published in 2012) to receive citations.

We retrieved 5,736 article records, of which 66 were deleted for duplicity, with a final sample of 5,670 articles. From each, the following information was obtained: article title, authorship, publication year, pagination, journal title, and number of citations of each article (per year and total).

To validate the consistency of the database, a subsample with an arbitrary value of 10% of the articles (567/5.670) was analyzed. We found some inconsistencies. The most frequent was the delay in indexing, i.e., new records of citations of articles appeared between the period of the sample extraction and that of the evaluation of the subsample. Thus, 60 articles presented late citations, equivalent to 10.6% of the 567. Of these 60 articles, four (0.7%) would no longer be classified as uncited. Other, less frequent, problems were: year of citation divergent from the date indicated in Scopus (0.5%) and a citation counted twice by Scopus (0.5%). The non-citation rate of the eight journals remained: 18.6% in the sample and 18.5% in the subsample. Given the small magnitude of problems and their insignificant effect on the estimates, the data studied were those extracted from the Scopus base, without corrections.

We calculated, for each article and in each journal, the proportion of non-citation in the whole period and the proportion of non-citation in the first three years after publication.

RESULTS

We observed a major difference in non-citation rates between the journals, ranging from 5.7% to 58.1%, a difference of 10 times between the lowest and the highest rates (Figure 1). Also noteworthy is the distribution in four statistically distinct groups: in the group with the highest non-citation, is the journal Hist Cienc Saude. Another group, comprised of social science health journals, included Physis, Interface, and Saude Soc. In an intermediate situation, we have Cienc Saude Colet and Rev Bras Epidemiol. The lowest non-citation indexes were identified in the Cad Saude Publica and Rev Saude Publica journals.

The non-citation behavior in the first years after publication also varies according to the journal (Figure 2). It is interesting to note that the non-citation dynamics in the last three years differ from the groups observed in Figure 1. There are four journals with a clear decline in non-citation: Cad Saude Publica, Cienc Saude Colet, Rev Bras Epidemiol, and Physis. In other three (Rev Saude Publica, Saude Soc, and Interface), there is an oscillation in non-citation, but rates in 2012 are similar to those of 2008, albeit with different magnitudes. Finally, the journal Hist Cienc Saude maintains high rates of non-citation.

DISCUSSION

Brazilian journals have marked differences between them in terms of non-citation but behave similarly to what has already been described in the literature regarding the peculiarities of each area. In fact, Pendlebury⁴4. Pendlebury DA. Science, citation and funding [letter to the editor]. Science. 1991;251(5000):1410-1. https://doi.org/10.1126/science.251.5000.1410-b
https://doi.org/10.1126/science.251.5000... , in an analysis of ISI-indexed journals between 1984 and 1988, already found these disparities between areas of knowledge, in which natural sciences, engineering, and medicine had lower rates, followed by the social sciences, and with the greatest non-citation in the humanities. Larivière et al.⁷7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82. found similar groupings decades later.

The journal of humanities in the field of public health, which prioritizes the publication of articles related to documentation, research, and museum study in the history of science and health, has the highest overall non-citation rate but the rate has been decreasing in the first three years after publication. This pattern is comparable to the humanities journals analyzed in the Web of Science database⁷7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82.; that is, in the Brazilian case, 79% of articles not cited in Hist Cienc Saude versus 90% in the humanities journals mentioned by Larivière et al.⁷7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82. Likewise, our health social science journals have similar behavior to those in the same field of knowledge indexed in the Web of Science, both in rates and in the reduction trend of non-citation.

On the other hand, it is noteworthy that our multidisciplinary public health journals (Rev Saude Publica and Cad Saude Publica) have comparatively lower levels of non-citation than those found by Larivière et al.⁷7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82. These authors found a rate of uncited articles for medicine in general, and not their specialties, of around 11%. These differences can hardly be attributed to the different bibliometric sources, Scopus in our study and Web of Science in Larivière et al.⁷7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82., because there is a high correlation between the impact indicators of these two bases⁸8. Archambault E, Campbell D, Gingras Y, Larivière V. Comparing bibliometric statistics obtained from the Web of Science and Scopus. J Assoc Inf Sci Technol. 2009;60(7):1320-6. https://doi.org/10.1002/asi.21062
https://doi.org/10.1002/asi.21062... . The reasons why Rev Saude Publica and Cad Saude Publica are able to attract more citations still need to be studied, but they seem to follow the relationship in which the more interdisciplinary, the greater the ability to attract citations⁹9. Chen S, Arsenault C, Larivière V. Are top-cited papers more interdisciplinary? J Informetr. 2015;9(4):1034-46. https://doi.org/10.1016/j.joi.2015.09.003
https://doi.org/10.1016/j.joi.2015.09.00... .

Similarly, the journals Cienc Saude Colet and Rev Bras Epidemiol, both recent in the field of publishing in the area (created in 1996 and 1998, respectively), have similar characteristics regarding the overall rate of non-citation and the reduction of articles without citation in first three years. Cienc Saude Colet approaches multidisciplinary public health journals, with fewer articles not cited, probably due to its equally multidisciplinary nature. This journal undeniably has a strong tradition of disseminating production in the social sciences in health. Rev Bras Epidemiol, which publishes in an area that has the article as one of its main means of dissemination, has been attracting more and more citations. The monitoring of non-citations of these two journals, henceforth, may elucidate if they will be in the group of multidisciplinary journals.

Our analysis privileged non-citation, as we intend to bring up the proportion of articles that seem to have had no impact or recognition in the scientific community. Of course, this may be a limitation, since the analyses focused on the metrics of the citations (indicators such as index H, SJR, FI, cites per documents, among others) may reach different bibliometric classifications.

Pendlebury⁴4. Pendlebury DA. Science, citation and funding [letter to the editor]. Science. 1991;251(5000):1410-1. https://doi.org/10.1126/science.251.5000.1410-b
https://doi.org/10.1126/science.251.5000... evaluates that a “certain level of non-citation in a journal is probably more an expression of the process of creation and dissemination of knowledge than any kind of performance measure,” recognizing that books are more important in the delivery of scientific communication among social and humanities scientists than in the exact and biological sciences. Thus, non-citation rates would represent more the forms of citing than the acceptance or rejection of the knowledge produced. The reasons for these differences have been speculated, but there are no comprehensive models to explain them. This gap opens room for further investigation.

REFERENCES

Publication Dates

History