Version from October 2007

1 Introduction

This document describes the criteria, policies and procedures for the evaluation of scientific journals, adopts by the SciELO Program for the admission and permanence of journals in the SciELO Public Health Collection <www.scielosp.org>.

The criteria, as well as the policies and procedures for its application, were discussed during the Seminar on Criteria for Selection and Evaluation of Scientific Journals, which took place in April 1999 in the FAPESP - Fundação de Amparo à Pesquisa do Estado de São Paulo <http://www.fapesp.br/> - headquarters in São Paulo, and approved by both FAPESP and BIREME - Latin American and Caribbean Health Sciences Information Center <www.bireme.br>. They will be henceforth referred to as SciELO Public Health Criteria, as their formulation and application have a close relationship with scientific publication in this field of knowledge.

2 SciELO Public Health Criteria Objectives

The SciELO Public Health Criteria have been defined under the context of the SciELO Program, coordinated by FAPESP and BIREME, with the support of CNPq - National Council for Scientific and Technological Development <www.cnpq.br>, whose main objective is to contribute in the development of national scientific research, through improvement and extension of the means of publication, dissemination and evaluation of scientific output, intensivelly using electronic publishing options.

The SciELO Program seeks to radically increase the visibility, accessibility and national and international credibility of scientific output from Latin America and the Caribbean in the short term, through the integrated online publication in national or regional collection of scientific journals. In the long term, the program seeks to contribute for the increase of the impact of national scientific output.

For the fulfillment of the SciELO Program objectives, it is essential to promote the improvement of national scientific communication in every aspect, making an effort to identify, stimulate and develop a central collection of scientific journals whose quality standards catch up with those of topnotch international scientific journals. Thus, the evaluation criteria, policies and procedures constitute a main action line of the SciELO Program.

The SciELO Public Health Criteria hold as specific objective to contribute for the formation and development of a central collection of scientific journals in Public Health, which includes:

a. To subsidize the evaluation of scientific journal, as to aid the determination of their admission in the SciELO Public Health Collection;
b. To track the development of scientific journals included in the SciELO Public Health Collection, as to determine their permanence in the collection;
c. To provide performance indicators of the SciELO Public Health Collection as a whole and of each journal that compounds it.

The contents and application of the SciELO Public Health Criteria shall accompany the dynamics of the SciELO Program, particularly regarding the formation and development process of the SciELO Public Health Collection.

3 The SciELO Public Health Advisory Committee

The Advisory Committee, based on the application of the SciELO Public Health Criteria, aims to propose:

a. The inclusion of new scientific journals in the SciELO Public Health Collection;
b. the exclusion of scientific journals from the SciELO Public Health Collection;
c. modifications in the admission and permanence evaluation criteria;
d. performance indicators of the journals indexed in the SciELO Public Health Collection, to be periodically published;
e. the definition and improvement of the committee operation, in order to efficiently fulfill the objectives here described.

The Committee has the following permanent composition:

a. Operational Coordinator of the SciELO Program, as chairman;
b. Eleven Scientific Editors, each one representing one of the journals indexed in the SciELO Public Health Collection.

The Committee may also count, in its meetings and activities, with the participation of external experts, ad hoc reviewers and consultants, whenever necessary.

4 Scope of the SciELO Public Health Collection

The SciELO Public Health Collection includes scientific journals that mainly publish original papers from scientific research outputs and other original significant contributions to the specific area of the journal. The process of review and approval of publishing of the contributions must be conducted by peer reviewers.

Journals from Latin American, Caribbean and Iberian countries that mainly publish public health-related articles, including politics, health planning and management, social sciences and public health, promotion and interventions in health and epidemiology are eligible for inclusion in the collection. The collection is also tilted to accept public health scientific journals from other parts of the world.

The collection of each journal in SciELO Public Health must start from its admission in the collection, and retrospective fascicles published from 1997 on will also be added. The articles may be written in English, Portuguese and Spanish.

The journals of the SciELO Public Health Collection don't have to be necessarily paper published.

5 SciELO Public Health Collection admission criteria

5.1 Criteria for automatic admission

The SciELO Public Health admission criteria consider former evaluations journals have been submitted to.

A journal is automatically eligible for admission in case it is indexed in MEDLINE/Index Medicus.

5.2 Criteria for admission of journals indexed in a national certified SciELO Collection

Every journal indexed in a national certified SciELO collection may be admitted in the SciELO Public Health Collection, once it is confirmed that it fulfills the national criteria and its publishing is up to date.

Currently, count as national certified collections the following: SciELO Argentina, SciELO Brazil, SciELO Chile, SciELO Colombia, SciELO Cuba, SciELO Spain, SciELO Portugal e SciELO Venezuela.

Additional information regarding criteria for certification of SciELO websites are available at: <http://www.scielo.org/criterios_certificacao_sites_pt.html>.

5.3 Criteria for admission of journals in the SciELO Public Health collection

When a journal is not eligible for inclusion in the SciELO Public Health collection based on the criteria described in the 5.1 and 5.2 topics, it shall be evaluated based on the following indicators:

5.3.1 Scientific content

Journals shall mainly publish articles resulting from scientific research and/or significant research in their areas of expertise. Journals may also include other types of contributions such as review articles, reports, releases and case studies which will not be considered original contributions.

The SciELO Public Health Advisory Committee may request additional peer reviewing to verify the predominance of original contributions.

5.3.2 Peer reviewing

Evaluation and approval of the contributions submitted must be performed by peers. The journal must formally specify the procedure used by peers for review and approval of contributions. Once a journal is accepted for inclusion in the SciELO collection the review procedure by the peers will be documented. It is mandatory to indicate the main schedules of the refereeing process, including reception and approval dates.

5.3.3 Editorial Board

Members of the journal’s editorial board shall be made public. Members shall be well-known national and international specialists duly identified in the publication.

Journals whose boards are made up mainly of members belonging to the same institution and/or whose articles come mainly from the same institution or geographical region will not be accepted for inclusion in the collection

5.3.4 Periodicity

Periodicity is an indicator of the scientific output flow that depends on the specific scope covered by the journal. It is also an indicator related to the opportunity and communication speed.

The minimum required periodicity and the minimum number of articles published per year are described below:

Thematic Area	Periodicity		Number of articles per year
	minimum	desired	minimum	desired
Public Health	trimestral	bimestrial	60	72

5.3.5 Minimum number of publications

The journal must have at least four numbers published to be considered for evaluation.

5.3.6 Punctuality

The journal must come out punctually, according to its periodicity.

5.3.7 Abstract, key words and title in English

The articles must contain title, abstract and key words in English and in the original language of the article.

5.3.8 Standardization

The journal must specify the standards followed in the presentation and structure of the texts, and in the presentation of bibliographic references and descriptors, so that it is possible to evaluate the compliance to the indicated standards.

5.3.9 Authors’ afiliation

The articles must contain complete information about the authors’ afiliation, including institution membership, e-mail, city and country.

5.3.10 Citations received

The journal must present a citation index compatible to other journals of the same scope, which is verified based on the citations received in other articles published in the SciELO Public Health Collection.

5.4 Procedures of the admission process

The admission of a journal in the SciELO Public Health Collection will only happen after a positive feedback of the SciELO Public Health Advisory Committee.

The evaluation procedures comprise three aspects: (1) format, related to the adequacy to the standards; (2) endogeny, related to the institutional and geographic concentration of the editorial boards and authors; and (3) contents, to ensure scientific quality.

The evaluation of format aspects includes the analysis of a set of characteristics, based on studies and international standards of scientific journals editing. There are mandatory items and journals are usually accepted with the fulfillment of 80% of the characteristics.

The endogeny evaluation is based on the affiliation declared by the editorial board, reviewers and authors. The canvass of trend of institutional and geographic concentration of these elements is considered a negative result in the admission of a journal in the collection.

The journals that satisfactorily meet these aspects will proceed to the content evaluation, based on a minimum number of opinions of 2 experts in public health, regarding the following characteristics:

a. Representativeness of the editorial board, reviewers and authors;
b. Scientific character of articles in the journal;
c. Peer reviewing process;
d. Relevance in the development of the area.

The SciELO Unit will contribute with the work of the Committee, by organizing the information and documentation required. Only the journals that fulfill the requirements from topics 5.3.1to 5.3.10 will be submitted to the Advisory Committee analysis.

The Committee feedback may contain improvement recommendations and modifications in the journals that shall be implemented in the period specified in the Committee’s report.

6 Criteria for performance evaluation for permanence in SciELO Public Health

As a rule, all the indicators adopted in the evaluation process for inclusion of titles in SciELO Public Health are also applied for deciding on its maintenance in the collection.

Adoption of the Committee's recommendations for the journal’s improvement will be taken into consideration for maintenance assessment of the journal.

The following are indicators associated with the specific performance of journals in SciELO, and will be used in maintenance evaluation:

6.1 Punctuality in file delivery

Punctuality is measured by the dates of arrival of electronic files of a journal in SciELO. File submission shall strictly comply with periodicity of the journal. SciELO shall report any delays in file submission.

Journals that are late will be evaluated by the SciELO Advisory Committee, and are subject to exclusion from the SciELO collection.

6.2 Journal use indicator

Journal use is measured monthly by the number of times it is accessed or viewed online. When the use of a journal is repeatedly low and/or tends to decrease when compared to other journals featuring the same subjects, the SciELO Advisory Committee will evaluate whether it should remain in the collection. The Committee will then suggest a study to find out the reasons for the decrease and will either propose possible solutions, or recommend the journal to be excluded from the SciELO collection.

6.3 Impact indicators

The impact indicator of a journal, measured by the number of citations received must be assessed together with the journal titles in the same field of expertise.

An increase in the impact factor or a position standing in the same average of similar journals will be considered a productive performance, which ensures the journal maintenance in the SciELO collection.

7 Title exclusion report process

Exclusion of a journal from the SciELO collection can only be carried out after a decision report is issued by the SciELO Advisory Committee.

In the case of unfavorable results obtained in the performance appraisal, the journal will be notified by the Committee on the aspects to be improved within the time period stipulated.

8 Appeals

Journal editors may appeal at any time opposing a decision by the SciELO Advisory Committee both in relation to inclusion and exclusion processes.

Appeals will be examined by the Advisory Committee and the decision may be re-examined. The Committee's report will be sent to the journal editor.

9 Re-inclusion process

Journals excluded from the SciELO collection can be re-introduced provided they meet the inclusion criteria required by SciELO Public Health. The SciELO Advisory Committee will analyze re-inclusion.

Re-inclusion of a journal title will not be made effective immediately after its exclusion from the SciELO collection. In order to be re-evaluated, a journal must demonstrate its compliance with the criteria for at least four consecutive issues or one year.