Abstract
The National Commission for incorporation of Health Technologies (CONITEC), established in 2011, advises the Ministry of Health in decisions related to the incorporation, exclusion or change of medicines, products and procedures in the Unified Health System (SUS).The study investigated the decision-making process, profile of demands and incorporation of new medicines in the SUS from January/2012 to June/2016, based on data available on the CONITEC website. All submissions were evaluated and characterized by technology and applicant type. The incorporations were analyzed according to the Anatomical-Therapeutic-Chemical classification, International Classification of Disease of the clinical indication and active record in the National Health Surveillance Agency. In the period, 485 submissions were received, 92.2% concerning requests for incorporation and 62.1% for medicines, of which 93 (30.1%) received a favorable recommendation for incorporation. Domestic demands were more successful than externally originated ones. Six unregistered drugs were incorporated. Infectious and parasitic diseases and musculoskeletal diseases constituted the main clinical indications. The recommendation of incorporation occurred mainly based on the additional clinical benefits and low budget impact.
Medicines, Health technology assessment; Decision making, Unified Health System
Introduction
The 1988 Federal Constitution of Brazil affirms the right to health as being a social right and establishes that the state ought to ensure health care, including public financing that allows universal access and equity in the distribution of medicines and other technologies in health.
The impossibility of the Brazilian National Health System (SUS), that is chronically underfunded, to meet health needs and growing demands for new technologies has triggered the phenomenon of health litigation for the provision of medical procedures or therapies that have not been incorporated into the public health system. Many of these legal cases seek to ensure the right of access for patients to expensive medicines that are not always available on SUS and many times without any proven benefits or in some cases are considered deleterious11. Biehl J, Petryna A, Gertner A, Amon JJ, Picon PD. Judicialisation of the right to health in Brazil. Lancet 2009; 373(9682):2182-2184.,22. Figueiredo TA, Pepe VLE, Osorio-de-Castro CGS. Um enfoque sanitário sobre a demanda judicial de medicamentos. Physis 2010; 20(1):101-118..
This situation has led to the approval of Law n° ١٢٤٠١ in April ٢٠١١ which aimed to regulate the concept of integrality and make therapeutic assistance more readily available. It also regulated the incorporation of health technologies in the context of SUS33. Guimarães R. Technological incorporation in the Unified Health System (SUS): the problem and ensuing challenges. Cien Saude Colet 2014; 19(12):4899-4908..
This law has established the National Committee for Health Technology Incorporation (Conitec) with the task of providing assistance to the Brazilian Ministry of Health (MoH) about the decisions on incorporation, exclusion or changes in new medicines, medical devices and procedures. It is also required to provide assistance in the constitution or changes in the standard treatment guidelines and in updating the National List of Essential Medicines44. Brasil. Lei nº 12.401, de 28 de abril de 2011. Altera a Lei no 8.080, de 19.09.1990, para dispor sobre a assistência terapêutica e a incorporação de tecnologia em saúde no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 24 abr.,55. Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez.. Also in 2011 the Decree n° 7646 provided regulation in the composition, competencies, and function of Conitec55. Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez..
These regulatory instruments defined the flow, criteria, and deadlines for the evaluation and incorporation of technologies in the public health system. The operational structure of Conitec consists of two bodies: the Plenary and the Executive Secretariat, with the first being responsible for making recommendations on the incorporation, disinvestment or alteration of technologies for SUS. The Plenary assembly has 13 members, including representatives from seven secretariats of the Ministry of Health, from its regulatory bodies (the Brazillian Health Regulatory Agency — Anvisa and National Regulatory Agency for Private Health Insurance and Plans — ANS), from the National Council of Health Secretaries (CONASS) and National Council of Municipal Health Secretaries (CONASEMS), and representatives from civil society, from the Federal Council for Medicine (CFM) and the National Council of Health (CNS). The society´s involvement in Conitec´s decisions is provided not only by the presence of those two representatives, but also by social participation in the consultations and public hearings55. Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez..
All requests for incorporation/exclusion of technologies submitted to Conitec implies the opening of an administrative process. A set of requirements must be fulfilled by the applicant and some documents must be submitted, including: number and validity of the technology registration in Anvisa; scientific evidence showing that the technology is at least as effective and safe as those available in SUS for the intended use; an economic evaluation study comparing the requested technology with those available in the SUS, and the price set by the Drug Market Regulation Chamber (CMED) of Anvisa, in the case of medicines55. Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez..
The Plenary´s initial recommendations must be submitted for public consultation for 20 days (which can be reduced to 10 days when the process is classified as an “emergency”). After contributions are evaluated, Conitec deliberates once again, and its final recommendation is forwarded to the Secretary of Science, Technology and Strategic Inputs of MoH (SCTIE) for a final decision, which is subsequently published in the Government’s official gazette. If the result is favorable towards incorporation, the technology must be made accessible to the population in a maximum of 180 days55. Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez..
The creation of the Committee represented an important step in the development and institutionalization of Health Technology Assessment (HTA) in the country. HTA aims to support the health system in taking financial decisions, and in the aquisition and appropriate use of technologies. It also has the task of disinvesting in obsolete technologies or those which are ineffective. Ir can also contribute to the increase in transparency and accountability in the decision-making process, since HTA supports the development of policies based on evidence66. International Network of Agencies for Health Technology Assessment (INAHTA). The International Network of Agencies for Health Technology Assessment. The Influence of Health Technology Assessment. A conceptual paper. 2014. [acessado 2016 set 22]. Disponível em: http://www.inahta.org/wp-content/uploads/2014/03/INAHTA_Conceptual-Paper_Influence-of-HTA1.pdf
http://www.inahta.org/wp-content/uploads... .
According to information of Conitec’s actions published in 2014, more than 100 new technologies were incorporated into SUS up until that period. This amount corresponds to about three times the annual average of new technologies incorporated in the period before the creation of Conitec77. Brasil. Ministério da Saúde (MS). Balanço Conitec: 2012-2014. Brasília: MS; 2014.. Changes that occurred in both form and process of incorporation of technologies based on the constitution of Conitec were significant, changing the dynamics and the quality of the entry of new products in SUS. All of these modifications, however, are still quite recent, and its potential impact has not yet been sufficiently studied due to the limited time.
This study examined the recommendation process and the profile of the incorporations of new medicines into SUS approved by Conitec between 2012 and June 2016.
Methodology
This is an exploratory study that is both descriptive and retrospective. It takes a qualitative-quantitative approach that is related to the requests presented to Conitec that occurred between January 1st, 2012 (the commencement year of operation) and June 30th, 2016.
The main data sources were public information available on Conitec´s website (http://conitec.gov.br/) including: (i) the records of the requests for health technologies that were submitted, (ii) technical recommendation reports from Conitec, (iii) officially published decisions from the SCTIE Secretary published in the Government´s official gazette, and (iv) contributions given through public consultations.
Information obtained from the electronic pages of the Conitec website was extracted by two independent reviewers on a database specially developed for this purpose, using EpiData® software. Divergences in the extractions were resolved by consensus.
One same technology may have been submitted to Conitec more than once, in cases of requests for distinct therapeutic indications, requests made in more than one occasion by different applicants, requests for changes in the target population or even the reapplication of a request that had been denied in a previous analysis. In addition, some requests that included different technologies in the same submission were also individualized, so as to constitute separate demands. In this way, individual units of analysis used in this work were demand/technology specific. The reason of request was categorized according to specific classification made by Conitec: incorporation, exclusion and widening of use, with this last one encompassing both changes in indication/use requests as of a new presentation/model.
The requested technologies were initially classified as medicine (including vaccines and biological medicines), medical device, medical procedure or STG/use protocols. The “STG/use protocol” aggregated Clinical Protocols and Therapeutic Guidelines, Diagnostic and Therapeutic Guidelines in Oncology and National Guidelines (as called by Conitec). These were guidance documents on the best practices to be followed by health professionals and managers, and protocols of use (more strict normative documents, establishing criteria, parameters and standards for the use of a specific technology for certain diseases/conditions). For some specific analysis, technologies were subdivided into “medicine” and “different from medicine”, that encompassed all the other technologies.
All the applicants´ names were registered, and the requests were categorized as: (1) “internal” when from MoH secretariats and departments, agencies connected to MoH (Anvisa and ANS), or from a state (SES) or municipal (SMS) health secretariats, and (2) “external” (other organs of the Federal Government, the judiciary, health institutions and teaching and research institutions, medical and professional societies, health care professionals, patients’ associations and other non-governmental organizations, and patients or their relative/caregiver). The status of the requests on 06/30/2016 was classified into two broad groups: “processes in course” and “closed processes”. Processes in course included the following situations: (a) analysis of document conformity, (b) compliant processes awaiting analysis by the Conitec technicians, (c) under public consultation, (d) under review after public consultation, and (e) awaiting scrutiny of the final recommendation by the SCTIE secretary. “Closed processes” included (a) discontinuance for (i) formal non-conformity of the documentation, (ii) due to a request from the applicant, (iii) or based on a decision from Conitec, (b) decisions of incorporation, (c) decisions of non-incorporation, (d) technology exclusion decisions, and (e) the request has already been incorporated in SUS.
The request for medicines that were the object of final recommendations by the Conitec Plenary constituted the second plan of analysis. The medicines were categorized by the Anatomical Therapeutic Chemical (ATC) Classification System up to the fifth level (active substance) using the WHO Collaborating Centre for Drug Statistics Methodology88. WHO Collaborating Centre for Drug Statistics Methodology. Anatomical-Therapeutic-Chemical classification index. Oslo: Norwegian Institute of Public Health; 2016. [acessado 2016 set 15] Disponível em: http://www.whocc.no/atc_ddd_index/
http://www.whocc.no/atc_ddd_index/... . The clinical condition registered in the request was classified according to the 10th version of the International Classification of Diseases (ICD)99. Brasil. Ministério da Saúde (MS). Departamento de Informática do SUS. 2016. [acessado 2016 set 15]. Disponível em: http://www.datasus.gov.br/cid10/V2008/cid10.htm
http://www.datasus.gov.br/cid10/V2008/ci... .
The carrying out of public consultations was checked as well as the total number of responses received.
The final recommendation referring to medicines submitted for a decision in the Plenary was categorized as: (i) incorporation of a new drug that was not present on the list for financing by SUS up to the date of the decision, (ii) incorporation of a new indication, in cases of medicines already present on the list but for different indications to the one approved by Conitec, (iii) maintenance of the drug on the list for financing, when it was found to have already been incorporated, (iv) non-incorporation of the medicine on SUS, (v) exclusion of medicine for a specific indication/ICD and (vii) exclusion just for a specific presentation of the medicine. The first three categories were subsequently aggregated as “incorporation”, with the rest being aggregated in the category “different from incorporation”.
All justifications present in the final recommendation reports identified as motivating the decision for “incorporation” by Conitec were registered. Finally, it was verified whether Conitec’s final recommendation was accepted by the SCTIE secretary in its final decision published in the Government’s Official Gazette. Stata® software version 12 was used for data tabulation and analysis. For descriptive analysis, we used absolute and relative frequencies for discrete variables and mean, median and standard deviation values for continuous variables. In the analysis of categorized variables, Pearson’s Chi-squared test was used considering significance level of 5%.
Data on requests, technical recommendation reports and contributions from the public consultations can be publicly accessed and are freely available on Conitec’s website, exempting this study from the appreciation and prior approval of the Ethics Committee in Research.
Results
Conitec received 485 submissions in the period of the study, 92.2% of which were related to requests for incorporation of new technologies. Medicines were the main object of application (62.1%) independent of the request´s nature. Even though the requests for exclusion on SUS represented just 7% of the requests, they were primarily concentrated on medicines (Table 1).
More than 95% of submissions were found to have been discontinued on 30/06/2016. Of the 17 submissions that were without final decisions yet, 14 were of medicines. Of the 301 requests related to medicines, 287 (95.3%) were discontinued. About 30% (86 processes) had early discontinuance, generally for non-conformity to procedures (53.5% of the cases) or for decisions taken by the applicant (36%). Twenty-one submissions for medicines had two or more different pleintiffs. Internal requests corresponded to 52.2% of submissions related to medicines during the period (Table 2). For external requests, those from the pharmaceutical industry were predominant (40.9% of total).
Two hundred and one processes related to medicines were deliberated by the Plenary (66.8%). Of these, 60.7% were originated internally, most of them being from the Health Care Secretariat (Secretaria de Atenção à Saúde -SAS) (23.4% of requests with deliberation). Ninety-three medicines received favorable recommendations for incorporation, including six vaccines and two immunobiological medicines (30.9% of total requests for medicines and 46.3% of those submitted to the Plenary). Internal requests for incorporation of medicines were more successful when compared with those that were originated externally: 82.8% of the requests that received favorable decisions for inclusion were from MoH, Anvisa or SES/SMS (p = 0.000) (Table 3).
Approximately 70% (139/201) of the decisions about medicines were the object of public consultations, while 62 went through more simplified processes, with no significant statistical differences in relation to favorable recommendations to incorporation (p = 0.186) (Table 3).
The number of contributions received in public consultations for medicines with favorable recommendation for incorporation were extremely diverse, varying from 0 to 530 (average of 108, median of 61). Nine medicines incorporated received more than 250 responses (just one had more than 500): fingolimod for the 3rd line of treatment of multiple sclerosis (530 responses); cinacalcet and paricalcitol for therapy related to secondary hyperparathyroidism on chronic renal diseases requiring dialysis; antiviral dolutegravir sodium for HIV-related infections; palivizumab for the prevention of infections related to the respiratory syncytial virus, and sofosbuvir, daclatasvir and simeprevir indicated for chronic hepatitis C. There was also rivastigmine as a skin patch for the treatment of light or moderate Alzheimer´s disease. Medicines with unfavorable recommendations for incorporation had an higher average number of responses (136); 21 medicines received more than 250 contributions with six receiving more than 500.
During the period, there was the recommendation for incorporation of six medicines without active registers on Anvisa (3% of the deliberations bythe Plenary) (Table 3). All of those requests came from MoH secretariats (two from the Health Surveillance Secretariat (Secretaria de Vigilância em Saúde - SVS), three from the SAS and one from SAS/SCTIE together) and four were made using the simplified process without public consultation.
Infectious and parasitic diseases (IPD) and musculoskeletal disorders accounted for 48.4% of all requests for medicines with a positive recommendation for incorporation. These two groups jointly with neoplasms, mental disorders and behavior and circulatory diseases made up 64.5% of incorporated medicines. Regarding the proportion of approvals amongst the requests that were subject to a deliberation, products related to mental health and IPD stood out (approximately 80% of the approved requests) (Table 3).
Antineoplastics and immunomodulatory agents (group L) accounted for more than a third of medicines that were incorporated as well as being the most common group amongst the requests without favorable recommendations for incorporation. We also took note of the participation of J (anti-infectious for systemic use) and N (the nervous system) groups (Table 3).
Regarding the distribution of deliberations and decisions by Plenary involving medicines, 2013 (28.9%) and 2015 (22.9%) stood out. However, 2012 showed a larger proportion of favorable recommendations for incorporation (68.4%) followed closely by 2015 (60.9%) (Graphic 1).
Distribution of medicines with recommendations submitted to the Plenary per year and final recommendation of the decision, Conitec, 2012 a 2016*.
Close to 75% of medicines incorporated by June 2016 had two or more justifications for this recommendation in the reports (an average of 2.2). The justifications that stood out (Table 4) were additional clinical benefits (40.9%), low impact on finances-budget (25.8%), a clinical need not filled by SUS and recommendations for incorporation in other countries (both with 21.5%).
One or more constraints to incorporation were present in 26.9% of recommendations for medicines approved during the period (an average of 2.4 constraints). The main condition stated in the reports to make inclusion on SUS feasible were the reduction of the selling price to the government and the incorporation connected to developing and updating a STG, specifying conditions for access and use.
The SCTIE secretary accepted all the recommendations made by Conitec’s Plenary without exception.
Discussion
Since 2006, with the creation of the former ‘Comissão de Incorporação de Tecnologias’ (CITEC) from the Ministry of Health, there have been flows and routines established for the analysis of the incorporation of health technologies. With the establishment of Conitec, the submission of requests to the Committee became a mandatory and indispensable process for the incorporation of technologies on SUS and the inclusion of new products on the list for financing.
Based on Law 12.401/2011 the request can be made uninterruptedly and without pre-defined periods for submission of new requests. Since then, almost 500 submissions were made corresponding to an average of 107 requests/year. Most were related to requests for the incorporation of medicines (261/485), whether of new drugs and presentations not presenton the SUS list until then or requests for the expansion of the use of medicines already available for clinical conditions/subpopulations that were not covered up to that point.
The emphasis of requests for incorporation of medicines certainly has a connection with the known access constraints to this important technology for the protection and recuperation of health. Medicines represent an expressive component in the increase of spending in health in the world and also in Brazil. Data from Health Satellite Accounts 2010-20131010. Instituto Brasileiro de Geografia e Estatística (IBGE). Conta-satélite de saúde: Brasil, 2010-2013. Rio de Janeiro: IBGE; 2015. indicates that expenditure involving medicines represents 1.6% of the Gross Domestic Product in Brazil and more than 20% of final spending on goods and health services in 2013.
Various medicines were the object of more than one request in the period, be they due to a different clinical recommendation, early discontinuance of the process or even prior decisions for non-incorporation. Amongst these, the drugs everolimus, fingolimod, cetuximab, golimumab, rituximab, tocilizumab, and trastuzumab stood out with six or more requests during the period.
Everolimus, which is used for rejection of transplants and oncology therapy, was requested eight times. One process was discontinued due to documental non-conformity and two due to a request from the applicant. The same pleintiff – SAS/MoH – subsequently requested again and had its request approved for the same indications requested before hand (immunosuppression in cardiac and hepatic transplants). Everolimus was also incorporated for pulmonary transplants (SCTIE’s request), but two of the incorporation requests made by the industry (for subependymal giant cell astrocytoma associated with tuberous sclerosis and advanced breast cancer) were refused.
Fingolimod is another case with repeated submissions (eight in total that came from manufacturers, organizations connected to clinical research and ordinary people). The first one was submitted in 2012 and all of them were for treating multiple sclerosis (MS). Two processes were discontinued due to non-conformity of the process. The medicine´s incorporation was requested separately for the 1st, 2nd and 3rd line treatment of the disease, with the first two being refused in July/2013. In May/2014, after public consultation (530 responses), fingolimod was incorporated for 3rd line treatment, restricted to “patients with remission-recurrent stages of MS with failure in the use of beta interferon and glatiramer as well as not being apt to use natalizumab and not having contraindications in using fingolimod”1111. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 113: Fingolimode para o tratamento da esclerose múltipla. 2014. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Incorporados/FingolimodeEM_FINAL.pdf
http://conitec.gov.br/images/Incorporado... . Such incorporation was conditioned to the updating of the STG with a clear definition of the use criterias, its availability in specialized centers having adequate infrastructure for monitoring cardiovascular risks, and a price’s reduction to below the values of the annual cost of treatment per patient of the products already available on SUS.
This and other cases gave rise to a suspicion that multiple requests – generally proposed at first for more broad populations, and that would continue to be restricted as the denials have been occuring, or with progressive reductions of proposed prices in order to minimize the budgetary impact – may be one of the strategies used to try the incorporation of medicines on SUS.
On the other hand, the number of processes that were discontinued prematurely due to lack of documentary compliance was significant (30.1% of the demands of medicines). There are no publicly available registers of the reasons for the refusals. A study on the submitted requests to Conitec up until July 2015 showed that the main justifications for the rejections were due to a lack of or inadequate economic evaluation and budgetary impact studies and of the evidence compilation on effectiveness and safety (totaling 51.3% of rejections)1212. Rabelo RB, Petramale CA, Silveira LC, Santos VCC, Gonçalves HC. A Comissão Nacional de Incorporação de Tecnologias no SUS: um balanço de seus primeiros anos de atuação. Revista Eletrônica Gestão & Saúde 2015; 6(Supl. 4):3225-3240.. It also refers to the progressive reduction of refusals (only 4% from January to July 2015) based on the availability of methodological guides for required studies and training made available for the associations of medicine manufacturers.
All of the rejections for non-conformity that were observed had external origins, despite the fact that some reports of “internal requests” were quite simple. In many cases those reports were the mere presentation of the technology’s caractheristics and of very simple estimates of budgetary impact. Data available, however, does not allow us to advance in the investigation of possible reasons for those apparent differences in treatment.
Of the 485 submissions made during the studied period, 320 had final deliberations until the end of data collection, 201 of which were related to medicines. Of these, 46.2% received favorable recommendations for incorporation which was a significantly reduced proportion compared to medical devices and procedures whose combined percentage of recommendations for incorporation was 79%.
The fluctuation observed in the number of recommendations submitted to the Conitec Plenary may reflect the variation in the number of submissions, whose annual distribution has not been shown by the available data. In the period of CITEC, there was no stipulated deadline for the evaluations and recommendations. All processes submitted without evaluations at the moment of regulation by the Law 12401/2011, had to be represented at Conitec1313. Associação da Indústria Farmacêutica de Pesquisa. Incorporação de Tecnologia no Sistema Único de Saúde (SUS). São Paulo: Interfarma; 2015., potentially contributing to the increases in the number of requests made in its first years of operation. The establishment of clearer rules for requests and decisions for incorporation may have also stimulated the pleas from the claimants.
Medicines that were entirely new on SUS responded to 44.1% of recommendations for incorporation while 51.5% represented approvals of new indications for medicines that were already available.
Amongst the last group, dolutegravir was incorporated in October 2015 for 3rd line treatment of HIV/AIDS. Conitec’s preliminary recommendation was against the incorporation, justified by the lack of evidence of long-term safety and in children under 12years of age, greater experience of real-life use with raltegravir and erroneous choice of the type of economic evaluation by the applicant. After public consultation (268 responses), with new information on drug safety and the presentation of new studies of cost-minimization and budgetary impact, carried out with updated prices for new medicine and alternative technology, the recommendation was unanimously changed for favourable to incorporation1414. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 182: Dolutegravir sódico para 3ª linha de tratamento da infecção pelo HIV. 2015. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Consultas/Relatorios/2015/Relatorio_Dolutegravir_2015.pdf
http://conitec.gov.br/images/Consultas/R... . In September 2016, there was a new decision for incorporation of dolutegravir for changes in the indication of its use for initial treatment (1st line) replacing efavirenz and being associated with “two in one” (tenofovir + lamivudine). The reasons suggested for the widening of use during a period smaler than a year included greater effectiveness (due to lower viral resistance rates through the time of treatment) and less adverse events as well as a significant reduction inprices (from US$ 5.10 to US$ 1.53) based on centralized purchaseson a large scale1515. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 227: Ampliação de uso dos medicamentos antirretrovirais dolutegravir (DTG) e darunavir (DRV), já disponibilizados pelo Ministério da Saúde para o tratamento da infecção pelo HIV. 2016. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2016/Relatorio-DolutegravirDarunavir-final-Republicacao.pdf.
http://conitec.gov.br/images/Relatorios/... .
This situation of changes amongst the initial and final recommendations after public consultation did not represent an exception as this was the case for 18.8% of the preliminary recommendations for non-incorporation of medicines. The presentation of new evidence on safety and effectiveness, new economic evaluations, budgetary impact, with correction of pointed mistakes and proposals for price reductions were the main reasons identified for the changes in recommendations.
Favorable recommendations for incorporation were significantly higher for internal requests from MoH. This aspect had already been pointed out by the association for the pharmaceutical industry (Interfarma)1313. Associação da Indústria Farmacêutica de Pesquisa. Incorporação de Tecnologia no Sistema Único de Saúde (SUS). São Paulo: Interfarma; 2015.. The extent to which the Committee responsible for evaluating the incorporation of new medicines belongs to the structure of the Ministry of Health, which finances what will be included, interferes in Conitec’s decisions needs to be the subject of in-depth studies. The international literature argues that the ‘ideal’ organizational structure for bodies with the responsibility for HTA at the national level would be hierarchically independent programs, even if financed by public funds1616. Banta D, Jonsson E. Commentary. Int J Technol Assess Health Care 2006; 22:280-282.,1717. Stephens JM, Handke B, Doshi JA. International survey of methods used in health technology assessment (HTA):does practice meet the principles proposed for good research? Comp Eff Res 2012; 2:29-44.. About a third of the medicine processes where decisions are made through the Plenary occurred in the simplified process and without public consultations. All of them came from MoH. The most part of the simplified processes approved during the period referred to requests for the widening of use.
Decree nº 7.646, that regulated the processes for the evaluation of requests for incorporation to be used in SUS, sets out that all the recommendations issued by the Plenary are to be submitted for public consultation55. Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez.. However, article 29 of this decree sets out the possibility of as implified administrative process in cases that are relevant to the public interest without details of applicable situations. According to Conitec reports, these processes concern “the widening of use or exclusion of technologies, new medicines presentations or incorporation with a tradition in use”, and with the requests “involving low-cost technologies and budgetary impact for SUS or being related to the elaboration or revision of STG”.
Six medicines without licensing by Anvisa were incorporated in the period: Biotin (biotidinase deficiency), injectable doxycycline and chloramphenicol oral suspension (spotted fever), hydroxyurea 100mg tablet (sickle cell disease), hydrocortisone cypionate (congenital adrenal hyperplasia), and hydroxocobalamin hydrochloride (cyanide poisoning). All came from ‘internal’ requests. Only two of the recommendations were submitted for public consultation. Main justifications for incorporation of these products were the clinical necessity that had not been filled, gravity of the clinical condition, absence of an alternative therapy on SUS or any additional clinical benefit about available options. These reports also mention that these drugs would be the object of processes for centralized purchases from abroad, by the Ministry of Health1818. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 57: Hidroxiureia para crianças com doença falciforme. 2013. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Incorporados/Hidroxiureia-final.pdf
http://conitec.gov.br/images/Incorporado... ,1919. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 96: Doxiciclina injetável e o Cloranfenicol suspensão para terapêutica da febre maculosa brasileira e outras riquetsioses. 2014. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Incorporados/Doxiciclina-e-Clorafenicol-FINAL.pdf
http://conitec.gov.br/images/Incorporado... .
Licensing is an essential regulatory tool in the evaluation of safety, efficacy, and quality of drugs, resulting in the protection of public health2020. Gava CM, Bermudez JA, Pepe VLE, Reis ALA. Novos medicamentos registrados no Brasil: podem ser considerados como avanço terapêutico? Cien Saude Colet 2010; 15(Supl. 3):3403-3413.. Even taking into account the justifications, medicine incorporation without registration goes against the clearly established criteria in the legal framework for the evaluation of submission for incorporation55. Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez.. The Brazilian legislation also impedes the aquisition, dispensation, and financing unlicensed medicines at all levels of the public health system2121. Luiza VL, Silva RM, Moraes EL, Mattos LV. Compras públicas de medicamentos no Brasil: uma análise a partir do marco regulatório. In: Hasenclever L, Oliveira MA, Paranhos J, Chaves G, organizadores. Desafios de Operação e Desenvolvimento do Complexo Industrial da Saúde. Rio de Janeiro: E-papers; 2016. p. 127-66.,2222. Brasil. Lei nº 6.360, de 23 de setembro de 1976. Dispõe sobre a Vigilância Sanitária a que ficam sujeitos os Medicamentos, as Drogas, os Insumos Farmacêuticos e Correlatos, Cosméticos, Saneantes e Outros Produtos, e dá outras Providências. Diário Oficial da União 1976; 24 set., except in cases set out in law2323. Brasil. Lei Federal nº 8.666, de 21 de junho de 1993. Regulamenta o art. 37, inciso XXI, da Constituição Federal, institui normas para licitações e contratos da Administração Pública e dá outras providências. Diário Oficial da União 1993; 22 jun.,2424. Brasil. Decreto nº 8.077, de 14 de agosto de 2013. Regulamenta as condições para o funcionamento de empresas sujeitas ao licenciamento sanitário, e o registro, controle e monitoramento, no âmbito da vigilância sanitária, dos produtos de que trata a Lei n° 6.360, de 23 de setembro de 1976, e dá outras providências. Diário Oficial da União 2013; 15 ago..
The high quantity of drugs for the treatment of chronic hepatitis C (boceprevir, telaprevir, sofosbuvir, daclatasvir and simeprevir) and for infections due to HIV (darunavir, dolutegravir sodic, maraviroc, and raltegravir) incorporated in the period were cases that stood out.
Telaprevir and boceprevir were incorporated in July 2012 for the treatment of infection for genotype 1 which is the most common form predominantly in the country2525. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 1: Inibidores de protease, telaprevir e boceprevir, para tratamento da hepatite crônica C. 2012. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2012/Boceprevir_Telaprevir_final.pdf
http://conitec.gov.br/images/Relatorios/... . They were excluded in May 2016 based on the introduction of new antivirals in July 2015. As justifications for the exclusion, a Conitec report showed that even if there were an increase in the chances of a sustained virologic response, prolonged treatments and a large number of daily tablets would still be needed. The inclusion of associated use of injectable interferon and ribavirinwould also be necessary. Another reason given was the significantly frequent adverse blood side effects, that would require other medicines for the treatment of neutropenia (filgrastim) and anemia (epoetin alfa)2626. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 222: Telaprevir, boceprevir, filgrastim e alfaepoetina para o tratamento da hepatite C. 2015. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2016/Relatorio_Telaprevir-Boceprevir-Filgrastim-Alfaepoetina_Hepatite_Recomendacao.pdf
http://conitec.gov.br/images/Relatorios/... . The new antivirals would have more favorable characteristics namely: greater safety and efficacy (≥ 80% of viral negativation for all genotypes, including in individuals with advanced cirrhosis and co-infections with HIV/HCV), treatment (oral use, two pills/day, less need for monitoring effectiveness and safety), favoring the administration and follow-up of the treatment for three months on average. Also, costs of the acquisition of treatments with the association of new drugs were mentioned as being inferior to those of the therapy with telaprevir and boceprevir2626. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 222: Telaprevir, boceprevir, filgrastim e alfaepoetina para o tratamento da hepatite C. 2015. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2016/Relatorio_Telaprevir-Boceprevir-Filgrastim-Alfaepoetina_Hepatite_Recomendacao.pdf
http://conitec.gov.br/images/Relatorios/... ,2727. Brasil. Ministério da Saúde (MS). Relatório Conitec nº 164: Sofosbuvir, daclatasvir e simeprevir para o tratamento da hepatite viral C crônica. 2015. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2015/Antivirais_HepatiteC_final.pdf
http://conitec.gov.br/images/Relatorios/... .
All five medicines were approved for incorporation not long after they were licensed in the country (in the last approved antivirals, between one and five months). Even with the comprehensive evidence-based search described in the recommendation report, they are very recent medicines with little time in market and limited experience of use. It is well known that wide-reaching information on effectiveness and full safety profile are usually unknown at the date of drug licensing because of small sample size, short duration and limited generalization of pre-approval trials, being necessary caution with new drugs2828. Ahmad SR. Adverse drug event monitoring at the Food and Drug Administration. J Gen Intern Med 2003; 18(1):57-60.. Efficacy of the new antivirals compared to the available options was based on indirect comparisons, given the absence of head-to-head studies. These drugs are very high-cost, and the estimated budgetary impact for SUS were between R$ 467 to R$ 666 million/year, considering the treatment of 15,000 infected individuals for different genotypes. Despite the importance of hepatitis C as a public health problem in Brazil, these several aspects are certainly a matter of great concern given the potential health impacts and scarce resources for health care in the country. Additional clinical benefits compared totechnologies that were already available, having low impact on the finances and budget and the clinical needs not being fulfilled, were all the main justifications used for incorporations of the new medicines on SUS in 2015. International studies that examined processes related to the reimbursement of drugs have been attracting attention for the important therapeutic value and additional benefits as relevant criteria for decision-making2929. Franken M, le Polain M, Cleemput I, Koopmanschap M. Similarities and differences between five European drug reimbursement systems. Int J Technol Assess Health Care 2012; 28(4):349-357.. On the other hand, estimates of the influence of small budgetary impacts was possibly amplified by the large number of recommendations for incorporation conducted as simplified processes, which are applied for requests that involve criteria including low costs and low budgetary impact for SUS’ forecast, among others.
Studies of the process and factors that influence the decisions of the responsible government bodies for the incorporation of technologies into the health system showed multiple and varied criteria in different countries3030. Schwarzer R, Siebert U. Methods, procedures, and contextual characteristics of health technology assessment and health policy decision making: comparison of health technology assessment agencies in Germany, United Kingdom, France, and Sweden. Int J Technol Assess Health Care 2009; 25(3):305-314.,3131. Spinner DS, Birt J, Walter JW, Bowman L, Mauskopf J, Drummond MF, Copley-Merriman C. Do different clinical evidence bases lead to discordant health-technology assessment decisions? An in-depth case series across three jurisdictions. Clinicoecon Outcomes Res 2013; 5:69-85.. These elements may differ even in relation to a single type of technology3232. Clement FM, Harris A, Li JJ, Yong K, Lee KM, Manns BJ. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. JAMA 2009; 302(13):1437-43.or in a single country and agency3333. Green W, Hutton J. Health technology assessments in England: an analysis of the NICE Medical Technologies Evaluation Programme. Eur J Health Econ 2014; 15(5):449-452.. Green and Hutton compared different existing programs in NICE related to HTA, and showed that clinical efficacy/effectiveness and cost-effectiveness are criteria uniformly used by the programs for decision-making. However, they also showed different degrees of requirements related to the quality of evidence, evaluation methods (presence of reference case, types of economic evaluations, cost-effectiveness threshold, etc.), infrastructure requirements, and the type of benefits for patients and for NHS compared to available technologies3333. Green W, Hutton J. Health technology assessments in England: an analysis of the NICE Medical Technologies Evaluation Programme. Eur J Health Econ 2014; 15(5):449-452..
A national study evaluating HTA experiences on 16 countries mapped 21 criteria related to the disease’s impact, the technology’s, economic issues, the quality of evidence and questions on equity, ethics, social and organizational issues. An examination of 12 reports drafted by Conitec showed that in the same way for the evaluated countries, information on effectiveness, safety, and cost-effectiveness were considered relevant in the process of the incorporation of technologies by the Committee3434. Pereira VC, Salomon FCR, Souza AB. Critérios para decisões sobre incorporação de tecnologias em saúde no Brasil e no mundo. Revista Eletrônica Gestão & Saúde 2015; 6(Supl. 4):3066-3093..
It is worth mentioning, however, that Conitec reports were selected by convenience samples. Also, many of the reports that were examined in the present study are limited and in turn are mere descriptions or simple estimates of budgetary impacts, reinforcing the importance of a necessary enhancement of the criteria and transparency of the processes used by Conitec.
Some limitations of the study deserve to be to be pointed out. There is no information on Conitec’s website about dates of submission. This absence makes it impossible to examine the compliance of some of the legal deadlines established by law such as a maximum of 270 days between submission and evaluation. Although all Conitec reports are publicly available, there is no access to the original documents submitted, only what is registered in the reports. The format, size, and content of reports are very heterogeneous and some are very simple. The legal requirement of an economic evaluation study comparing the proposed technology with those available on SUS was not present in all reports, especially in cases of simplified processes. Lastly, it did not always seem clear, specially in internal requests, which were based on information brought by the requesters and which were based on evidence obtained by Conitec’s technicians.
The creation of Conitec certainly constituted an improvement in the institutionalization of HTA in the Brazilian Health System. Its structure nowadays represents a central aspect of the complex process of making decisions that regulate the financing and access to pharmaceutical products on SUS. The results of this study give us signs of incremental rationality and the presence of clinical and economic evidence based on decisions related to medicines throughout the period. However, it also shows the necessity for continuous investment in scientific rigor as well as transparency and the independence of decisions. This need is even more relevant in the context of chronic under-investments in public health care system with a tendency for the recrudescence of the situation in moments of economic crisis. The evolution of the implementation of processes in SUS which guarantee universal access to all citizens is still a work in progress.
References
- 1Biehl J, Petryna A, Gertner A, Amon JJ, Picon PD. Judicialisation of the right to health in Brazil. Lancet 2009; 373(9682):2182-2184.
- 2Figueiredo TA, Pepe VLE, Osorio-de-Castro CGS. Um enfoque sanitário sobre a demanda judicial de medicamentos. Physis 2010; 20(1):101-118.
- 3Guimarães R. Technological incorporation in the Unified Health System (SUS): the problem and ensuing challenges. Cien Saude Colet 2014; 19(12):4899-4908.
- 4Brasil. Lei nº 12.401, de 28 de abril de 2011. Altera a Lei no 8.080, de 19.09.1990, para dispor sobre a assistência terapêutica e a incorporação de tecnologia em saúde no âmbito do Sistema Único de Saúde. Diário Oficial da União 2011; 24 abr.
- 5Brasil. Decreto nº 7.646, de 21 de dezembro de 2011. Dispõe sobre a Comissão Nacional de Incorporação de tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde, e dá outras providências. Diário Oficial da União 2011; 22 dez.
- 6International Network of Agencies for Health Technology Assessment (INAHTA). The International Network of Agencies for Health Technology Assessment. The Influence of Health Technology Assessment. A conceptual paper. 2014. [acessado 2016 set 22]. Disponível em: http://www.inahta.org/wp-content/uploads/2014/03/INAHTA_Conceptual-Paper_Influence-of-HTA1.pdf
» http://www.inahta.org/wp-content/uploads/2014/03/INAHTA_Conceptual-Paper_Influence-of-HTA1.pdf - 7Brasil. Ministério da Saúde (MS). Balanço Conitec: 2012-2014 Brasília: MS; 2014.
- 8WHO Collaborating Centre for Drug Statistics Methodology. Anatomical-Therapeutic-Chemical classification index Oslo: Norwegian Institute of Public Health; 2016. [acessado 2016 set 15] Disponível em: http://www.whocc.no/atc_ddd_index/
» http://www.whocc.no/atc_ddd_index/ - 9Brasil. Ministério da Saúde (MS). Departamento de Informática do SUS. 2016. [acessado 2016 set 15]. Disponível em: http://www.datasus.gov.br/cid10/V2008/cid10.htm
» http://www.datasus.gov.br/cid10/V2008/cid10.htm - 10Instituto Brasileiro de Geografia e Estatística (IBGE). Conta-satélite de saúde: Brasil, 2010-2013. Rio de Janeiro: IBGE; 2015.
- 11Brasil. Ministério da Saúde (MS). Relatório Conitec nº 113: Fingolimode para o tratamento da esclerose múltipla. 2014. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Incorporados/FingolimodeEM_FINAL.pdf
» http://conitec.gov.br/images/Incorporados/FingolimodeEM_FINAL.pdf - 12Rabelo RB, Petramale CA, Silveira LC, Santos VCC, Gonçalves HC. A Comissão Nacional de Incorporação de Tecnologias no SUS: um balanço de seus primeiros anos de atuação. Revista Eletrônica Gestão & Saúde 2015; 6(Supl. 4):3225-3240.
- 13Associação da Indústria Farmacêutica de Pesquisa. Incorporação de Tecnologia no Sistema Único de Saúde (SUS) São Paulo: Interfarma; 2015.
- 14Brasil. Ministério da Saúde (MS). Relatório Conitec nº 182: Dolutegravir sódico para 3ª linha de tratamento da infecção pelo HIV. 2015. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Consultas/Relatorios/2015/Relatorio_Dolutegravir_2015.pdf
» http://conitec.gov.br/images/Consultas/Relatorios/2015/Relatorio_Dolutegravir_2015.pdf - 15Brasil. Ministério da Saúde (MS). Relatório Conitec nº 227: Ampliação de uso dos medicamentos antirretrovirais dolutegravir (DTG) e darunavir (DRV), já disponibilizados pelo Ministério da Saúde para o tratamento da infecção pelo HIV. 2016. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2016/Relatorio-DolutegravirDarunavir-final-Republicacao.pdf
» http://conitec.gov.br/images/Relatorios/2016/Relatorio-DolutegravirDarunavir-final-Republicacao.pdf - 16Banta D, Jonsson E. Commentary. Int J Technol Assess Health Care 2006; 22:280-282.
- 17Stephens JM, Handke B, Doshi JA. International survey of methods used in health technology assessment (HTA):does practice meet the principles proposed for good research? Comp Eff Res 2012; 2:29-44.
- 18Brasil. Ministério da Saúde (MS). Relatório Conitec nº 57: Hidroxiureia para crianças com doença falciforme. 2013. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Incorporados/Hidroxiureia-final.pdf
» http://conitec.gov.br/images/Incorporados/Hidroxiureia-final.pdf - 19Brasil. Ministério da Saúde (MS). Relatório Conitec nº 96: Doxiciclina injetável e o Cloranfenicol suspensão para terapêutica da febre maculosa brasileira e outras riquetsioses. 2014. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Incorporados/Doxiciclina-e-Clorafenicol-FINAL.pdf
» http://conitec.gov.br/images/Incorporados/Doxiciclina-e-Clorafenicol-FINAL.pdf - 20Gava CM, Bermudez JA, Pepe VLE, Reis ALA. Novos medicamentos registrados no Brasil: podem ser considerados como avanço terapêutico? Cien Saude Colet 2010; 15(Supl. 3):3403-3413.
- 21Luiza VL, Silva RM, Moraes EL, Mattos LV. Compras públicas de medicamentos no Brasil: uma análise a partir do marco regulatório. In: Hasenclever L, Oliveira MA, Paranhos J, Chaves G, organizadores. Desafios de Operação e Desenvolvimento do Complexo Industrial da Saúde Rio de Janeiro: E-papers; 2016. p. 127-66.
- 22Brasil. Lei nº 6.360, de 23 de setembro de 1976. Dispõe sobre a Vigilância Sanitária a que ficam sujeitos os Medicamentos, as Drogas, os Insumos Farmacêuticos e Correlatos, Cosméticos, Saneantes e Outros Produtos, e dá outras Providências. Diário Oficial da União 1976; 24 set.
- 23Brasil. Lei Federal nº 8.666, de 21 de junho de 1993. Regulamenta o art. 37, inciso XXI, da Constituição Federal, institui normas para licitações e contratos da Administração Pública e dá outras providências. Diário Oficial da União 1993; 22 jun.
- 24Brasil. Decreto nº 8.077, de 14 de agosto de 2013. Regulamenta as condições para o funcionamento de empresas sujeitas ao licenciamento sanitário, e o registro, controle e monitoramento, no âmbito da vigilância sanitária, dos produtos de que trata a Lei n° 6.360, de 23 de setembro de 1976, e dá outras providências. Diário Oficial da União 2013; 15 ago.
- 25Brasil. Ministério da Saúde (MS). Relatório Conitec nº 1: Inibidores de protease, telaprevir e boceprevir, para tratamento da hepatite crônica C. 2012. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2012/Boceprevir_Telaprevir_final.pdf
» http://conitec.gov.br/images/Relatorios/2012/Boceprevir_Telaprevir_final.pdf - 26Brasil. Ministério da Saúde (MS). Relatório Conitec nº 222: Telaprevir, boceprevir, filgrastim e alfaepoetina para o tratamento da hepatite C. 2015. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2016/Relatorio_Telaprevir-Boceprevir-Filgrastim-Alfaepoetina_Hepatite_Recomendacao.pdf
» http://conitec.gov.br/images/Relatorios/2016/Relatorio_Telaprevir-Boceprevir-Filgrastim-Alfaepoetina_Hepatite_Recomendacao.pdf - 27Brasil. Ministério da Saúde (MS). Relatório Conitec nº 164: Sofosbuvir, daclatasvir e simeprevir para o tratamento da hepatite viral C crônica. 2015. [acessado 2016 out 15]. Disponível em: http://conitec.gov.br/images/Relatorios/2015/Antivirais_HepatiteC_final.pdf
» http://conitec.gov.br/images/Relatorios/2015/Antivirais_HepatiteC_final.pdf - 28Ahmad SR. Adverse drug event monitoring at the Food and Drug Administration. J Gen Intern Med 2003; 18(1):57-60.
- 29Franken M, le Polain M, Cleemput I, Koopmanschap M. Similarities and differences between five European drug reimbursement systems. Int J Technol Assess Health Care 2012; 28(4):349-357.
- 30Schwarzer R, Siebert U. Methods, procedures, and contextual characteristics of health technology assessment and health policy decision making: comparison of health technology assessment agencies in Germany, United Kingdom, France, and Sweden. Int J Technol Assess Health Care 2009; 25(3):305-314.
- 31Spinner DS, Birt J, Walter JW, Bowman L, Mauskopf J, Drummond MF, Copley-Merriman C. Do different clinical evidence bases lead to discordant health-technology assessment decisions? An in-depth case series across three jurisdictions. Clinicoecon Outcomes Res 2013; 5:69-85.
- 32Clement FM, Harris A, Li JJ, Yong K, Lee KM, Manns BJ. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. JAMA 2009; 302(13):1437-43.
- 33Green W, Hutton J. Health technology assessments in England: an analysis of the NICE Medical Technologies Evaluation Programme. Eur J Health Econ 2014; 15(5):449-452.
- 34Pereira VC, Salomon FCR, Souza AB. Critérios para decisões sobre incorporação de tecnologias em saúde no Brasil e no mundo. Revista Eletrônica Gestão & Saúde 2015; 6(Supl. 4):3066-3093.
Publication Dates
- Publication in this collection
Aug 2017
History
- Received
26 Oct 2016 - Reviewed
24 Mar 2017 - Accepted
26 Mar 2017