Abstract

In health technology assessment (HTA), decision criteria are considered relevant to support the complex deliberative process that requires simultaneous consideration of multiple factors. The aim was to identify and analyze the decision criteria that have been used by the National Health Technology Assessment Commission (CONITEC) when recommending the incorporation of technologies for the treatment of cancer. Descriptive study, based on reports from CONITEC, between 2012 and 2018, on oncology technologies. The data were collected in a specific extraction form and analyzed using descriptive statistics. 39 reports were analyzed, 15 of them did not present any explicit decision criteria. Medicines were the most frequently evaluated type of technology. The most frequent types of cancers were: breast cancer, head and neck cancer, colorectal cancer, non-Hodgkin’s lymphoma and lung cancer. The most frequently considered criteria were: financial impact and effectiveness. The study identified the decision criteria that have been most used in the area of ​​oncology, however, the lack of transparency in relation to the weight of these criteria makes it difficult to understand their influence on the result of the decisions taken.

Key words:
Health technology assessment; Health policy; Decision; Cancer

Introduction

According to estimates by the World Health Organization (WHO) in 2019, cancer is among the four leading causes of death in high- and middle-income countries, constituting the most important barrier to raising life expectancy worldwide¹1 World Health Organizatio (WHO). The top 10 causes of death [Internet]. WHO 2020; 9 Dec. [cited 2022 may 18]. Available from: https://www.who.int/news-room/fact-Sheets/detail/the-top-10-causes-of-death
https://www.who.int/news-room/fact-Sheet... ^,22 International Agency for Research on Cancer (IARC). Latest global cancer data: cancer burden rises to 19.3 million new cases and 10.0 million cancer deaths in 2020 [Internet]. IARC 2020; 15 Dec. [cited 2022 may 18]. Available from: https://www.iarc.who.int/faq/latest-global-cancer-data-2020-qa/
https://www.iarc.who.int/faq/latest-glob... . The prevalence of cancer in developing countries has reached levels close to those recorded in developed countries, and in Brazil, for each year of the 2020-2022 triennium, there have been an estimated 625,000 new cases of cancer³3 Brasil. Ministério da Saúde (MS). Incidência de câncer no Brasil. Rio de Janeiro: MS; 2020..

To confront this challenge, in 2005, the Ministry of Health proposed to institute the National Policy on Oncology Care through an ordinance, covering promotion, prevention, diagnosis, treatment, rehabilitation, and palliative care, to be implemented throughout the country⁴4 Brasil. Ministério da Saúde (MS). Portaria SAS n° 741, de 19 de dezembro de 2005. [acessado 2022 may 18]. Disponível em: https://bvsms.saude.gov.br/bvs/saudelegis/sas/2005/prt0741_19_12_2005.html
https://bvsms.saude.gov.br/bvs/saudelegi... . However, it is important to understand the budget impact of this policy, as well as to optimize the allocation of resources and to establish priorities for the approach to cancer within the scope of the Unified Health System (SUS).

In this regard, the incorporation of health technologies is a challenging activity that could contribute to the sustainability of public policies, but which requires careful consideration of multiple factors, such as: disease burden, clinical benefit, safety, level of innovation, quality of clinical evidence, cost-effectiveness, budgetary impact, and other relevant sources of evidence⁵5 Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and value: impact on decision Making - the EVIDEM framework and potential applications. BMC Health Serv Res 2008; 8:270.^,66 Angelis A, Kanavos P. Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in health technology assessment and beyond: the advance value framework. Soc Sci Med 2017; 188:137-156.. By observing these factors in the context of SUS, resources are being allocated in an efficient way based on the principles of health technology assessment (HTA). To this end, decision criteria in HTA processes can be defined as factors that are considered relevant to a decision-making process⁷7 Marsh K, Lanitis T, Neasham D, Orfanos P, Caro J. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics 2014; 32:345-365. and that, in principle, should support the complex deliberative process that requires simultaneous consideration of multiple factors⁸8 Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, Goetghebeur MM. From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decision making. Cost Eff Resour Alloc 2012; 10(1):9..

Decisions aimed at incorporating new technologies in the field of oncology have become increasingly complex and multifaceted due to rapid advances in research and the heterogeneity of available scientific evidence.⁹9 Adunlin G, Diaby V, Montero AJ, Xiao H. Multicriteria decision analysis in oncology. Health Expect 2015; 18(6):1812-1826. Some oncology treatment decisions are less complex, with clear evidence to support a better approach, while others may depend on considerations of conflicting factors and the absence of shared opinions about the role of evidence and value judgments in health⁵5 Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and value: impact on decision Making - the EVIDEM framework and potential applications. BMC Health Serv Res 2008; 8:270.^,99 Adunlin G, Diaby V, Montero AJ, Xiao H. Multicriteria decision analysis in oncology. Health Expect 2015; 18(6):1812-1826..

The technologies used in the field of oncology in Brazil must be evaluated through an HTA process by the National Health Technology Assessment Commission (CONITEC) to be incorporated for reimbursement by SUS¹⁰10 Elias FTS, Araújo DV. How Health Economic Evaluation (HEE) contributes to decision-making in public health care: the case of Brazil. Z Evid Fortbild Qual Gesundhwes 2014; 108(7):405-412.. Based on the National Health Technology Management Policy (PNGTS)¹¹11 Brasil. Ministério da Saúde (MS). Secretaria de Ciência Tecnologia e Insumos Estratégico. Política Nacional de Gestão de Tecnologias em Saúde. Brasília: MS; 2010., CONITEC considers eight relevant criteria for the incorporation of technologies into SUS: safety, efficacy, effectiveness, efficiency, and budgetary, ethical, social, and environmental impacts of the technology in question. The criteria used by CONITEC when preparing recommendation reports on the incorporation of technologies for SUS are part of a complex decision-making process that includes: a) a comprehensive, systematic review of the scientific literature; b) a critical assessment of the best available evidence; c) consideration of the population’s health needs and health policy priorities; d) market characterization and choice of comparable therapeutic alternatives; e) an evaluation of the logistics for the implementation of the new technology, according to SUS care protocols; and f) an evaluation of the technology’s effectiveness, safety, cost-effectiveness, and budgetary impact, according to studies presented by the applicant for the incorporation of the technology in question, in addition to other criteria based on the PNGTS (from the perspective of SUS)¹⁰10 Elias FTS, Araújo DV. How Health Economic Evaluation (HEE) contributes to decision-making in public health care: the case of Brazil. Z Evid Fortbild Qual Gesundhwes 2014; 108(7):405-412.^,1111 Brasil. Ministério da Saúde (MS). Secretaria de Ciência Tecnologia e Insumos Estratégico. Política Nacional de Gestão de Tecnologias em Saúde. Brasília: MS; 2010..

Previous studies carried out in our country have shown that the nature and type of evidence used in CONITEC recommendation reports are unclear¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40. and that economic and clinical factors are those most frequently used to guide CONITEC recommendations¹⁰10 Elias FTS, Araújo DV. How Health Economic Evaluation (HEE) contributes to decision-making in public health care: the case of Brazil. Z Evid Fortbild Qual Gesundhwes 2014; 108(7):405-412.. However, it is unknown how these factors may have influenced the recommendations for incorporating technologies related to the approach to cancer in Brazil.

Thus, this study aims to identify and analyze the decision-making criteria that have been used by CONITEC in recommending the incorporation of technologies for the treatment of cancer within the scope of SUS, based on an analysis of CONITEC’s recommendation reports from 2012 to 2018.

Methods

This is a descriptive study, based on official open access reports from CONITEC that deal with oncology technologies. CONITEC recommendation reports were chosen as empirical material for analysis as, in addition to being the final product of the entire recommendation flow of a technology for incorporation into SUS, they illustrate the use of scientific evidence in this process and the criteria prioritized by decision-makers.

Data were read and extracted from all recommendation reports related to the approach to cancer, publicly available on the CONITEC website, published between July 1, 2012, and July 1, 2018. This period was selected because it included a relatively broad period of six years, covering two administrations of the federal government

A standardized form was prepared prior to data extraction to collect variables of interest and a pilot phase was conducted aimed at extracting data from three eligible reports to standardize the terms to be used, train the reviewers, and validate the form. Two independent reviewers then extracted data from the reports using the standardized form. Disagreements between reviewers were resolved through discussion and consensus.

For the purpose of this study, information was initially extracted from the reports regarding the type of technology, description of the characteristics and current use of the technology, year of the report, and type of diagnosis, based on previous scientific publications¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40.^,1313 Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. London: Oxford; 2015.. The diseases targeted by the technologies evaluated in the reports were classified according to the sections of the tenth revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10). Only oncology-related reports were included.

Secondly, the reports were evaluated in order to extract information regarding the inclusion of studies and considerations that inform decision-making criteria, based on the proposal suggested by Merlin et al.:¹⁴14 Merlin T, Tamblyn D, Ellery B, INAHTA Quality Assurance Group. What's in a name? Developing definitions for common health technology assessment product types of the International Network of Agencies for Health Technology Assessment (INAHTA). Int J Technol Assess Health Care 2014; 30(4):430-437. 1) safety assessment, 2) evaluation of effectiveness, 3) analysis of cost-effectiveness, 4) acquisition costs and budgetary impact, and 5) organizational considerations.

The extracted data were then evaluated for the presence of information or studies described in the body of the report with regard to each of the eight criteria for incorporating technologies contained in the PNGTS: safety, effectiveness, and efficiency, as well as budgetary, ethical, social, and environmental impacts of the technology in question¹¹11 Brasil. Ministério da Saúde (MS). Secretaria de Ciência Tecnologia e Insumos Estratégico. Política Nacional de Gestão de Tecnologias em Saúde. Brasília: MS; 2010.. Afterward, the weight of these criteria was evaluated in terms of the presence of comments or justifications related to each of them in the final recommendations of the reports (Chart 1). Thus, by examining the content of the final recommendations, the reviewers identified references to studies included by the applicant in the report, along with information from these studies or mention of the decision-making criteria (or even definitions of these criteria), which could characterize the relevance of these contents for the recommendations made, according to the corresponding decision-making criteria. Likewise, the reports with changes in their final recommendations after public consultation by CONITEC were identified, and a textual analysis of the final recommendations sought to identify information regarding the decision-making criteria that could justify changing the technology recommendation.

The information from each report was collected using an extraction form developed for this purpose and analyzed in a descriptive way. Frequencies and percentages were used for data analysis, using Microsoft Excel spreadsheets.

Results

Thirty-nine CONITEC recommendation reports were included regarding the incorporation of technologies for approaching cancer in SUS. The number of reports corresponded to 13.8% of the total reports by CONITEC in 2012, 16.7% in 2013, 21.9% in 2014, 11.8% in 2015, 8.9% in 2016, 3.1% in 2017, and 15.6% in 2018. Of these, 22 (56.4%) resulted in positive recommendations for incorporation and 17 (43.6%) resulted in negative recommendations. Table 1 shows the types of technologies evaluated between 2012 and 2018. Medicines were the most frequently evaluated type of technology in the period, with the highest number of reports being produced in 2013 and 2014 (9 reports in each year).

Table 2 presents the types of cancer that were most often considered in the CONITEC recommendation reports, highlighting the most frequent types: breast cancer (8 reports), head and neck cancer (4 reports), colorectal cancer (4 reports), non-Hodgkin lymphoma (4 reports), and lung cancer (3 reports).

Among the thirty-nine recommendation reports that were analyzed, fifteen (36.46%) did not present any information that reported an explicit decision-making criterion or based on the classifications considered in the study. The other twenty-four reports presented criteria informed by evidence as described in the classification by Merlin et al.,¹⁴14 Merlin T, Tamblyn D, Ellery B, INAHTA Quality Assurance Group. What's in a name? Developing definitions for common health technology assessment product types of the International Network of Agencies for Health Technology Assessment (INAHTA). Int J Technol Assess Health Care 2014; 30(4):430-437. and in several cases, the report presented more than one criterion. Table 3 provides a description of the decision-making criteria presented in the reports, based on guidelines considered by CONITEC and in the classification of Merlin et al.¹⁴14 Merlin T, Tamblyn D, Ellery B, INAHTA Quality Assurance Group. What's in a name? Developing definitions for common health technology assessment product types of the International Network of Agencies for Health Technology Assessment (INAHTA). Int J Technol Assess Health Care 2014; 30(4):430-437. Of these, 43.59% presented information regarding the criterion of effectiveness/efficacy and 38.46% related to the criteria of acquisition costs and budgetary impact, which were those most frequently informed with evidence submitted by the applicant for the incorporation.

Table 4 presents the frequency with which the PNGTS decision-making criteria are reported and considered in the body and in the final recommendations of the CONITEC reports, the most frequent in the body of the report being budget impact (in 87.18% of the reports) and effectiveness/efficacy (in 74.36% of the reports). Regarding the final recommendation, the most frequent criteria were effectiveness/efficacy (in 43.59% of the reports) and budgetary impact (in 33.33% of the reports).

Table 5 presents the decision-making criteria considered in the changes made in the CONITEC recommendation after public consultation. There were three technology cases that resulted in a change in CONITEC’s position after public consultation, referring to the following drugs: gefitinib, erlotinib, and pertuzumab. Additional evidence was presented during the public consultation in the field of technology effectiveness/efficacy, budgetary impact, information on efficiency (based on cost-effectiveness analysis), which motivated the change in recommendation. The effectiveness/efficacy criterion was the only one that was considered in all changes made in the final recommendation after public consultation.

Discussion

This study aimed to identify and analyze decision-making criteria that have been used by CONITEC in the processes of incorporating technologies in the approach to neoplasms. The analysis of the recommendations related to cancer made by the commission from 2012 to 2018 showed that acquisition costs and budgetary impact were the criterion that most frequently appeared in the body of the reports whereas the effectiveness/efficacy criterion appeared more frequently in the final recommendation section. However, when there is a change in the commission’s initial recommendation based on public consultations and the PNGTS, the effectiveness/efficacy and budgetary impact criteria are those most frequently considered. Other criteria appearing in the PNGTS do not appear in the reports, and it was not possible to understand how these criteria are added or reconsidered after public consultation.

Regardless of the relevance of the decision-making criteria, it was initially possible to observe some distinction between the types of cancer most often evaluated in the reports and their estimated incidence. In Brazil, the estimate for each year of the 2020-2022 triennium indicates there will be 625,000 new cases (450,000, excluding non-melanoma skin cancer cases), with non-melanoma skin cancer being the most frequent (177,000), followed by breast and prostate (66,000 each), colon and rectum (41,000), lung (30,000), and stomach (21,000) cancers³3 Brasil. Ministério da Saúde (MS). Incidência de câncer no Brasil. Rio de Janeiro: MS; 2020.. By contrast, Table 2 shows three of these types of neoplasms being among the most frequently evaluated. In descending order, they are breast, head and neck, colorectal, non-Hodgkin’s lymphoma, and lung.

In a previous study, Elias et al. showed that CONITEC typically prioritizes clinical criteria (for example, efficacy) and that recommendations favorable to the incorporation of the required technologies are often guided by clinical or therapeutic impact criteria (gain in efficacy is mentioned in 71% of the evaluated reports) as compared to other criteria.¹⁰10 Elias FTS, Araújo DV. How Health Economic Evaluation (HEE) contributes to decision-making in public health care: the case of Brazil. Z Evid Fortbild Qual Gesundhwes 2014; 108(7):405-412. Regarding economic considerations, the budgetary impact criterion (mentioned in 38% of the reports) appears most frequently in the recommendations, although in a much lower percentage than in the therapeutic impact criterion¹⁰10 Elias FTS, Araújo DV. How Health Economic Evaluation (HEE) contributes to decision-making in public health care: the case of Brazil. Z Evid Fortbild Qual Gesundhwes 2014; 108(7):405-412..

In another study, Yuba et al.¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40. also analyzed CONITEC’s recommendation reports and concluded that the characteristics of the evidence that were considered mandatory for the decisions were very different, indicating problems in the decision-making process¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40.. One of the most important findings of that study was CONITEC’s lack of adherence to its own internal regulations in terms of evidence to be considered and, therefore, of the decision-making criteria that guided the final recommendations¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40..

Given the progressive increase in healthcare costs, explicit approaches to the prioritization and allocation of resources have become critical and illustrate how decision-making criteria is a key element in ensuring transparency and, therefore, legitimacy of the decisions being made⁸8 Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, Goetghebeur MM. From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decision making. Cost Eff Resour Alloc 2012; 10(1):9.. In accordance with the findings of Elias et al., this study reveals that the effectiveness/efficacy criterion is the one most often considered in CONITEC’s final recommendations¹⁰10 Elias FTS, Araújo DV. How Health Economic Evaluation (HEE) contributes to decision-making in public health care: the case of Brazil. Z Evid Fortbild Qual Gesundhwes 2014; 108(7):405-412., although information on the budgetary impact criterion is more frequent found in the body of the report, and other criteria are not considered or reported, despite being mentioned in the PNGTS¹¹11 Brasil. Ministério da Saúde (MS). Secretaria de Ciência Tecnologia e Insumos Estratégico. Política Nacional de Gestão de Tecnologias em Saúde. Brasília: MS; 2010.. Regarding the budgetary impact criterion, of the 101 reports evaluated by Yuba et al., only 19.8% included a complete economic assessment, despite the commission’s internal regulations establishing them as mandatory for all demands for incorporating technologies¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40..

Considering the clear difference in the scientific evidence, depending on the type of demand for technology incorporation (internal or external to SUS)¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40., this study’s findings suggest a need for greater transparency in CONITEC’s internal decision-making processes, in addition to showing an overestimation of certain criteria in the final recommendation (clinical, in particular) in relation to other PNGTS criteria. In this light, it is possible to conclude that clinical criteria (such as efficacy and effectiveness) actually end up exerting a greater influence on the commission’s final recommendations, although economic criteria (such as budgetary impact and cost-effectiveness) are frequently cited throughout the reports.

In a literature review of studies on health-related decision-making, Guindo et al. identified a wide variety that have often been considered⁸8 Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, Goetghebeur MM. From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decision making. Cost Eff Resour Alloc 2012; 10(1):9.. In the forty articles, the five most frequently mentioned criteria were in descending order: equity/fairness, efficacy/effectiveness, stakeholder interests and pressures, cost-effectiveness, and strength of evidence. In another study, the results from a literature review showed that 25 of the 40 studies included in that search dealt with “health outcomes” as the main group of criteria in defining priorities for health technology assessment, and that the higher frequency of “health effects/benefits” criteria (in eight of the studies) is in line with the findings of this study¹⁵15 Mobinizadeh M, Raeissi P, Nasiripour AA, Olyaeemanesh A, Tabibi SJ. The health systems' priority setting criteria for selecting health technologies: a systematic review of the current evidence. Med J Islam Repub Iran 2016; 30:329.. Thus, the relevance of clinical criterion (effectiveness/efficacy) seems to be corroborated by studies conducted in other contexts, according to the literature reviews. However, the equity/justice criterion (the one most frequently represented in studies of different nationalities, according to the study by Guindo et al.)⁸8 Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, Goetghebeur MM. From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decision making. Cost Eff Resour Alloc 2012; 10(1):9., which may even be associated with the ethical impacts of health decisions, is not explicitly presented in the revised CONITEC reports. Nonetheless, the non-inclusion of this criterion in the reports may be due to a difficulty in considering equity in a pragmatic way; which points to the need for systematic approaches that facilitate the operationalization of this criterion in the decision-making process.

To assist in the process of including criteria that are difficult to quantify, there is a wide set of techniques that can help make the decision-making process more systematic and transparent, often called multi-criteria decision analysis (MCDA)¹⁶16 Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health 2012; 15(8):1172-1181.

17 Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, Longrenn T, Mussen F, Peacock S, Watkins J, Ijzerman M. Multiple criteria decision analysis for health care decision making - an introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health 2016; 19(1):1-13.^-1818 Youngkong S, Baltussen R, Tantivess S, Mohara A, Teerawattananon Y. Multicriteria decision analysis for including health interventions in the universal health coverage benefit package in Thailand. Value Health 2012; 15(6):961-970.. The use of structured methodological approaches, such as MCDA, could be an option to facilitate decision-making processes and assist in preparing recommendations within the scope of CONITEC. The MCDA is comprised of a set of methods to support decision-making that explicitly takes into account multiple criteria, allowing different stakeholders to participate¹⁶16 Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health 2012; 15(8):1172-1181.. The basic aspects of the method involve: 1) the selection of alternatives to be evaluated, 2) the choice of criteria through which the alternatives will be compared, 3) the measurement of the value attributed to the performance of the technologies in each of the criteria, and 4) the measurement of the weight of the criteria in order to reflect the relative importance of each one¹⁶16 Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health 2012; 15(8):1172-1181.^,1717 Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, Longrenn T, Mussen F, Peacock S, Watkins J, Ijzerman M. Multiple criteria decision analysis for health care decision making - an introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health 2016; 19(1):1-13..

The use of an MCDA methodology could allow the assessment of stakeholder preferences (in relation to decision-making criteria), help ensure that they are more systematically taken into account in the decisions, and estimate the overall value of each one, based on monitoring the stages of a discussion leading up to collective decisions⁸8 Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, Goetghebeur MM. From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decision making. Cost Eff Resour Alloc 2012; 10(1):9.. In HTA processes, one of MCDA’s greatest contributions is to increase transparency in the incorporation of technology in a more participatory and legitimate way.

When it comes to oncology, a previous systematic review of current practices and applications of MCDA showed cancer as being the most researched type of disease¹⁴14 Merlin T, Tamblyn D, Ellery B, INAHTA Quality Assurance Group. What's in a name? Developing definitions for common health technology assessment product types of the International Network of Agencies for Health Technology Assessment (INAHTA). Int J Technol Assess Health Care 2014; 30(4):430-437.. In another review dedicated to MCDA applications in oncology, although the eight studies reviewed were focused on clinical decision-making (six for cancer screening), the authors recognized the importance of the approach to assessing stakeholder preferences and developing clinical guidelines⁹9 Adunlin G, Diaby V, Montero AJ, Xiao H. Multicriteria decision analysis in oncology. Health Expect 2015; 18(6):1812-1826.. Furthermore, other studies reviewing the scientific literature have highlighted the importance of MCDA methods in prioritizing interventions for coverage or reimbursement, supporting decisions in health technology assessment¹⁹19 Adunlin G, Diaby V, Xiao H. Application of multicriteria decision analysis in health care: a systematic review and bibliometric analysis. Health Expectations 2015; 18(6):1894-1905.

20 Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc 2006; 4:14.

21 Campolina AG, De Soárez PC, Amaral FV, Abe JM. Análise de decisão multicritério para alocação de recursos e avaliação de tecnologias em saúde: tão longe e tão perto? Cad Saude Publica 2017; 33(10):e00045517.

22 Blinman P, King M, Norman R, Viney R, Stockler MR. Preferences for cancer treatments: An overview of methods and applications in oncology. Ann Oncol 2012; 23(5):1104-1110.

23 Tanios N, Wagner M, Tony M, Baltussen R, van Til J, Rindress D, Kind P, Goetghebeur MM, International Task Force on Decision Criteria. Which criteria are considered in healthcare decisions? Insights from an international survey of policy and clinical decision makers. Int J Technol Assess Health Care 2013; 29(4):456-465.^-2424 Golan O, Hansen P, Kaplan G, Tal O. Health technology prioritization: which criteria for prioritizing new technologies and what are their relative weights? Health Policy (New York). 2011; 102(2-3):126-135..

Therefore, when preparing an MCDA, it is important to consider that the selection and structuring of criteria must follow good practices that justify their selection and adequate reporting in published studies, as recommended by the Professional Society for Health Economics and Outcomes Research (ISPOR)¹⁷17 Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, Longrenn T, Mussen F, Peacock S, Watkins J, Ijzerman M. Multiple criteria decision analysis for health care decision making - an introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health 2016; 19(1):1-13.. The ISPOR recommendations highlight the importance of investigating reports that guide decision-making, such as those prepared by HTA agencies, e.g., the National Institute for Health and Care Excellence (NICE) in the United Kingdom, 25 and, in the case of Brazil, CONITEC’s role in preparing HTA reports.

As with MCDA, other prioritization and decision support processes could be considered, such as portfolio analyses, which address the question of how to combine interventions within or across programs to maximize a given objective, and approaches based on program budgeting and marginal analysis that aim to maximize defined objectives by choosing from a range of interventions, keeping budget constraints in mind²⁶26 Isaranuwatchai W, Bayoumi AM, Renahy E, Chef R, O'Campo P. Using decision methods to examine the potential impact of intersectoral action programs. BMC Res Notes 2018; 11:506.. Even so, these approaches use various criteria, such as those identified and analyzed in this study, in an attempt to reflect the reality that decision-makers typically face when having to deal with a set of objectives while evaluating the value of a new technology²⁷27 Jansen MP, Helderman J-K, Boer B, Baltussen R. Fair processes for priority setting: putting theory into practice; comment on "Expanded HTA: enhancing fairness and legitimacy". Int J Health Policy Manag 2017; 6(1):43-47..

Although none of these approaches is used to support decisions on the incorporation of health technologies, identifying and evaluating the frequency of decision-making criteria in recommendation reports from an HTA commission, such as CONITEC, already allows a structure or framework of value to be used as a way to assist the commission’s deliberative work. In this regard, the implementation of various frameworks of value has been proposed by international bodies, such as the American Society of Clinical Oncology (ASCO), the American Heart Association (AHA)/American College of Cardiology (ACC), the Institute for Clinical and Economic Review (ICER), the Memorial Sloan Kettering Cancer Center (MSKCC), and the National Comprehensive Cancer Center (NCCC)²⁸28 Diaby V, Ali AA, Montero AJ. Value assessment frameworks in the United States: a call for patient engagement. Pharmacoeconomics Open 2019; 3(1):1-3.. The main objective of these frameworks is to inform the decisions of different stakeholders, based on robust clinical evidence and shared value judgments²⁷27 Jansen MP, Helderman J-K, Boer B, Baltussen R. Fair processes for priority setting: putting theory into practice; comment on "Expanded HTA: enhancing fairness and legitimacy". Int J Health Policy Manag 2017; 6(1):43-47..

Therefore, the main contributions of this study are, first, to present the decision-making criteria that have been considered most frequently by CONITEC when evaluating technologies for the approach to cancer and, second, to show that, as the reports do not clearly present the weights of these criteria and the value assigned to technology performance; it is difficult to understand the influence these criteria have on the outcome of the decisions.

However, these results need to be viewed with care insofar as qualitative analyses were not conducted with the participants of the aforementioned decision-making processes for a better understanding of how the decision-making criteria were considered. Another limitation to be considered is that it is not possible to ascertain which other criteria were not considered or why they were not reported from the analysis of reports. A third limitation refers to the fact that the evaluation was based on reports that may not include the criteria that really weighed on the decision, meaning the analysis is limited to recorded information. Another thing to understand is that not all public consultations were considered, which could provide additional information about the deliberative processes. Finally, as reports from mid-2018 onwards were not evaluated, it is not possible to verify changes in CONITEC’s decision-making criteria in recent years.

Ultimately, it is important to remember that HTA is a socio-technical process by nature and that its recommendations need to reflect its social aspects (who participates, how, and in what way) and technical aspects (which methods are used) in a systematic and transparent way. Despite the challenges imposed by this process, the advances made in CONITEC’s performance in our country is undeniable, considering there has been a significant rise in the production of literature in the HTA area in Brazil since 2008, as well as in the development of training courses in systematic reviews, technical-scientific reports, and promulgation guidelines.¹²12 Yuba TY, Novaes HMD, Soárez PC. Challenges to decision-making processes in the National HTA Agency in Brazil: operational procedures, evidence use and recommendations. Health Res Policy Syst 2018; 16(1):40. In terms of the challenges posed by innovations in the field of oncology, demystifying the decision-making criteria and how they are considered is essential to ensuring the legitimacy of decisions to incorporate new health technologies.

Conclusion

This study identified that the decision criteria that have been more informed in the CONITEC recommendation reports for the incorporation of technologies for the treatment of cancer, from 2012 to 2018, were: first, budgetary impacts, followed by effectiveness /efficiency. Yet, in the final recommendations, effectiveness/efficacy was the most frequently considered.

References

Publication Dates

History