Introduction

As we could see in the previous articles of the economic evaluation series,¹1. Silva EN, Silva MT, Pereira MG. Estudos de avaliação econômica em saúde: definição e aplicabilidade aos sistemas e serviços de saúde. Epidemiol Serv Saude. 2016 jan-mar;25(1):205-7.

2. Silva EN, Silva MT, Pereira MG. Identificação, mensuração e valoração de custos em saúde. Epidemiol Serv Saude. 2016 abr-jun; 25(2):437-9.

3. Silva MT, Silva EN, Pereira MG. Desfechos em estudos de avaliação econômica em saúde. Epidemiol Serv Saude. 2016 jul-set;25(3):663-6.^-44. Silva EN, Silva MT, Pereira MG. Modelos analíticos em estudos de avaliação econômica. Epidemiol Serv Saude. 2016 out-dez; 25(4):855-8. there is a large set of information required for decision makers on health expenditures and outcomes, and the way they spread over time. In order to obtain this information, epidemiological, economic, mathematical and statistical methods are used, but they all have limitations which are inherent to any scientific method. During the development process of economic evaluation, some uncertainties may arise, which can substantially impact the main findings of the analysis.⁵5. Briggs AH, Fenwick E, Karnon J, Paltiel AD, Sculpher MJ, Weinstein M. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM modeling good research practices task force-6. Value in Health. 2012 ;15:835-42. When uncertainty is considered, researchers try to quantify the influence of data and adopted assumptions on the conclusion of the research. This article outlines three types of uncertainty in economic evaluation: methodological, structural and parameter.

Methodological uncertainty

The methodological uncertainty arises when there are different perceptions about how the ideal model for economic evaluation should be.⁶6. Bilcke J, Beutels P, Brisson M, Jit M. Accounting for methodological, structural, and parameter uncertainty in decision-analytic models: a practical guide. Med Decis Making. 2011 Jul-Aug;31(4):675-92. The study developer chooses a set of methodological decisions,¹1. Silva EN, Silva MT, Pereira MG. Estudos de avaliação econômica em saúde: definição e aplicabilidade aos sistemas e serviços de saúde. Epidemiol Serv Saude. 2016 jan-mar;25(1):205-7.

2. Silva EN, Silva MT, Pereira MG. Identificação, mensuração e valoração de custos em saúde. Epidemiol Serv Saude. 2016 abr-jun; 25(2):437-9.^-33. Silva MT, Silva EN, Pereira MG. Desfechos em estudos de avaliação econômica em saúde. Epidemiol Serv Saude. 2016 jul-set;25(3):663-6. such as analysis perspective, length of time horizon, discount rate, type of health outcome and method for valuing costs. These choices influence the result and, consequently, the decision-making. For instance, not all interventions impact on patient’s survival, such as those related to hearing loss and erectile dysfunction, although they substantially influence their quality of life. Thus, if the researcher chooses clinical outcomes (life years gained) instead of utility (quality of life), he or she would probably rule out the positive effects of interventions, affecting decision-making.

The way to deal with this kind of uncertainty is to adopt national good practice guidelines on how to conduct economic evaluation studies; or even, in the absence of such guidelines, find support in international recommendations. By following regulatory guidelines, the ability of the developer to influence the results of the analysis is reduced. In addition, comparability of results between different analyzes increases.

Structural uncertainty

It happens when the available evidence on the natural history of the disease and the impact of the strategies under investigation are non-existent, limited or contradictory.⁷7. Silva EN, Galvão TF, Pereira MG, Silva MT. Estudos de avaliação econômica de tecnologias em saúde: roteiro para análise crítica. Rev Panam Salud Publica. 2014 mar;35(3):219-27. In the absence of good quality evidence, the analytical model will be built⁴4. Silva EN, Silva MT, Pereira MG. Modelos analíticos em estudos de avaliação econômica. Epidemiol Serv Saude. 2016 out-dez; 25(4):855-8. inappropriately. In this case, among the most common errors, we can mention: (i) disregard of any relevant health state; (ii) choice of constant transition probabilities when they are variable; and (iii) use of fragile assumptions to extrapolate short-term results in long-term results.

This structural uncertainty can be overcome when considering at least two hypotheses in which one has a more favorable assumption to the intervention under investigation whilst the other portrays less favorable assumptions. There is also the possibility of introducing a random parameter to the analytical model, which would signal the probability of each assumption to be true.⁶6. Bilcke J, Beutels P, Brisson M, Jit M. Accounting for methodological, structural, and parameter uncertainty in decision-analytic models: a practical guide. Med Decis Making. 2011 Jul-Aug;31(4):675-92.

Parameter uncertainty

It is defined as the inability to employ true numerical values of the parameters used in the analytical model,⁸8. Walker D, Fox-Rushby J. Allowing for uncertainty in economic evaluations: qualitative sensitivity analysis. Health Policy Plan. 2001 Dec;16(4):435-43. such as transition probabilities, quality-adjusted life year and costs. Parameter uncertainty arises from several factors, mainly: (i) some parameters are unknown at the time of completion of the analysis, for example, the price of a medication not available in the health system; (ii) the main consequences of a given intervention are unknown at population level, since scientific evidence derived from samples or biased studies; and (iii) the reliability of the information available may be questionable.

Parameter uncertainty is examined by sensitivity analysis, which can be deterministic or probabilistic. In general, the difference between those two lies in the way of representing the variation of parameters. In the case of deterministic sensitivity analysis, a set of specific values that express the plausibility of variation of parameters is used. It works as a confidence interval in which there are lower and higher values than the measure of central tendency (mean or median, for example). In the case of probabilistic sensitivity analysis, we use random distributions instead of specific values, in the parameters variation.⁹9. Ministério da Saúde (BR). Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia. Diretrizes metodológicas: diretriz de avaliação econômica. 2. ed. Brasília: Ministério da Saúde; 2014.

There is also uncertainty regarding the variability among individuals, whilst some extracts or social groups respond differently to the intervention or have distinct perceptions and values. For example, differences between youngsters and the elderly. One way to overcome this uncertainty is the subgroup analysis.⁷7. Silva EN, Galvão TF, Pereira MG, Silva MT. Estudos de avaliação econômica de tecnologias em saúde: roteiro para análise crítica. Rev Panam Salud Publica. 2014 mar;35(3):219-27.

Concluding remarks

Uncertainty is inherent to any economic evaluation. Thus, it is necessary to analyze its impact on the outcome of the study. Each type of uncertainty has its peculiarities, some of them summarized in Figure 1. In the case of methodological uncertainty, it must follow national and/or international guidelines. As for structural uncertainty, we recommend the use of alternative analytical models, if there is limited information on the natural history of the disease or the result of interventions over time. And as for the parameter uncertainty, we suggest the sensitivity analysis.

References

Publication Dates