Abstract

Objective

To evaluate the quality of economic data provided in applications to the World Health Organization (WHO) Model List of Essential Medicines and to evaluate the role of these data in decision-making by the expert committee that considers the applications.

Methods

We analysed applications submitted to the WHO Expert Committee on the Selection and Use of Essential Medicines between 2002 and 2013. The completeness of data on the price and cost-effectiveness of medicines was extracted from application documents and coded using a four-point scale. We recorded whether or not the expert committee discussed economic information and the outcomes of each application. Associations between the completeness of economic data and application outcomes were assessed using χ2 tests.

Findings

The expert committee received 134 applications. Only eight applications (6%) included complete price data and economic evaluation data. Many applicants omitted or misinterpreted the economic evaluation section of the application form. Despite the lack of economic data, all applications were reviewed by the committee. There was no significant association between the completeness of economic information and application outcomes. The expert committee tried to address information gaps in applications by further review and analysis of data related to the application.

Conclusion

The World Health Organization should revise the instructions to applicants on economic data requirements; develop new mechanisms to assist applicants in completing the application process; and define methods for the use of economic data in decision-making.

Résumé

Objectif

Évaluer la qualité des données économiques fournies dans les candidatures à la Liste Modèle des Médicaments Essentiels de l'Organisation Mondiale de la Santé (OMS) et évaluer le rôle de ces données dans la prise de décision du comité d'experts chargé d'examiner les candidatures.

Méthodes

Nous avons analysé les candidatures soumises au comité d'experts de l'OMS concernant la sélection et l'utilisation des médicaments essentiels entre 2002 et 2013. Toutes les données relatives aux prix et au rapport coût-efficacité des médicaments ont été tirées des dossiers de candidature et leur exhaustivité a été codée à l'aide d'une échelle à quatre points. Nous avons déterminé si le comité d'experts examinait les informations économiques et l'issue de chaque candidature. La relation entre le caractère exhaustif des données économiques et l'issue des candidatures a été évaluée par des tests du χ2.

Résultats

Le comité d'experts a reçu 134 candidatures. Seules huit d'entre elles (6%) incluaient des données complètes sur les prix et des données d'évaluation économique. De nombreux candidats ont négligé ou mal compris la section du formulaire de candidature relative à l'évaluation économique. Malgré le manque de données économiques, toutes les candidatures ont été examinées par le comité. Aucune relation significative entre l'exhaustivité des informations économiques et l'issue des candidatures n'a été établie. Le comité d'experts a tenté de remédier à l'insuffisance des informations fournies dans les candidatures en procédant à un examen et à une analyse plus poussés des données relatives à la candidature.

Conclusion

L'Organisation Mondiale de la Santé doit revoir les indications données aux candidats en ce qui concerne les exigences relatives aux données économiques, élaborer de nouveaux mécanismes pour aider les candidats à aller au bout du processus de candidature et définir des méthodes d'utilisation des données économiques dans la prise de décision.

Resumen

Objetivo

Evaluar la calidad de los datos económicos proporcionados en las solicitudes para la Lista Modelo de Medicamentos Esenciales de la Organización Mundial de la Salud (OMS) y valorar el papel que desempeña dicha información en el proceso de toma de decisiones del Comité de Expertos que tiene en cuenta las solicitudes.

Métodos

Se analizaron las solicitudes entregadas al Comité de Expertos de la OMS en relación con la selección y el uso de medicamentos esenciales entre 2002 y 2013. La exactitud de los datos sobre el precio y la costoeficacia de los medicamentos se extrajo de los documentos de solicitud y se calificó mediante una escala de cuatro puntos. Se registró si en el Comité de Expertos se discutió o no la información económica y los resultados de cada solicitud. Las asociaciones entre la exactitud de los datos económicos y los resultados de las solicitudes se evaluaron usando pruebas χ2.

Resultados

El comité de expertos recibió 134 solicitudes. Solamente ocho solicitudes (6%) incluían datos completos relativos al precio y datos relativos a la evaluación económica. Muchos solicitantes omitieron o malinterpretaron la sección de evaluación económica del formulario de solicitud. A pesar de la falta de datos económicos, el Comité revisó todas las solicitudes. No hubo ningún vínculo significativo entre la exactitud de la información económica y los resultados de la solicitud. El Comité de Expertos trató de remediar la falta de información en las solicitudes mediante una revisión más exhaustiva y un análisis de los datos relacionados con dicha solicitud.

Conclusión

La Organización Mundial de la Salud deberá revisar las instrucciones para los solicitantes que figuran en los requisitos de los datos económicos, desarrollar nuevas formas de asistir a los solicitantes a la hora de completar el proceso de solicitud y definir métodos para el uso de datos económicos en el proceso de toma de decisiones.

ملخص

الغرض تقييم مستوى جودة البيانات الاقتصادية المقدمة في طلبات الإدراج ضمن القائمة النموذجية للأدوية الأساسية بمنظمة الصحة العالمية (WHO)، وتقييم الدور الذي تلعبه هذه البيانات في اتخاذ القرارات من جانب لجنة الخبراء التي تنظر في الطلبات.

الطريقة لقد قمنا بتحليل الطلبات المقدمة للجنة الخبراء بمنظمة الصحة العالمية بشأن اختيار الأدوية الأساسية واستخدامها في الفترة ما بين عامي 2002 و2013. وتم استخلاص درجة اكتمال البيانات المتعلقة بسعر الأدوية ومعقولية تكلفتها من خلال المستندات التي تضمنتها الطلبات، كما تم التعبير عن ذلك بالترميز، مع الاستعانة بمعيار من أربع نقاط. وسجلنا ما إذا كانت لجنة الخبراء قد قامت بمناقشة المعلومات الاقتصادية والنتائج فيما يتعلق بكل طلب، أم لا. وتم تقييم حالات الارتباط بين درجة اكتمال البيانات الاقتصادية من جهة، ونتائج الطلبات من جهة أخرى، وذلك باستخدام اختبارات خي مربع χ2.

النتائج تسلمت لجنة الخبراء 134 طلباً. وتضمنت ثمانية طلبات فقط (6%) بيانات مكتملة عن السعر وبيانات عن التقييم الاقتصادي. وقد حذف الكثير من مقدمي الطلبات قسم التقييم الاقتصادي بنموذج الطلب أو أساءوا تفسيره. وبرغم نقص البيانات الاقتصادية، قامت اللجنة بمراجعة جميع الطلبات. ولم يوجد أي ارتباط واضح بين درجة اكتمال المعلومات الاقتصادية، ونتائج الطلب، بل حاولت لجنة الخبراء سد فجوات البيانات في الطلبات المقدمة عن طريق إجراء المزيد من المراجعة والتحليل للبيانات المتعلقة بالطلب.

الاستنتاج يجب على منظمة الصحة العالمية مراجعة التعليمات الخاصة بمقدمي الطلبات، والمتعلقة بمتطلبات البيانات الاقتصادية، كما يجب عليها وضع آليات جديدة لمساعدة مقدمي الطلبات في إكمال إجراءات تقديم الطلب، وكذلك تحديد بعض الطرق لاستخدام البيانات الاقتصادية في اتخاذ القرارات.

摘要

目的 评估向世界卫生组织 (WHO) 申请的基本药物标准清单中所提供的经济数据的特性。同时也评估专家委员会审议申请时，这些数据在他们制定决策中所起的作用。

方法 我们分析了在 2002 年至 2013 年间提交给 WHO 专家委员会的关于选择和使用基本药物的申请。药物价格和成本效益的数据完整性从申请文件中提取，并且用四点量表编码。 我们记录了专家委员会是否讨论了经济信息以及每份申请的结果。 我们用 χ2 检验对经济数据的完整性和申请结果之间的关联性进行了评估。

结果 专家委员会共收到 134 份申请。 只有8 份申请 (6%) 包含完整的价格数据和经济评估数据。 许多申请都遗漏或曲解了申请表的经济评估部分。 尽管缺少经济数据，但委员会还是审查了所有的申请。 经济信息的完整性和申请结果之间并无重大关联。 专家委员会尽力通过进一步审查和分析申请相关数据来处理申请中的信息缺失。

结论 世界卫生组织应该修改供申请人查阅的经济数据要求说明，制定有助于申请人完成申请过程的新机制，规定决策中经济数据起作用的方法。

Резюме

Цель

Оценить качество экономических данных, предоставленных в заявках на примерный список основных лекарств, составленный Всемирной организацией здравоохранения (ВОЗ), и оценить роль этих данных при принятии комитетом экспертов решений по этим заявкам.

Методы

Мы проанализировали заявки, подаваемые в комитет экспертов ВОЗ по отбору и использованию основных лекарств за период между 2002 и 2013 гг. Полный объем данных о ценах и соотношении эффективности и расходов для конкретных лекарств были получены по документации, прилагаемой к заявкам. Данный объем оценивается по четырехбалльной шкале. Мы также отметили наличие или отсутствие обсуждения экономической информации комитетом экспертов и то, какое решение было принято по каждой из заявок. Связь между полнотой экономических данных и результатом рассмотрения заявки была оценена по критерию χ2.

Результаты

Комитет экспертов получил за это время 134 заявки. Из них только в восьми (6%) имелись полные данные о ценах и экономическая оценка. Во многих заявках раздел экономической оценки был не заполнен или заполнен неправильно. Несмотря на недостаток экономической информации, комитет рассмотрел каждую из заявок. Связи между полнотой экономической информации и результатом рассмотрения заявки выявлено не было. Комитет экспертов пытался заполнить пробелы в заявках путем дальнейшего изучения и анализа данных, связанных с отдельной заявкой.

Вывод

Комитету экспертов следует пересмотреть инструкции, выдаваемые заявителям, в части требований, касающихся экономических данных, разработать новые механизмы помощи заявителям в заполнении заявок, а также определить методы использования экономических данных при принятии решений.

Introduction

Essential medicines are those that satisfy the priority health care needs of the population.¹1. Essential medicines [Internet]. Geneva: World Health Organization; 2014. Available from: Available from: http://www.who.int/topics/essential_medicines/en/ [cited 2014 August 10].
http://www.who.int/topics/essential_medi... The World Health Organization (WHO) introduced the first Model List of Essential Medicines in 1977. Countries are encouraged to use the model list as a guide for their decisions on pharmaceutical selection and procurement. Between 1977 and 2007, over 130 countries introduced national lists of essential medicines, modifying the WHO model list for their national context.²2. The world medicines situation 2011.; Geneva: World Health Organization 2011.</edb>

The model list is updated every two years, following an application and review process by an expert committee. Any individual or institution may submit an application. Before each committee meeting, applications are published on a website for public comment and experts (committee members and external advisors) review each application.³3. EB109/8. Provisional agenda item 3.6. WHO medicines strategy. In: 109th Executive Board Meeting, Geneva, 7 December 2001.; Geneva: World Health Organization 2001. Available from: Available from: http://www.who.int/selection_medicines/committees/subcommittee/2/eeb1098%5B1%5D.pdf [cited 2015 Jul 13].
http://www.who.int/selection_medicines/c... These reviews and any public comments are also published on the website before the committee meeting. Since 2002, essential medicines have been selected via an evidence-based process, with due regard to public health relevance, evidence on efficacy and safety and comparative cost-effectiveness.¹1. Essential medicines [Internet]. Geneva: World Health Organization; 2014. Available from: Available from: http://www.who.int/topics/essential_medicines/en/ [cited 2014 August 10].
http://www.who.int/topics/essential_medi... Detailed information on the decision criteria can be found at http://www.who.int/selection_medicines/en/. The model list is divided into a core list and a complementary list. The core list includes medicines that meet the criteria of efficacy, safety and comparative cost-effectiveness,¹1. Essential medicines [Internet]. Geneva: World Health Organization; 2014. Available from: Available from: http://www.who.int/topics/essential_medicines/en/ [cited 2014 August 10].
http://www.who.int/topics/essential_medi... but expensive patented medicines are not necessarily excluded.³3. EB109/8. Provisional agenda item 3.6. WHO medicines strategy. In: 109th Executive Board Meeting, Geneva, 7 December 2001.; Geneva: World Health Organization 2001. Available from: Available from: http://www.who.int/selection_medicines/committees/subcommittee/2/eeb1098%5B1%5D.pdf [cited 2015 Jul 13].
http://www.who.int/selection_medicines/c... ⁴4. Hogerzeil HV. The concept of essential medicines: lessons for rich countries. BMJ. 2004 Nov 13;329(7475):1169-72. http://dx.doi.org/10.1136/bmj.329.7475.1169 PMID:15539676
http://dx.doi.org/10.1136/bmj.329.7475.1... The complementary list includes medicines that may require specialized facilities, that are consistently more costly, or less cost-effective in a variety of settings.⁵5. World Health Organization. The selection and use of essential medicines. World Health Organ Tech Rep Ser. 2014;985(985):i-xiv, 1-219, back cover. PMID:25080778

Additions to the model list can have a major impact on global and national decisions, with significant budgetary, ethical and health implications.⁶6. Schiff GD, Galanter WL, Duhig J, Koronkowski MJ, Lodolce AE, Pontikes P, et al. A prescription for improving drug formulary decision making. PLoS Med. 2012;9(5):1-7. http://dx.doi.org/10.1371/journal.pmed.1001220 PMID:22629233
http://dx.doi.org/10.1371/journal.pmed.1... ⁷7. Sinclair D, Gyansa-Lutterodt M, Asare B, Koduah A, Andrews E, Garner P. Integrating global and national knowledge to select medicines for children: the Ghana National Drugs Programme.. PLoS Med 2013;10(5):e1001449. http://dx.doi.org/10.1371/journal.pmed.1001449 PMID:23704834
http://dx.doi.org/10.1371/journal.pmed.1... Other studies have examined application processes and criteria for national decisions on adopting new vaccines,⁸ national health technology assessment programmes⁹9. Golan O, Hansen P, Kaplan G, Tal O. Health technology prioritization: which criteria for prioritizing new technologies and what are their relative weights? Health Policy. 2011 Oct;102(2-3):126-35. http://dx.doi.org/10.1016/j.healthpol.2010.10.012 PMID:21071107
http://dx.doi.org/10.1016/j.healthpol.20... ¹⁰10. Neumann PJ, Drummond MF, Jönsson B, Luce BR, Schwartz JS, Siebert U, et al.; International Working Group for HTA Advancement. Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations? Int J Technol Assess Health Care. 2010 Jan;26(1):71-8. http://dx.doi.org/10.1017/S0266462309990833 PMID:20059783
http://dx.doi.org/10.1017/S0266462309990... and financing decisions for health technologies.¹¹11. Stafinski T, Menon D, Philippon DJ, McCabe C. Health technology funding decision-making processes around the world: the same, yet different. Pharmacoeconomics. 2011 Jun;29(6):475-95. http://dx.doi.org/10.2165/11586420-000000000-00000 PMID:21568357
http://dx.doi.org/10.2165/11586420-00000... A recent study found that applications to the model list for mental health medicines generally provided low-quality and incomplete evidence across several required dimensions.¹²12. Barbui C, Purgato M. Decisions on WHO's essential medicines need more scrutiny. BMJ. 2014;349 jul31 1:g4798. http://dx.doi.org/10.1136/bmj.g4798 PMID:25081250
http://dx.doi.org/10.1136/bmj.g4798... There is no global consensus on how to use economic data in decision-making for medicines and health technologies.¹³13. Neumann PJ, Cohen JT, Weinstein MC. Updating cost-effectiveness-the curious resilience of the $50,000-per-QALY threshold. N Engl J Med. 2014 Aug 28;371(9):796-7. http://dx.doi.org/10.1056/NEJMp1405158 PMID:25162885
http://dx.doi.org/10.1056/NEJMp1405158... ¹⁷17. Stolk P, Willemen MJ, Leufkens HG. Rare essentials: drugs for rare diseases as essential medicines. Bull World Health Organ. 2006 Sep;84(9):745-51. http://dx.doi.org/10.2471/BLT.06.031518 PMID:17128345
http://dx.doi.org/10.2471/BLT.06.031518...

Here, we evaluate the extent to which applicants and the expert committee adhere to the instructions and procedures on economic data and analysis for applications to the model list. We assess the application process rather than the substance of decisions (specifically, compliance with instructions for applicants and the quality of data on price and on cost-effectiveness). We also assess whether economic considerations are included in the final report by the expert committee. The overall goal of this study is to improve the quality, transparency and clarity of the process for reviewing applications to the model list.

Methods

Data set

We included applications to the model list for medicines intended for use in adults. We analysed final reports from the twelfth (2002) to nineteenth (2013) meetings of the WHO expert committee; the sixteenth meeting reviewed paediatric medicines only and was not included. We did not review applications for new formulations of existing medicines or applications for reinstatement.

Definition of variables

We assessed the extent to which applicants complied with the instructions provided. The instructions ask applicants to provide "a range of costs of the proposed medicine" that reflect "average generic world market prices", with a clearly specified source of data. The WHO-recommended sources for these data include the International Drug Price Indicator Guide, the United Nations Children's Fund, Médecins Sans Frontières or WHO itself. We allocated one point for each of the following: price given; offered range of price or average or median price; clear source; WHO-recommended source. The application was classified as complete if the application attained a score of four; applications with scores between one and three points (inclusive) were classified as incomplete. If no pricing data were provided, this was considered missing (zero points). If the applicant explicitly mentioned that no pricing data exist, this was classified as not available.

Completeness of economic evaluation data was scored based on WHO instructions for comparative cost-effectiveness presented as range of cost per routine outcome, and a clear citation, preferably of WHO-recommended sources. We also accepted citations from the published scientific literature as one of the recommended data sources. Adherence was classified as complete if applications met all of the following criteria: economic evaluation value provided; comparator given; clear source; WHO-recommended source. As with price, applications scoring between one and three inclusive were classified as incomplete. Omitted economic evaluation sections were classified as missing; cases where the applicant explicitly noted the lack of available data on economic evaluation were classified as not available. There were instances in which the applicant provided administration costs for medicines, without an outcomes-based denominator, under the heading of cost-effectiveness. These applications were classified as having provided economic evaluation data that were not applicable.

Based on the final reports from each meeting, the outcome was coded as a dichotomous variable: either rejected or deferred (coded 0) versus accepted to the core or complementary lists (coded 1). Applicant type and therapeutic class were categorized based on information provided in the application. We recorded the proportion of applications reviewed by the committee that were added to the model list, by therapeutic class. We also recorded whether the discussion and decisions mentioned price or economic evaluation.

Data entry and analysis

A data entry form was generated based on the variables listed above. There were two independent data reviewers, each of whom extracted data from the applications; discrepancies were resolved by consensus. Application data were entered using CSPro version 5 (United States Census Bureau, Washington, United States of America).

Qualitative information from discussions and application decision categories were coded by hand. We extracted the narrative sections about the reviewed medicines from the final report for each committee meeting; text concerning economic considerations was collated in an Excel spreadsheet. We identified whether the report mentioned economic data or the absence of such data, as well as which data sources were cited (i.e. the application itself or new additional data).

Associations between the completeness of economic data and the application outcomes were assessed using χ2 tests in Stata version 12.1 (StataCorp. LP, College Station, USA). We conducted sensitivity analyses that dropped each scoring criterion in turn.

Results

We analysed 134 applications for new medicines from 2002 to 2013 (Table 1). Application authors varied by year, but were most commonly submitted by WHO departments (25.4%; 34/134), academic institutions (22.4%; 30/134) and nongovernmental organizations (21.6%; 29/134). Therapeutic classes of medicines also varied between meetings; applications for cancer and diabetes medicines were more common in recent years. Of 134 applications analysed, 52 were rejected and decisions were deferred for an additional 10. Among the 72 applications added to the model list, 67 were added to the core list and five to the complementary list.

Across all meeting years, only 6.0% of applications (8/134) provided complete price and economic evaluation data (Table 2). Since no time trends were apparent, we report totals for all years; analyses by year are available from the corresponding author. If we include applications that reported economic data were not available, assuming that these applicants made an effort to include data but that none were available, 13.4% (18/134) of applications fully followed the instructions on price and economic evaluation. Only 20.9% (28/134) and 18.7% (25/134) of applications provided complete price and economic evaluation information, respectively. Over a third (36.6%; 49/134) included no economic evaluation information, one-fifth (20.9%; 28/139) stated that economic evaluation information was unavailable and 17.9% (24/134) presented financial information that was unrelated to economic evaluation (e.g. total treatment cost).

All applications were reviewed by the committee. For the applications with complete economic data, 52.0% (13/25) was added to the model list. When we included applications that stated that economic data were not available, 50.9% (27/53) was added to the model list. Among applications with no price data or no economic evaluation, over one-third and over half, respectively, of these applications were added to the model list.

There was variation in the extent of economic information provided by applicant type. Academic authors were most likely to provide complete price and economic evaluation information. Applicants from WHO did not include economic evaluation information in half of their applications and industry applicants very rarely provided complete information on price or economic evaluation. Detailed data on the completeness of economic information by applicant type and therapeutic class are available from the corresponding author.

For applications with complete price data, 64.3% (18/28) were added to the model list, a percentage that was not significantly higher than for applications with incomplete or absent price information (50.5%; 50/99; P= 0.2). For applications with complete economic evaluation information 52.0%, (13/25) were added to the model list, compared to 54.1% for those with incomplete, missing, inapplicable or unavailable information (59/109; P= 0.9).

We explored whether the committee's discussions of economic factors reflected the content of the applications. As shown in Table 3, price data or economic evaluations were discussed in some cases, even where this information had not been included in the application. Our qualitative analysis suggests four ways in which this happened. First, the committee sometimes noted the lack of information in the application (two applications with missing price data, seven applications with missing economic evaluation); in three of these cases, the medicines were nonetheless added to the core list. Second, the committee sometimes referenced economic information, suggesting an alternative, although unspecified, source of supplementary data (six applications). Third, the committee sometimes did further research: for example, the report mentioned a review of cost-effectiveness data prepared by the secretariat, or indicated that the secretariat (usually comprised of WHO staff who support the committee process) performed its own economic evaluation (four applications). Fourth, in two instances where an economic evaluation was missing, the committee discussed price data included in the application but referred to it as "cost-effectiveness," suggesting a misinterpretation.

We explored whether the distinction between core and complementary lists was followed in the decision process. A feature that distinguishes the complementary list is its inclusion of medicines requiring specialized diagnostic or monitoring facilities and/or specialist medical care and/or specialist training.⁵ According to our analysis, 50% (46/92) of applications that we classified as belonging to the complementary list were nonetheless added to the core list by the committee. This included all psychiatric medicines and antiretroviral medicines for treating human immunodeficiency virus infections.

We conducted sensitivity analyses, to examine the relative importance of each scoring criterion. The most influential variables were: for price data, the average generic world price and for economic evaluations, the choice of comparator (i.e. presenting relative or incremental data). The fourth criterion, use of a WHO-recommended source for economic data, might be seen as an optional point: dropping this criterion from the analysis did not substantially increase the number of applications scored as complete (from 20.9% complete on price to 28.4% and from 18.7% complete for economic evaluation to 19.4%).

Discussion

Very few applications complied fully with the instructions on providing price data and economic evaluations. The quality of information provided in applications to the Model List of Essential Medicines can be improved substantially. Despite the majority of applications being submitted without the required economic information, the committee reviewed all applications. There was no significant association between the completeness of economic information and the outcome of applications: provision of economic data in an application is not necessary for a positive decision by the committee.

Our qualitative findings suggest that in some cases, the committee found alternative ways to address information gaps - such as internal literature reviews and analyses. However, such data collection and analysis requires additional time on the part of the committee and reviewers. Since the committee still considered and approved incomplete applications, this may have reinforced applicants' decisions to not comply with instructions. There is little evidence that applicants have improved their efforts to submit requested information over time.

There are several possible reasons why applicants often did not provide complete or robust economic data. First, this information may be hard to collect: high-quality price and economic evaluation data that are relevant for resource-poor settings may be difficult to find in the literature; applicants may lack the technical skills to conduct their own economic evaluation. Such issues should, however, be less of a barrier for applicants from private companies, who have access to proprietary data, but applications from companies had some of the lowest rates of provision of economic information. Second, the application instructions may be unclear or insufficiently detailed, as suggested at a recent committee meeting. Application instructions for the recent twentieth committee meeting were revised,¹⁸18. 20th Expert Committee on the Selection and Use of Essential Medicines [Internet].; Geneva: World Health Organization 2015. Available from: Available from: http://www.who.int/selection_medicines/committees/expert/20/en/ [cited 2015 July 1].
http://www.who.int/selection_medicines/c... but still do not fully incorporate recent recommendations, such as asking applicants to provide comprehensive search strategies or stating that only complete applications would be reviewed. Third, the committee itself may not be clear about whether economic studies should be a necessary component of the decision process. Our qualitative analysis suggests that other considerations - such as safety and tolerability - are sometimes given higher priority even in cases of unfavourable price or economic evaluation data. However, it is often unclear which criteria were emphasized during the committee's review and decision process. There are also challenges in using economic criteria, such as comparative cost-effectiveness, in decision-making for medicines. These challenges include information deficiencies and a lack of universal standards on appropriate thresholds.

Our study has some limitations. We did not assess the accuracy of information provided and when an applicant noted that no data were available, we did not attempt to verify this. Our analysis of committee discussions and decisions was limited to the publicly-available meeting reports, which may not capture all aspects of the meeting. To overcome the various data limitations, we relaxed certain criteria in the application instructions to allow more flexibility in assessing adherence to the instructions: for example, we expanded the average generic world market price criterion to permit inclusion of median prices and prices per country-group (e.g. low-income countries); we allowed other published data sources (e.g. peer-reviewed manuscripts) or internal sources (e.g. clinical trial dossiers) for economic evaluation.

In conclusion, we have three recommendations to improve the review and decision-making process for the Model List of Essential Medicines. WHO should: (i) provide clear and detailed instructions about how much and what kind of economic data are required; (ii) develop mechanisms to assist applicants in completing an application; and (iii) provide clear rules about how economic data will be used in making decisions about applications and about the consequences of not providing economic data (Box 1). To implement these recommendations, the current application form needs to be revised and examples of high-quality applications and an interactive, guided, application process should be provided. Many countries are debating how to use economic data and analysis in decision-making processes about essential medicines. WHO could assist these countries by providing clear examples of how applications should be prepared and how decisions should be reached. The model list is an important global tool and decisions about new additions have significant implications for national policies and budgets, as well as clinical decision-making.

References

Publication Dates

History