The health workforce demand: a systematic literature review

A procura por recursos humanos em saúde: revisão sistemática da literatura

Diana Fernandes Lopes Ana Luísa Ramos Eduardo Anselmo de Castro About the authors

Abstract

Understanding imbalances between the supply and demand of the human resour- ces for health (HRH) is essential for enhancing health outcomes. Addressing the HRH demand is particularly challenging, especially given the deficit of accurate data and surplus of unresolved methodological flaws. This study presents a systematic review of the literature surrounding HRH demand and answers the following key questions: How has HRH demand been addressed? What are the harms and barriers that accompany HRH demand modeling? This systematic review was performed following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) statement. Relevant keywords were used in a thorough search of the PubMed/MEDLINE, SCOPUS, and Web of Science databases. A total of 2,599 papers were retrieved and evaluated according to their title and abstract. Of these, the full-text of 400 papers was analyzed, 53 of which successfully met the inclusion criteria in our study. While the topic’s relevance is widespread, it still lacks a validated approach to model HRH demand adequately. The main characteristics of the applied methods are presented, such as their application complexity by health policymakers. Opportunities and orientations for further research are also highlighted.

Key words:
Needs and Demand; Health Workforce; Systematic Review; Planning

Resumo

A compreensão dos desequilíbrios entre a demanda e a oferta da mão de obra em saúde (MOS) é essencial para a melhoria dos resultados em saúde. A demanda por MOS, em particular, é bastante desafiante dada a falta de dados e aspetos metodológicos que ainda estão por abordar. Este estudo apresenta uma revisão sistemática da literatura procurando responder às seguintes questões: Como tem sido abordada a demanda por MOS? Quais são as barreiras e limitações neste processo? A metodologia adotada baseou-se no PRISMA statement (Preferred Reporting Items for Systematic Reviews and Meta-analyses). A pesquisa foi realizada nas bases de dados PubMed/MEDLINE, SCOPUS e Web of Science, usando palavras-chave relevantes. No total 2 599 artigos foram recolhidos e analisados de acordo com os respetivos títulos e resumos. O texto integral de 400 artigos foi alvo de análise, tendo sido selecionados 53 artigos com base em critérios pré-definidos. Apesar da relevância do tema, não existe, ainda, um método para modelar a procura por MOS validado pela literatura científica. Esta revisão, além de destacar oportunidades e apontar direções de investigação futura, apresenta as principais características da modelação da procura de MOS, como a sua aplicabilidade na ótica do utilizador (decisor político).

Palavras-chave:
Necessidades e Demandas; Mão de obra em saúde; Revisão sistemática; Planejamento

Introduction

Human resources for health (HRH) mark the cornerstone of any given health system and society since they provide vital health services to the population. Despite the emphasis of the health workforce planning on the international public health agenda, as has been highlighted by the World Health Organization (WHO)11 World Health Organization (WHO). Models and tools for health workforce planning and projections. Genebra: WHO; 2010., the Organization for Economic Co-operation and Development (OECD)22 Ono T, Lafortune G, Schoenstein M. Health workforce planning in OECD countries: A review of 26 projection models from 18 countries. Paris: OECD Publishing; 2013. (OECD Health Working Papers, N. 62), the Joint Action on European Health Workforce Planning and Forecasting promoted by the European Union (EU) Public Health Programme 2008-201333 Kroezen M, Hoegaerden M van, Batenburg R. The Joint Action on Health Workforce Planning and Forecasting: Results of a European programme to improve health workforce policies. Health Policy (New York) 2018; 122:87-93. and by several renowned, international peer-reviewed scientific journals, it has become widely addressed as a significant worldwide concern, with several countries reporting a huge gap between HRH demand and supply, a critical issue which has led to a variety of troubling consequences.

Despite their worldwide relevance, demand analyses still have unresolved methodological weaknesses. A confluence of forces has contributed to this situation. First, there is a confusion between the terms demand for and needs of HRH44 Canvin RW, Boldy DP. Resource planning within the healthcare planning teams. Soc Sci Med 1976; 10:171-176.,55 Lopes MA, Almeida ÁS, Almada-Lobo B. Handling healthcare workforce planning with care: where do we stand? Hum Resour Health 2015; 13:38.. For example, the term “demand” is very often used as a quantification of the utilization of health care services, also called “expressed needs”44 Canvin RW, Boldy DP. Resource planning within the healthcare planning teams. Soc Sci Med 1976; 10:171-176. (later converted to a certain amount of HRH needed). Likewise, it is also used to express the health care needs of a population by using incidence and prevalence rates. Unlike this example, the concept of HRH need is not purely quantitative but also normative. As such, it does not always align with the economic and financial considerations that might limit its definition66 Dussault G, Vujicic M. Demand and supply of human resources for health. Int Encycl Public Heal 2008; 77-84.. Less popular definitions include “felt need” or “want”, as expressed on behalf of the general population, but this definition is neither representative of real need nor useful for planning purposes44 Canvin RW, Boldy DP. Resource planning within the healthcare planning teams. Soc Sci Med 1976; 10:171-176.. Different definitions hinder the development of a clear understanding of the concept of HRH and, in discussions on the topic, often generate understanding impasses.

Besides the lack of consensus on defining HRH demand, there is also a deficit of valid and accurate data on the health workforce in general, particularly concerning demand-related indicators, which compromises the construction of robust models that could effectively address this topic77 Leone C, Conceição C, Dussault G. Trends of cross-border mobility of physicians and nurses between Portugal and Spain. Hum Resour Health 2013;11:1-11.. However, noteworthy is that many efforts have been devoted to the design and development of methods to model the HRH demand, which are found in various studies in the literature, differing on their nature, scale and methodology.

Given the above difficulties in addressing the problem of HRH demand, this study aims to provide a systematic review of the literature surrounding the modeling of the HRH demand. A review is necessary not only given the global recognition and relevance of this topic but also because of the significant efforts that have been dedicated to this field, which, although they have produced inconclusive evidence on a valid approach to address the topic, may provide clues as to the better way forward in the effective and accurate modeling of HRH demand. Previous reviews found in the literature in this field are characterized by a broad scope, encompassing both supply and demand (needs) but focusing on specific HRH fields55 Lopes MA, Almeida ÁS, Almada-Lobo B. Handling healthcare workforce planning with care: where do we stand? Hum Resour Health 2015; 13:38.,88 Roberfroid D, Leonard C, Stordeur S. Physician supply forecast: better than peering in a crystal ball? Hum Resour Health 2009;7:1-13. or particular periods99 O'Brien-Pallas L, Baumann A, Donner G, Murphy GT, Lochhaas-Gerlach J, Luba M. Forecasting models for human resources in health care. J Adv Nurs 2001; 33:120-129.. This study contributes to the literature by expanding its lenses beyond specific HRH fields or period and, instead, by addressing the demand for all workers providing health care to populations, and by identifying the advantages and disadvantages of existing approaches in the light of their practicability by policymakers. As such, to the best of our knowledge, this study is the first to comprehensively cover the issue of HRH demand for a wide variety of health professions, periods, and across different application scales.

This paper starts by describing the design of the systematic review (Methods), followed by the results of the review with an overview of selected papers (Results). Finally, we reflect on our review’s primary outcomes and summarize significant potentials and concerns for research going forward (Discussion).

Methods

This study aims to provide a systematic review of the literature surrounding the modeling of HRH demand and intends to answer the following research questions:

How has HRH demand been previously addressed in real-world applications?

What are the advantages and barriers of modeling demand for HRH?

This systematic review methodology is based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) statement1010 Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med 2009; 151:264-269..

Literature Search

The literature search was conducted using two general databases - SCOPUS and Web of Science - and one specific database - PubMed/MEDLINE. A widespread search strategy was developed to assess the maximum number of appropriate studies in each database. All the possible formulations of the terms presented in Appendix A (combined with Boolean operators) were used in the search, performed on January 7, 2019.

Eligibility Criteria

The systematic review included papers in English published up to the end of 2018. Two screening rounds were performed following the removal of duplicates. At first, two authors independently reviewed all titles and abstracts of the retrieved papers. Search results that were outside the study’s context were excluded. Next, the full-text of the results was evaluated according to the inclusion and exclusion criteria.

Data Extraction

The search results were incorporated into Microsoft Excel Professional Plus 2016, in which the following information was gathered and registered: (i) authors; (ii) title; (iii) abstract; (iv) publication year; and (v) source of publication. Next, studies were extensively analyzed to evaluate their relevance for inclusion in the present review, focusing the following aspects: (i) objectives; (ii) health profession; (iv) methods; (v) scale; and (vi) typology of health care services. Any hesitation or disagreement would require authors to discuss each case until they reached a consensus. The gathered data was then analyzed to provide initial answers to the research questions mentioned above.

Results

Selected Studies

A total of 2,946 titles were identified in the initial search, including 1,260 papers from the Web of Science, 1,211 from Scopus, and 475 papers from PubMed. Following the initial screening and removal of duplicates, 2,599 papers remained. After reviewing the titles and abstracts of these remaining papers, 2,199 were excluded from the study. After these screening procedures, 400 papers remained - the full text of which was assessed and evaluated per the eligibility criteria, which resulted in eliminating an additional 347 papers. Excluded studies from this review involved papers that:

  • Were only available as abstracts;

  • Reported reviews of demand-related methods;

  • Only discussed HRH demand planning good practices or guidelines;

  • Included epidemiological studies or other indirect issues, such as:

  • Papers discussing the link between the use of health care services (e.g., the number of visits) and demographic indicators;

  • Studies focusing on the accessibility of a population to health care facilities;

  • Encompassed ad hoc techniques (e.g., in which demand estimations were exclusively based on current service utilization rates and disregarded experts’ opinions on whether these health delivery patterns were adequate or optimal), non-specified methods, or theoretical papers (non-applicative) to avoid the inclusion of methods that might be impractical in real-world contexts.

In total fifty-three papers1111 Pizam A, Neumann Y. Planning of medical manporwer. Long Range Plann 1976; 9:44-52.

12 Song F, Rathwell T. Future demand for doctors in China. Health Policy Plan 1993; 8:394-404.

13 Murphy GT, MacKenzie A, Alder R, Birch S, Kephart G, O'Brien-Pallas L. An applied simulation simulation model for estimating the supply of and requirements for registered nurses based on population health needs. Policy, Polit Nurs Pract 2009; 10:240-251.

14 Tuulonen A, Salminen H, Linna M, Perkola M. The need and total cost of Finnish eyecare services: A simulation model for 2005-2040. Acta Ophthalmol 2009; 87:820-829.

15 Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrisey JP. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv 2009; 60(10):1323-1328.

16 Withy K, Sakamoto D. Assessing physician workforce using insurance claims data and focus groups compared to benchmarks. Ethn Dis 2010; 20(Supl. 1):211-214.

17 Gallagher JE, Kleinman ER, Harper PR. Modelling workforce skill-mix: How can dental professionals meet the needs and demands of older people in England? Br Dent J 2010; 208(3):116-118.

18 Zimbelman JL, Juraschek SP, Zhang X, Lin VW. Physical therapy workforce in the United States: forecasting nationwide shortages. PM R 2010; 2(11):1021-1029.

19 Vanderby SA, Carter MW, Latham T, Ouzounian M, Hassan A, Tang GH, Teng CJ, Kingsbury K, Feindel CM. Modeling the cardiac surgery workforce in Canada. Ann Thorac Surg 2010; 90(2):467-473.

20 Hofer AN, Abraham JM, Moscovice I. Expansion of coverage under the patient protection and affordable care act and primary care utilization. Milbank Q 2011; 89(1):69-89.

21 Kim JSC, Cooper RA, Kennedy DW. Otolaryngology-head and neck surgery physician work force issues: An analysis for future specialty planning. Otolaryngol Neck Surg 2012; 146(2):196-202.

22 Yuji K, Imoto S, Yamaguchi R, Matsumura T, Murashige N, Kodama Y, Minayo S, Imai K, Kami M. Forecasting Japan's Physician Shortage in 2035 as the First Full-Fledged Aged Society. PLoS One 2012;7(11):e50410.

23 Lee PP, Jackson CA, Relles DA. Estimating eye care workforce supply and requirements. Ophthalmology 1995; 102(12):1964-1972.

24 Stein ML, Rudge JW, Coker R, Weijden C van der, Krumkamp R, Hanvoravongchai P, Chavez I, Putthasri W, Phommasack B, Adisasmito W, Touch S, Sat le M, Hsu YC, Kretzschmar M, Timen A. Development of a resource modelling tool to support decision makers in pandemic influenza preparedness: The AsiaFluCap Simulator. BMC Public Health 2012; 12:1-14.

25 Hooker RS, Williams JH, Papneja J. Dietetics supply and demand: 2010-2020. J Acad Nutr Diet 2012; 112(3):S75-S91.

26 Murphy GT, Birch S, MacKenzie A, Alder R, Lethbridge L, Little L. Eliminating the shortage of registered nurses in Canada: An exercise in applied needs-based planning. Health Policy (New York) 2012; 105(2-3):192-202.

27 Dall TM, Storm MV, Chakrabarti R, Drogan O, Keran CM, Donofrio PD, Henderson VW, Kaminski HJ, Stevens JC, Vidic TR. Supply and demand analysis of the current and future US neurology workforce. Neurology 2013(5); 81:470-478.

28 Gallagher JE, Lim Z, Harper PR. Workforce skill mix: Modelling the potential for dental therapists in state-funded primary dental care. Int Dent J 2013; 63(2):57-64.

29 Wu M-H, Yu J-Y, Huang C-H. Theoretical system dynamics modeling for Taiwan pediatric workforce in an era of national health insurance and low birth rates. Pediatr Neonatol 2013; 54(6):389-396.

30 Harper P, Kleinman E, Gallagher J, Knight V. Cost-effective workforce planning: Optimising the dental team skill-mix for England. J Enterp Inf Manag 2013; 26(1/2):91-108.

31 Murphy GT, MacKenzie A, Guy-Walker J, Walker C. Needs-based human resources for health planning in Jamaica: Using simulation modelling to inform policy options for pharmacists in the public sector. Hum Resour Health 2014; 12:1-11.

32 Vanderby SA, Carter MW, Latham T, Feindel C. Modelling the future of the Canadian cardiac surgery workforce using system dynamics. J Oper Res Soc 2014; 65:1325-1335.

33 Edwards JP, Datta I, Hunt JD, Stefan K, Ball CG, Dixon E, Grondin SC. A novel approach for the accurate prediction of thoracic surgery workforce requirements in Canada. J Thorac Cardiovasc Surg 2014; 148(1):7-12.

34 Holliman CJ, Wuerz RC, Hirshberg AJ. Analysis of factors affecting U.S. emergency physician workforce projections. Acad Emerg Med 1997; 4(7):731-735.

35 Taba ST, Atkinson SR, Lewis S, Chung KSK, Hossain L. A systems life cycle approach to managing the radiology profession: an Australian perspective. Aust Heal Rev 2015;3 9(2):228-239.

36 Schell GJ, Lavieri MS, Li X, Toriello A, Martyn KK. Strategic modelling of the pediatric nurse practitioner workforce. Pediatrics 2015; 135(2):298-306.

37 Brailsford S, Silva D. How many dentists does Sri Lanka need? Modelling to inform policy decisions. J Oper Res Soc 2015; 66:1566-1577.

38 Senese F, Tubertini P, Mazzocchetti A, Lodi A, Ruozi C, Grilli R. Forecasting future needs and optimal allocation of medical residency positions: The Emilia-Romagna region case study. Hum Resour Health 2015; 13(1):7.

39 Lin V, Zhang X, Dixon P. Occupational therapy workforce in the United States: Forecasting nationwide shortages. PM R 2015; 7(9):946-954.

40 Ansah JP, Korne DD, Bayer S, Pan C, Jayabas kar T, Matchar DB, Lew N, Phua A, Koh V, Lamoureux E, Quek D. Future requirements for and supply of ophthalmologists for an aging population in Singapore. Hum Resour Health 2015; 13(1):1-13.

41 Robboy SJ, Gupta S, Crawford JM, Cohen MB, Karcher DS, Leonard DG, Magnani B, Novis DA, Prystowsky MB, Powell SZ, Gross DJ, Black-Schaffer WS. The pathologist workforce in the United States: II. An interactive modeling tool for analyzing future qualitative and quantitative staffing demands for services. Arch Pathol Lab Med 2015; 139(11):1413-1430.

42 Nirel N, Grinstien-Cohen O, Eyal Y, Samuel H, Ben-Shoham A. Models for projecting supply and demand for nurses in Israel. Isr J Health Policy Res 2015; 4:1-12.

43 Hu W, Lavieri MS, Toriello A, Liu X. Strategic health workforce planning. IIE Trans 2016; 48(12):1127-1138.

44 Borracci RA, Milin E, Gelpi R. Long-term estimate of the number of doctors in Argentina. Argentine J Cardiol 2016; 84(1):25-30.

45 Lipscomb J, Parmigiani G, Hasselblad V. Combining expert judgement by hierarchical modelling: An application to physician staffing. Manage Sci 1998; 44(2):149-1461.

46 Jager R, Berg N van der, Hoffmann W, Jordan RA, Schwendicke F. Estimating future dental services' demand and supply: A model for Northern Germany. Community Dent Oral Epidemiol 2016; 44(2):169-179.

47 Murphy GT, Birch S. Simulating future supply of and requirements for human resources for health in high-income OECD countries. Hum Resour Health 2016; 14(1):1-18.

48 Laurence CO, Karnon J. Improving the planning of the GP workforce in Australia: a simulation model incorporating work transitions, health need and service usage. Hum Resour Health 2016; 14:1-14.

49 Lodi A, Tubertini P, Grilli R, Mazzocchetti A, Ruozi C, Senese F. Needs forecast and fund allocation of medical specialty positions in Emilia-Romagna (Italy) by system dynamics and integer programming. Heal Syst 2016; 5(3):213-236.

50 Hoope-Bender P ten, Nove A, Sochas L, Mattews Z, Homer C, Pozo-Martin F. The 'Dream Team' for sexual, reproductive, maternal, newborn and adolescent health: an adjusted service target model to estimate the ideal mix of health care professionals to cover population need. Hum Resour Health 2017; 15(1):46.

51 Oh Y. The future prospects of supply and demand for urologists in Korea. Investig Clin Urol 2017; 58(6):400-408.

52 Laurence CO, Karnon J. What will make a difference? Assessing the impact of policy and non-policy scenarios on estimations of the future GP workforce. Hum Resour Health 2017; 15(1):1-15.

53 Ishikawa T, Fujiwara K, Ohba H, Suzuki T, Ogasawara K. Forecasting the regional distribution and sufficiency of physicians in Japan with a coupled system dynamics-geographic information system model. Hum Resour Health 2017; 15(1):1-9.

54 Liu JX, Goryakin Y, Maeda A, Bruckner T, Scheffler R. Global health workforce labor market projections for 2030. Hum Resour Health 2017; 15:1-12.

55 Segal L, Guy S, Leach M, Groves A, Turnbull C, Furber G. A needs-based workforce model to deiver tertiary-level community menatl health care for distressed infants, children, and adolescents in South Australia: a mixed-methods study. Lancet Public Heal 2018; 3(6):e296-303.

56 Angus DC, Kelley MA, Schmitz RJ, White A, Popovich J, Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS). Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease - Can we meet the requirements of an aging population? J Am Med Assoc 2000; 284:2762-2770.

57 Cruz-Gomes S, Lopes AM, Almada-Lobo B. A labor requirements function for sizing the health workfoce. Hum Resour Health 2018; 16:1-12.

58 Laurence CO, Heywood T, Bell J, Atkinson K, Karnon J. The never ending road: improving, adapting and refining a needs-based model to estimate future general practitioner requirements in two Australian states. Fam Pract 2018; 35:193-198.

59 Lopes MA, Almeida AS, Almada-Lobo B. Forecasting the medical workforce: a stochastic agent-based simulation approach. Health Care Manag Sci 2018; 21(1):52-75.

60 Schubert A, Eckhout G, Cooperider T, Kuhel A. Evidence of a current and lasting national anesthesia personnel shortfall: Scope and implications. Mayo Clin Proc 2001; 76(10):995-1010.

61 Rizza RA, Vigersky RA, Rodbard HW, Ladenson PW, Young WFJ, Surks MI, Kahn R, Hogan PF A model to determine workforce needs for endocrinologists in the United States until 2020. J Clin Endocrinol Metab 2003; 88(5):1979-1987.

62 Vijan S, Inadomi J, Hayward RA, Hofer TP, Fendrick AM. Projections of demand and capacity for colonoscopy related to increasing rates of colorectal cancer screening in the United States. Aliment Phamacology Ther 2004; 20(5):507-5015.
-6363 Scheffler RM, Liu JX, Dal Poz MR. Forecasting the global shortage of physicians: An economic- and needs-based approach. Bulletin of the World Health Organization 2008; 86:516-523. were included in this systematic review (Figure 1) - this result did not consider manual search, i.e., bibliographies of the selected papers were not searched.

Figure 1
PRISMA flowchart of the study selection (cut-off date: 7th of January 2019).

Overview of the Selected Studies

The selected studies’ publication years ranged from 1976 to 2018, and most were published from 2009 onwards (Figure 2a). About 81% of the selected studies were published in the last decade, and of these, 67% were published in the last five years (2014-2018), which reveals this topic has gained momentum in the last years.

Figure 2
Overview of the general aspects of the selected publications.

Approximately 34% of the papers described U.S.-based applications, and 17% described European-based applications (Figure 2b). Five studies were conducted in Canada and Australia, respectively, and two studies were conducted in both Israel and Japan. Four studies were applied to more than one country: WHO countries (n=2), OECD countries (n=1), and Southeast Asian countries (n=1). Appendix B provides a map with the spatial distribution of the selected studies per country of affiliation. Regarding the scale of application (Figure 3a), most studies (n=36) were applied at a national level (country). Nine studies were implemented at a state level; these ranged from single-state studies (e.g., South Australia) to studies involving multiple states (e.g., any of the individual 50 U.S. states). The remaining eight studies were conducted in specific regions.

Figure 3
Characterization of the selected articles according to: (a) scale of application and (b) scientific field.

Forty-one of the 53 studies derived from different journals, which denotes a great variety of scope and research areas. Almost half of the papers were published in clinical journals (i.e., journals dedicated to one or more health professions), and most were included in medical journals. The remaining 27 studies were published in health policy and management journals (Figure 3b). The list of journal titles organized according to the journal’s characteristics is provided in Appendix C.

Also, noteworthy is that the papers of Angus et al.56 and Thomas et al.1515 Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrisey JP. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv 2009; 60(10):1323-1328. were the most often cited, and both were published in journals devoted to the medical profession (Appendix D). Mainly, Angus et al.5656 Angus DC, Kelley MA, Schmitz RJ, White A, Popovich J, Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS). Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease - Can we meet the requirements of an aging population? J Am Med Assoc 2000; 284:2762-2770. aimed to estimate the requirements for adult care and pulmonologists in the U.S. In contrast, Thomas et al.1515 Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrisey JP. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv 2009; 60(10):1323-1328. focused on mental health providers divided into two professional groups, namely, prescribers (psychiatrists) and non-prescribers (social workers, psychologists, nurses, among others).

Health Professions and Typology of Services

More than half of the selected studies were focused on the medical profession (n=34). Ten papers approached the profession globally, and the remaining papers were related to one or more specialties. Regarding the latter, it is clear from our review that significant endeavors have been devoted to studying HRH demand in the medical specialties of family medicine, ophthalmology, and surgical specialties (see Appendix E).

Nursing and dental professions (e.g., dentists, hygienists, therapists, and technicians) are the second occupations with the highest number of studies (five studies each), with some being focused on specific nursing specialties, such as orthopedics, pediatrics, and neonatal fields. Four papers addressed other health professions (e.g., pharmacy, nutrition science, among others).

Despite 90% of the studies being focused on a single health career, since 2010, an increasing number of studies have been addressing multiple professions2424 Stein ML, Rudge JW, Coker R, Weijden C van der, Krumkamp R, Hanvoravongchai P, Chavez I, Putthasri W, Phommasack B, Adisasmito W, Touch S, Sat le M, Hsu YC, Kretzschmar M, Timen A. Development of a resource modelling tool to support decision makers in pandemic influenza preparedness: The AsiaFluCap Simulator. BMC Public Health 2012; 12:1-14.,4747 Murphy GT, Birch S. Simulating future supply of and requirements for human resources for health in high-income OECD countries. Hum Resour Health 2016; 14(1):1-18.,5050 Hoope-Bender P ten, Nove A, Sochas L, Mattews Z, Homer C, Pozo-Martin F. The 'Dream Team' for sexual, reproductive, maternal, newborn and adolescent health: an adjusted service target model to estimate the ideal mix of health care professionals to cover population need. Hum Resour Health 2017; 15(1):46.,5454 Liu JX, Goryakin Y, Maeda A, Bruckner T, Scheffler R. Global health workforce labor market projections for 2030. Hum Resour Health 2017; 15:1-12.,5555 Segal L, Guy S, Leach M, Groves A, Turnbull C, Furber G. A needs-based workforce model to deiver tertiary-level community menatl health care for distressed infants, children, and adolescents in South Australia: a mixed-methods study. Lancet Public Heal 2018; 3(6):e296-303.. For instance, Stein et al.2424 Stein ML, Rudge JW, Coker R, Weijden C van der, Krumkamp R, Hanvoravongchai P, Chavez I, Putthasri W, Phommasack B, Adisasmito W, Touch S, Sat le M, Hsu YC, Kretzschmar M, Timen A. Development of a resource modelling tool to support decision makers in pandemic influenza preparedness: The AsiaFluCap Simulator. BMC Public Health 2012; 12:1-14. covered both physicians and nurses, Murphy et al.4747 Murphy GT, Birch S. Simulating future supply of and requirements for human resources for health in high-income OECD countries. Hum Resour Health 2016; 14(1):1-18. addressed physicians, nurses and midwives, and Hoope-Bender et al.5050 Hoope-Bender P ten, Nove A, Sochas L, Mattews Z, Homer C, Pozo-Martin F. The 'Dream Team' for sexual, reproductive, maternal, newborn and adolescent health: an adjusted service target model to estimate the ideal mix of health care professionals to cover population need. Hum Resour Health 2017; 15(1):46. focused on sexual, reproductive, maternal, neonatal and adolescent HRH. Therapists and psychiatrists were also included in the remaining studies (Figure 4a).

Figure 4
Characterization of the selected articles according to: (a) profession, and (b) typology of healthcare services.

Concerning the typology of the health care services addressed, more than half of the selected studies were dedicated to secondary health care services (Figure 4b). This finding was expected since central hospitals gather a wide variety of health care professionals. Four studies focused on the demand for HRH, focusing on primary health care services. All the typologies of health care services (including at the tertiary level) were covered in nine studies. The remaining studies (n=11) were geared to HRH demand in health care services not classified according to the primary, secondary, or tertiary health care typology or are a combination of two services in this typology.

Approaching HRH Demand

A large amount of research, from various perspectives, has been made in this field: From a demand-based approach to needs-based ap proaches and benchmarking.

The demand side approaches, also known as expressed needs or utilization-based approaches44 Canvin RW, Boldy DP. Resource planning within the healthcare planning teams. Soc Sci Med 1976; 10:171-176., are commonly used to quantify the use of health care services by a population. HRH requirements are then calculated, adjusting the current amount of HRH delivered with the expected demographic changes in the target population. Hence, estimates translate neither the need nor the demand for HRH but solely preserve the status quo by assuming that satisfied demand is represented by (a) the current use of the health care services, (b) the current workforce-to-population ratio, and (c) the economic projections that relate social, political and economic factors to the consumption of health care services6363 Scheffler RM, Liu JX, Dal Poz MR. Forecasting the global shortage of physicians: An economic- and needs-based approach. Bulletin of the World Health Organization 2008; 86:516-523..

In the needs-based or epidemiological approaches, needs are expressed in terms of the required number of HRH that keep the population healthy. Typically, it involves epidemiological information (i.e., the observed health status of a population, through the use of incidence, prevalence, among other indicators) and evidence about adequate care delivery patterns (expressed in terms of HRH), which are commonly acquired through expert knowledge. HRH needs can be inferred from the gap between the observed and targeted health status. For instance, inspired by the work developed by the Graduate Medical Education National Advisory Committee in 19816464 McNutt DR. GMENAC: its manpower forecasting framework. Am J Public Health 1981;71(10):1116-1124., Lee et al.23 estimated the public health need for ophthalmologists by considering (a) the incidence and prevalence rates for 97 eye-related diseases and conditions, (b) the clinical population for each disease or condition, and (c) the population at risk. The projected incidence and prevalence rates were then applied to population projections to infer the number of affected individuals. Next, some adjustments were made to distinguish patients with different characteristics (e.g., patients suffering from multiple conditions) by consulting an advisory panel. Worktime requirements were attained by a survey of more than 2,000 ophthalmologists and converted to the required HRH1212 Song F, Rathwell T. Future demand for doctors in China. Health Policy Plan 1993; 8:394-404.. This study illustrated that a considerable amount of data is required to determine a specific medical specialty’s needs, which makes this approach impractical for evaluating a wide range of HRH needs. This is even more critical with limited epidemiological data, making the demand-based approach the easiest, least time-consuming, and least expensive proxy for needs.

Benchmarking relies on identifying regions or countries comparable - both at the demographic and health level, but which are strikingly different in terms of the costs and deployment of the HRH. The primary approach is to apply these benchmarks (recognized as a reference of good practices) to the general population or build plausible scenarios. This approach’s limitations include the definition of clear criteria for selecting the reference that can be best compared to the context under analysis.

This classification of approaches was proposed by Roberfroid et al.8 and is widely disseminated, enabling the understanding of the process. Nonetheless, demand is typically modeled by more than a single approach. In particular, Rizza et al.61 used three approaches to determine the demand for endocrinologists in the U.S. through 2020. Specifically, this study included scenarios in which demand was affected by (a) population growth; (b) utilization rates (e.g., patient visits), (c) changes in the incidence and prevalence of diabetes; and (d) benchmarks coupled with economic aspects6161 Rizza RA, Vigersky RA, Rodbard HW, Ladenson PW, Young WFJ, Surks MI, Kahn R, Hogan PF A model to determine workforce needs for endocrinologists in the United States until 2020. J Clin Endocrinol Metab 2003; 88(5):1979-1987..

Methods

HRH demand models have become progressively complicated and have adopted a system-wide perspective. The main dis tinctions are the premises underpinning the foundation of problem formulation and the available data (which is translated through the type of variables considered in the models); the number of methods used; how they are integrated; which type of software is used; and the inclusion of expert knowledge. Despite these efforts, there remains no established nor accepted approach to estimate the demand (or needs) for HRH adequately. This section presents the main characteristics of different methods and their advantages and disadvantages, which provides valuable insights for policymakers and the development of future research.

Most selected studies (n=28) used simulation-based methods to model HRH demand, while others used regression methods (n=17) and others relied on straightforward models (n=8) (see Table 1). A brief description of each methodological category is mentioned below.

Table 1
Distribution of the selected studies per methodology.

Straightforward models

In this category, demand is given by simple mathematical formulations relating key demand parameters3636 Schell GJ, Lavieri MS, Li X, Toriello A, Martyn KK. Strategic modelling of the pediatric nurse practitioner workforce. Pediatrics 2015; 135(2):298-306.,5050 Hoope-Bender P ten, Nove A, Sochas L, Mattews Z, Homer C, Pozo-Martin F. The 'Dream Team' for sexual, reproductive, maternal, newborn and adolescent health: an adjusted service target model to estimate the ideal mix of health care professionals to cover population need. Hum Resour Health 2017; 15(1):46.,5555 Segal L, Guy S, Leach M, Groves A, Turnbull C, Furber G. A needs-based workforce model to deiver tertiary-level community menatl health care for distressed infants, children, and adolescents in South Australia: a mixed-methods study. Lancet Public Heal 2018; 3(6):e296-303.,5959 Lopes MA, Almeida AS, Almada-Lobo B. Forecasting the medical workforce: a stochastic agent-based simulation approach. Health Care Manag Sci 2018; 21(1):52-75. or a combination of experts’ opinions with alternative scenarios that assume different work hypotheses1616 Withy K, Sakamoto D. Assessing physician workforce using insurance claims data and focus groups compared to benchmarks. Ethn Dis 2010; 20(Supl. 1):211-214.,4141 Robboy SJ, Gupta S, Crawford JM, Cohen MB, Karcher DS, Leonard DG, Magnani B, Novis DA, Prystowsky MB, Powell SZ, Gross DJ, Black-Schaffer WS. The pathologist workforce in the United States: II. An interactive modeling tool for analyzing future qualitative and quantitative staffing demands for services. Arch Pathol Lab Med 2015; 139(11):1413-1430.,4242 Nirel N, Grinstien-Cohen O, Eyal Y, Samuel H, Ben-Shoham A. Models for projecting supply and demand for nurses in Israel. Isr J Health Policy Res 2015; 4:1-12. (e.g., the steadiness of service utilization rates2323 Lee PP, Jackson CA, Relles DA. Estimating eye care workforce supply and requirements. Ophthalmology 1995; 102(12):1964-1972.). These models have a very narrow scope, whereas more general approaches, such as simulation-based methods or a combination of the latter with other tools, are more robust and can support more features of the demand for HRH. Although these straightforward methods might be the simplest and least costly technique to assess HRH demand, they might also be the least accurate. For these reasons, these methods’ results should be thoroughly analyzed since they are typically representative of a particular context.

Regression methods

Regression methods are widely known and, through simple means, offer an overall picture of what happened in the past, what is happening in the present, and what will happen in the future. The main advantage of these tools consists of identifying the relationships between relevant variables (e.g., the utilization rates of services and economic, demographic, social, and epidemiological indicators), which serves as a mechanism for better understanding the realities of the health system.

The first steps in regression analyses of HRH demand can be traced back to 1976, in which Pizam et al.1111 Pizam A, Neumann Y. Planning of medical manporwer. Long Range Plann 1976; 9:44-52. estimated the demand for doctors by using a linear regression method that related total requirements to socioeconomic variables (e.g., population growth and GNP per capita). Other regression approaches have been applied for this purpose, relating demand variables through linear regression methods1818 Zimbelman JL, Juraschek SP, Zhang X, Lin VW. Physical therapy workforce in the United States: forecasting nationwide shortages. PM R 2010; 2(11):1021-1029.,3939 Lin V, Zhang X, Dixon P. Occupational therapy workforce in the United States: Forecasting nationwide shortages. PM R 2015; 7(9):946-954.,6363 Scheffler RM, Liu JX, Dal Poz MR. Forecasting the global shortage of physicians: An economic- and needs-based approach. Bulletin of the World Health Organization 2008; 86:516-523., generalized linear models5454 Liu JX, Goryakin Y, Maeda A, Bruckner T, Scheffler R. Global health workforce labor market projections for 2030. Hum Resour Health 2017; 15:1-12., bivariate and linear analyses (through Ordinary Least-Squares)1515 Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrisey JP. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv 2009; 60(10):1323-1328.,4646 Jager R, Berg N van der, Hoffmann W, Jordan RA, Schwendicke F. Estimating future dental services' demand and supply: A model for Northern Germany. Community Dent Oral Epidemiol 2016; 44(2):169-179., multivariate regression models 2020 Hofer AN, Abraham JM, Moscovice I. Expansion of coverage under the patient protection and affordable care act and primary care utilization. Milbank Q 2011; 89(1):69-89., extrapolation2121 Kim JSC, Cooper RA, Kennedy DW. Otolaryngology-head and neck surgery physician work force issues: An analysis for future specialty planning. Otolaryngol Neck Surg 2012; 146(2):196-202.,2525 Hooker RS, Williams JH, Papneja J. Dietetics supply and demand: 2010-2020. J Acad Nutr Diet 2012; 112(3):S75-S91.,2828 Gallagher JE, Lim Z, Harper PR. Workforce skill mix: Modelling the potential for dental therapists in state-funded primary dental care. Int Dent J 2013; 63(2):57-64.,3434 Holliman CJ, Wuerz RC, Hirshberg AJ. Analysis of factors affecting U.S. emergency physician workforce projections. Acad Emerg Med 1997; 4(7):731-735.,6060 Schubert A, Eckhout G, Cooperider T, Kuhel A. Evidence of a current and lasting national anesthesia personnel shortfall: Scope and implications. Mayo Clin Proc 2001; 76(10):995-1010.,6161 Rizza RA, Vigersky RA, Rodbard HW, Ladenson PW, Young WFJ, Surks MI, Kahn R, Hogan PF A model to determine workforce needs for endocrinologists in the United States until 2020. J Clin Endocrinol Metab 2003; 88(5):1979-1987., autoregressive models (e.g., the Autoregressive Integrated Moving Average method)5151 Oh Y. The future prospects of supply and demand for urologists in Korea. Investig Clin Urol 2017; 58(6):400-408., and other statistical analyses5656 Angus DC, Kelley MA, Schmitz RJ, White A, Popovich J, Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS). Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease - Can we meet the requirements of an aging population? J Am Med Assoc 2000; 284:2762-2770.,5757 Cruz-Gomes S, Lopes AM, Almada-Lobo B. A labor requirements function for sizing the health workfoce. Hum Resour Health 2018; 16:1-12.. Typically, these methods do not manage sensitivity and robustness analyses per se, but in this study, sensitivity analyses were performed considering stochastic error distribution of variables, literature reviews, survey data, and expert opinion. Finally, it is essential to note that some regression methods require specialized software and specific expertise, and their results are employed to feed simulation-based models4444 Borracci RA, Milin E, Gelpi R. Long-term estimate of the number of doctors in Argentina. Argentine J Cardiol 2016; 84(1):25-30..

Simulation-based methods

Simulation-based models provide a general overview of the problem, representing the relationships between variables in a transparent learning process. These generic approaches can either integrate assumptions from a scalar level or more sophisticated mathematical formulations. Following the terminology proposed by Banks et al.6565 Banks J, Carson II JS. Discrete-Event System Simulation. In: Fabrycky WJ, Mize JH, editors. Prentice-Hall International Series in Industrial and Systems Engineering. New Jersey: Prentice-Hall, Inc. Englewood Cliffs; 1984., simulation models can be either dynamic, static and deterministic, or stochastic. The term “dynamic” refers to systems that evolve over time instead of a static simulation that represents a system at a particular point in time. Deterministic simulations receive a well-known set of inputs and return a unique set of outputs; a stochastic simulation model receives random variables as inputs and returns random outputs.

System dynamics (SD) has been one of the most popular methods used in the selected studies (n=14) and has gained momentum in the past decade1414 Tuulonen A, Salminen H, Linna M, Perkola M. The need and total cost of Finnish eyecare services: A simulation model for 2005-2040. Acta Ophthalmol 2009; 87:820-829.. Created in the 1950s, SD attempts to manage the behavior of complex systems analyzed over time using stocks, flows, loops, specific functions, and time delays through differential equations solved numerically with specific software1414 Tuulonen A, Salminen H, Linna M, Perkola M. The need and total cost of Finnish eyecare services: A simulation model for 2005-2040. Acta Ophthalmol 2009; 87:820-829. (e.g., Vensim or STELLA).

These studies typically include two populations (HRH and inhabitants) and relate them to crucial parameters by assuming working hypotheses. For instance, some studies have determined the demanded number of HRH as a target provider’s function per capita2626 Murphy GT, Birch S, MacKenzie A, Alder R, Lethbridge L, Little L. Eliminating the shortage of registered nurses in Canada: An exercise in applied needs-based planning. Health Policy (New York) 2012; 105(2-3):192-202.. Other studies have estimated the number of HRH, considering variations in population needs and delivery patterns over time. Regarding the latter, these patterns revolve around operational indicators - like the number of patients served with a given number of full-time equivalents (FTEs) or headcounts; or service utilization through the number of outpatient visits1313 Murphy GT, MacKenzie A, Alder R, Birch S, Kephart G, O'Brien-Pallas L. An applied simulation simulation model for estimating the supply of and requirements for registered nurses based on population health needs. Policy, Polit Nurs Pract 2009; 10:240-251.,1919 Vanderby SA, Carter MW, Latham T, Ouzounian M, Hassan A, Tang GH, Teng CJ, Kingsbury K, Feindel CM. Modeling the cardiac surgery workforce in Canada. Ann Thorac Surg 2010; 90(2):467-473.,2626 Murphy GT, Birch S, MacKenzie A, Alder R, Lethbridge L, Little L. Eliminating the shortage of registered nurses in Canada: An exercise in applied needs-based planning. Health Policy (New York) 2012; 105(2-3):192-202.,3232 Vanderby SA, Carter MW, Latham T, Feindel C. Modelling the future of the Canadian cardiac surgery workforce using system dynamics. J Oper Res Soc 2014; 65:1325-1335.,3535 Taba ST, Atkinson SR, Lewis S, Chung KSK, Hossain L. A systems life cycle approach to managing the radiology profession: an Australian perspective. Aust Heal Rev 2015;3 9(2):228-239.,4040 Ansah JP, Korne DD, Bayer S, Pan C, Jayabas kar T, Matchar DB, Lew N, Phua A, Koh V, Lamoureux E, Quek D. Future requirements for and supply of ophthalmologists for an aging population in Singapore. Hum Resour Health 2015; 13(1):1-13.. Also, some studies complemented the estimates with international studies and experts’ opinions1414 Tuulonen A, Salminen H, Linna M, Perkola M. The need and total cost of Finnish eyecare services: A simulation model for 2005-2040. Acta Ophthalmol 2009; 87:820-829.,3737 Brailsford S, Silva D. How many dentists does Sri Lanka need? Modelling to inform policy decisions. J Oper Res Soc 2015; 66:1566-1577.,5353 Ishikawa T, Fujiwara K, Ohba H, Suzuki T, Ogasawara K. Forecasting the regional distribution and sufficiency of physicians in Japan with a coupled system dynamics-geographic information system model. Hum Resour Health 2017; 15(1):1-9.. A combination of SD with other tools was also found within the selected studies. For instance, Ishikawa et al.5353 Ishikawa T, Fujiwara K, Ohba H, Suzuki T, Ogasawara K. Forecasting the regional distribution and sufficiency of physicians in Japan with a coupled system dynamics-geographic information system model. Hum Resour Health 2017; 15(1):1-9. made use of SD combined with GIS technologies to analyze the dynamic change and to measure regional disparities.

Microsimulation models have also been used to simulate micro-units’ demand-related behavior (person-level) over time2727 Dall TM, Storm MV, Chakrabarti R, Drogan O, Keran CM, Donofrio PD, Henderson VW, Kaminski HJ, Stevens JC, Vidic TR. Supply and demand analysis of the current and future US neurology workforce. Neurology 2013(5); 81:470-478.,3333 Edwards JP, Datta I, Hunt JD, Stefan K, Ball CG, Dixon E, Grondin SC. A novel approach for the accurate prediction of thoracic surgery workforce requirements in Canada. J Thorac Cardiovasc Surg 2014; 148(1):7-12.. Edwards et al.3333 Edwards JP, Datta I, Hunt JD, Stefan K, Ball CG, Dixon E, Grondin SC. A novel approach for the accurate prediction of thoracic surgery workforce requirements in Canada. J Thorac Cardiovasc Surg 2014; 148(1):7-12. predicted the thoracic surgery workforce requirements by fixing the published rate of (in)operable lung cancers as indicative for the overall demand. Some key parameters were not included, such as economic variables or disease complexity. Dall et al.2727 Dall TM, Storm MV, Chakrabarti R, Drogan O, Keran CM, Donofrio PD, Henderson VW, Kaminski HJ, Stevens JC, Vidic TR. Supply and demand analysis of the current and future US neurology workforce. Neurology 2013(5); 81:470-478. estimated the demand for neurologists by patients’ willingness and ability to pay for services, given patient needs and the cost of services.

Predictive equations included (a) the Poisson regression, which was used to quantify the relationship between patient characteristics and the number of consultations; and (b) the logistic regression used to determine the probability of emergency visits and hospitalizations for specific conditions, considering patient demographics and health characteristics as explanatory variables. Despite the methodological innovation in some of these methodological approaches, some authors argued that a lack of documentation limits knowledge transfer, which might lead to new modelers merely reinventing the wheel6666 Li J, O'Donoghue C. A survey of dynamic microsimulation models: uses, model structure and methodology. Int J Microsimulation 2013; 6(2):3-55..

Table 1 displays the distribution of the selected studies per methodology. Chart 1 summarizes the advantages and disadvantages of the typologies of methods considered in this review and highlights the benefits, challenges, and issues regarding HRH planning.

Chart 1
Pros and cons of selected methods.

Social component

Experts’ consultation is key for estimating HRH demand6767 Dussault G, Fronteira I. Human resources for health (HRH) plan component of national health plan 2011-15 (Portugal). Lisboa: Alto Comissariado da Saude; 2010., especially when it comes to accessing the staff standards that meet the population’s needs and defining plausible scenarios. Fifteen studies carried out this component through either focus groups, Delphi panels, or interviews1414 Tuulonen A, Salminen H, Linna M, Perkola M. The need and total cost of Finnish eyecare services: A simulation model for 2005-2040. Acta Ophthalmol 2009; 87:820-829.,1616 Withy K, Sakamoto D. Assessing physician workforce using insurance claims data and focus groups compared to benchmarks. Ethn Dis 2010; 20(Supl. 1):211-214.,1919 Vanderby SA, Carter MW, Latham T, Ouzounian M, Hassan A, Tang GH, Teng CJ, Kingsbury K, Feindel CM. Modeling the cardiac surgery workforce in Canada. Ann Thorac Surg 2010; 90(2):467-473.,2323 Lee PP, Jackson CA, Relles DA. Estimating eye care workforce supply and requirements. Ophthalmology 1995; 102(12):1964-1972.,2424 Stein ML, Rudge JW, Coker R, Weijden C van der, Krumkamp R, Hanvoravongchai P, Chavez I, Putthasri W, Phommasack B, Adisasmito W, Touch S, Sat le M, Hsu YC, Kretzschmar M, Timen A. Development of a resource modelling tool to support decision makers in pandemic influenza preparedness: The AsiaFluCap Simulator. BMC Public Health 2012; 12:1-14.,3232 Vanderby SA, Carter MW, Latham T, Feindel C. Modelling the future of the Canadian cardiac surgery workforce using system dynamics. J Oper Res Soc 2014; 65:1325-1335.,4141 Robboy SJ, Gupta S, Crawford JM, Cohen MB, Karcher DS, Leonard DG, Magnani B, Novis DA, Prystowsky MB, Powell SZ, Gross DJ, Black-Schaffer WS. The pathologist workforce in the United States: II. An interactive modeling tool for analyzing future qualitative and quantitative staffing demands for services. Arch Pathol Lab Med 2015; 139(11):1413-1430.,4242 Nirel N, Grinstien-Cohen O, Eyal Y, Samuel H, Ben-Shoham A. Models for projecting supply and demand for nurses in Israel. Isr J Health Policy Res 2015; 4:1-12.,4545 Lipscomb J, Parmigiani G, Hasselblad V. Combining expert judgement by hierarchical modelling: An application to physician staffing. Manage Sci 1998; 44(2):149-1461.,4949 Lodi A, Tubertini P, Grilli R, Mazzocchetti A, Ruozi C, Senese F. Needs forecast and fund allocation of medical specialty positions in Emilia-Romagna (Italy) by system dynamics and integer programming. Heal Syst 2016; 5(3):213-236.,5353 Ishikawa T, Fujiwara K, Ohba H, Suzuki T, Ogasawara K. Forecasting the regional distribution and sufficiency of physicians in Japan with a coupled system dynamics-geographic information system model. Hum Resour Health 2017; 15(1):1-9.,5555 Segal L, Guy S, Leach M, Groves A, Turnbull C, Furber G. A needs-based workforce model to deiver tertiary-level community menatl health care for distressed infants, children, and adolescents in South Australia: a mixed-methods study. Lancet Public Heal 2018; 3(6):e296-303.,5656 Angus DC, Kelley MA, Schmitz RJ, White A, Popovich J, Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS). Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease - Can we meet the requirements of an aging population? J Am Med Assoc 2000; 284:2762-2770.. Typically, experts were individuals with different backgrounds, such as HRH, academic researchers, or management personnel. Other studies do not specify and describe the underlying participatory technique3030 Harper P, Kleinman E, Gallagher J, Knight V. Cost-effective workforce planning: Optimising the dental team skill-mix for England. J Enterp Inf Manag 2013; 26(1/2):91-108.,3535 Taba ST, Atkinson SR, Lewis S, Chung KSK, Hossain L. A systems life cycle approach to managing the radiology profession: an Australian perspective. Aust Heal Rev 2015;3 9(2):228-239.,3737 Brailsford S, Silva D. How many dentists does Sri Lanka need? Modelling to inform policy decisions. J Oper Res Soc 2015; 66:1566-1577.,3838 Senese F, Tubertini P, Mazzocchetti A, Lodi A, Ruozi C, Grilli R. Forecasting future needs and optimal allocation of medical residency positions: The Emilia-Romagna region case study. Hum Resour Health 2015; 13(1):7.,4040 Ansah JP, Korne DD, Bayer S, Pan C, Jayabas kar T, Matchar DB, Lew N, Phua A, Koh V, Lamoureux E, Quek D. Future requirements for and supply of ophthalmologists for an aging population in Singapore. Hum Resour Health 2015; 13(1):1-13.,4848 Laurence CO, Karnon J. Improving the planning of the GP workforce in Australia: a simulation model incorporating work transitions, health need and service usage. Hum Resour Health 2016; 14:1-14.,5050 Hoope-Bender P ten, Nove A, Sochas L, Mattews Z, Homer C, Pozo-Martin F. The 'Dream Team' for sexual, reproductive, maternal, newborn and adolescent health: an adjusted service target model to estimate the ideal mix of health care professionals to cover population need. Hum Resour Health 2017; 15(1):46.,5252 Laurence CO, Karnon J. What will make a difference? Assessing the impact of policy and non-policy scenarios on estimations of the future GP workforce. Hum Resour Health 2017; 15(1):1-15.. An impressive amount of studies selected in this review did not include this component, thus revealing different understandings about the potential of experts’ knowledge or the limitations of time restrictions that undergirded the development of their participatory techniques.

Discussion

Barriers to the modeling of HRH demand

Estimating the demand for HRH is a crucial component in HRH planning, which affects population well-being. How many HRH does the population need? How should we model the demand for HRH? These are pertinent questions, but answering them is challenging due to the following elements:

  1. Lack of reliable and accurate data regarding the key drivers (from epidemiological information and costs to an exhaustive list of indicators for measuring utilization rates such as office visits), which is shared among all the selected studies. The existence of valid and updated data is paramount for us to understand where we are coming from, appraise the current situation, and develop an informed understanding of the future.

  2. Arbitrariness in the definition of the drivers representing current and future demand, especially given the lack of an accurate description of what demand is and how it should be measured. The previous section revealed studies following different perspectives, which lead to considering distinct data sets. We highly recommend the previous consultation of the work developed by O’Brien-Pallas et al.9 to avoid this non-uniformed pattern of research development and application, which provides a full list of data requirements for approaching demand within certain health professions. We believe this proposal is paramount for all HRH.

  3. Hard-to-predict changes in HRH demand, as these changes are affected by uncertain factors - namely deriving from changes in demography, patient complexity (e.g., the demand-related behavior of micro-units (person-level)2727 Dall TM, Storm MV, Chakrabarti R, Drogan O, Keran CM, Donofrio PD, Henderson VW, Kaminski HJ, Stevens JC, Vidic TR. Supply and demand analysis of the current and future US neurology workforce. Neurology 2013(5); 81:470-478.,3333 Edwards JP, Datta I, Hunt JD, Stefan K, Ball CG, Dixon E, Grondin SC. A novel approach for the accurate prediction of thoracic surgery workforce requirements in Canada. J Thorac Cardiovasc Surg 2014; 148(1):7-12.), technology, policy, and economy. There is currently no consensus among researchers on how to handle and predict the consequences of such changes. For instance:
    1. On the one hand, some authors have argued that new technology increases the number of treatable medical conditions and could reduce HRH demand; on the other hand, other studies advocate the opposite stance4242 Nirel N, Grinstien-Cohen O, Eyal Y, Samuel H, Ben-Shoham A. Models for projecting supply and demand for nurses in Israel. Isr J Health Policy Res 2015; 4:1-12.. Moreover, while these debates in the literature show an association between new technological devices and management, quality, and cost of care, only one of the selected studies have spent time discussing this topic4242 Nirel N, Grinstien-Cohen O, Eyal Y, Samuel H, Ben-Shoham A. Models for projecting supply and demand for nurses in Israel. Isr J Health Policy Res 2015; 4:1-12..

    2. Regarding the epidemiological profile of different populations, it was challenging to quantify adequate health services (and HRH) for a wide range of conditions, highlighted by the lack of consensus among researchers and experts on this topic. Some studies assume that older age groups consume more health care services and, consequently, invoke higher health expenditures as discussed by Lopes et al.5959 Lopes MA, Almeida AS, Almada-Lobo B. Forecasting the medical workforce: a stochastic agent-based simulation approach. Health Care Manag Sci 2018; 21(1):52-75.. In contrast, other authors disagree with this assumption and focus instead on the positive influence of preventive medicine, better care, and healthier lifestyles that promise to delay the onset of chronic diseases and disability within a given lifetime, while considering residual changes in life expectancy. Also, from this perspective, morbidity would remain during a shorter time in a life cycle4242 Nirel N, Grinstien-Cohen O, Eyal Y, Samuel H, Ben-Shoham A. Models for projecting supply and demand for nurses in Israel. Isr J Health Policy Res 2015; 4:1-12..

    3. Political changes with impact at the level of HRH required in each facility, either through reforms in the organization of service delivery or through modifications in HRH’s role, which are subject to change. For instance, the nurse-to-physician ratio in European countries is already changing, and, as such, political measures and international evidence points to an expansion of nurses’ role in these contexts6868 Dussault G. Plano Nacional de Saúde 2012-2016: Roteiro de intervenção recursos humanos em saúde (RHS). [document on the Internet]. 2014 [cited 2017 Dez .3]. Available from: https://pns.dgs.pt/roteiros-de-intervencao-do-plano-nacional-de-saude/
      https://pns.dgs.pt/roteiros-de-intervenc...

      69 Delamaire ML, Lafortune G. Nurses in advanced roles: a description and evaluation of experiences in 12 developed countries 2010. [document on the Internet]. 2010 [cited 2018 Jul 1]. Available from: https://www.oecd-ilibrary.org
      https://www.oecd-ilibrary.org...
      -7070 Buchan J, Seccombe I. Overstretched. Under-resourced. The UK nursing labour market review 2012. London: Royal College of Nursing (RCN); 2012..

  4. HRH demand models require monitoring and updating data throughout the years, which unavoidably involves specific expertise and costs that might not be feasible for policymakers.

Research needs

The relevance of modeling the demand for HRH is acknowledged in the literature, and this review intends to provide some guidance to policymakers, researchers, and those involved in improving HRH modeling. Several challenges emerged from this review and pointed to chal lenges that may need to be confronted to improve future research in this area:

  1. Enhance the databases’ quality and quantity regarding key drivers, as a requirement for any sound HRH demand modeling.

  2. Extend research to other HRH, namely to diagnosis and therapy technicians, and to psychologists, as most (90%) focused on a single health career.

  3. Adequately address HRH demand at a territorial level. Few studies covered this topic2424 Stein ML, Rudge JW, Coker R, Weijden C van der, Krumkamp R, Hanvoravongchai P, Chavez I, Putthasri W, Phommasack B, Adisasmito W, Touch S, Sat le M, Hsu YC, Kretzschmar M, Timen A. Development of a resource modelling tool to support decision makers in pandemic influenza preparedness: The AsiaFluCap Simulator. BMC Public Health 2012; 12:1-14.,4646 Jager R, Berg N van der, Hoffmann W, Jordan RA, Schwendicke F. Estimating future dental services' demand and supply: A model for Northern Germany. Community Dent Oral Epidemiol 2016; 44(2):169-179.,5353 Ishikawa T, Fujiwara K, Ohba H, Suzuki T, Ogasawara K. Forecasting the regional distribution and sufficiency of physicians in Japan with a coupled system dynamics-geographic information system model. Hum Resour Health 2017; 15(1):1-9., possibly due to the methods required to do so (which involve including geographic information systems that are not widely established in the health sector) and due to the lack of a generally accepted definition of adequate access4646 Jager R, Berg N van der, Hoffmann W, Jordan RA, Schwendicke F. Estimating future dental services' demand and supply: A model for Northern Germany. Community Dent Oral Epidemiol 2016; 44(2):169-179.. Hence, there is room for expanding methods to:
    • Characterize the future population’s needs regarding the various typologies of health services. Such tools depend on epidemiological data, which should be available at the territorial level;

    • Evaluate the territorial level equity of health care access and other public services (e.g., in schools). According to Ono et al.22 Ono T, Lafortune G, Schoenstein M. Health workforce planning in OECD countries: A review of 26 projection models from 18 countries. Paris: OECD Publishing; 2013. (OECD Health Working Papers, N. 62), it remains a policy issue in most OECD countries because it directly affects the ability to attract and retain HRH in underserved areas.

  4. Develop a compromise between simplicity and accuracy of the demand modeling, as simple methodsmay not provide reliable results but, at the same time, overly complicated methods requiring extensive implementation time or exaggerated expertise requirements run the risk of triggering rejection by policymakers, as stressed in Chart 1.

Limitations

Sound search procedures (i.e., PRISMA) were used to select papers for this systematic review. Nevertheless, some papers may have been overlooked in this process. The multiple designations and diverse terminology that abound within the field might have caused the authors to miss papers featuring keywords that are not typically used in existing terminologies. Moreover, the search was restricted to English-language studies and peer-reviewed published papers, which might have limited the search results”. However, we believe that inherent risks of our selection process are overshad owed by the large number of studies included in this review, which covers a broad range of HRH and provide a host of different perspectives and methods.

Finally, it is worth noting that the scientific quality of the studies was not appraised. However, we consider that this step was also overshadowed by another consideration - the definition and implementation of a rigorous list of exclusion criteria and a careful analysis of the methods applied in each paper.

Acknowledgments

This work was supported by the Science and Technology Foundation (FCT), Portuguese Ministry of Science and Higher Education (BD 133124/2017).

References

  • 1
    World Health Organization (WHO). Models and tools for health workforce planning and projections. Genebra: WHO; 2010.
  • 2
    Ono T, Lafortune G, Schoenstein M. Health workforce planning in OECD countries: A review of 26 projection models from 18 countries. Paris: OECD Publishing; 2013. (OECD Health Working Papers, N. 62)
  • 3
    Kroezen M, Hoegaerden M van, Batenburg R. The Joint Action on Health Workforce Planning and Forecasting: Results of a European programme to improve health workforce policies. Health Policy (New York) 2018; 122:87-93.
  • 4
    Canvin RW, Boldy DP. Resource planning within the healthcare planning teams. Soc Sci Med 1976; 10:171-176.
  • 5
    Lopes MA, Almeida ÁS, Almada-Lobo B. Handling healthcare workforce planning with care: where do we stand? Hum Resour Health 2015; 13:38.
  • 6
    Dussault G, Vujicic M. Demand and supply of human resources for health. Int Encycl Public Heal 2008; 77-84.
  • 7
    Leone C, Conceição C, Dussault G. Trends of cross-border mobility of physicians and nurses between Portugal and Spain. Hum Resour Health 2013;11:1-11.
  • 8
    Roberfroid D, Leonard C, Stordeur S. Physician supply forecast: better than peering in a crystal ball? Hum Resour Health 2009;7:1-13.
  • 9
    O'Brien-Pallas L, Baumann A, Donner G, Murphy GT, Lochhaas-Gerlach J, Luba M. Forecasting models for human resources in health care. J Adv Nurs 2001; 33:120-129.
  • 10
    Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med 2009; 151:264-269.
  • 11
    Pizam A, Neumann Y. Planning of medical manporwer. Long Range Plann 1976; 9:44-52.
  • 12
    Song F, Rathwell T. Future demand for doctors in China. Health Policy Plan 1993; 8:394-404.
  • 13
    Murphy GT, MacKenzie A, Alder R, Birch S, Kephart G, O'Brien-Pallas L. An applied simulation simulation model for estimating the supply of and requirements for registered nurses based on population health needs. Policy, Polit Nurs Pract 2009; 10:240-251.
  • 14
    Tuulonen A, Salminen H, Linna M, Perkola M. The need and total cost of Finnish eyecare services: A simulation model for 2005-2040. Acta Ophthalmol 2009; 87:820-829.
  • 15
    Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrisey JP. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv 2009; 60(10):1323-1328.
  • 16
    Withy K, Sakamoto D. Assessing physician workforce using insurance claims data and focus groups compared to benchmarks. Ethn Dis 2010; 20(Supl. 1):211-214.
  • 17
    Gallagher JE, Kleinman ER, Harper PR. Modelling workforce skill-mix: How can dental professionals meet the needs and demands of older people in England? Br Dent J 2010; 208(3):116-118.
  • 18
    Zimbelman JL, Juraschek SP, Zhang X, Lin VW. Physical therapy workforce in the United States: forecasting nationwide shortages. PM R 2010; 2(11):1021-1029.
  • 19
    Vanderby SA, Carter MW, Latham T, Ouzounian M, Hassan A, Tang GH, Teng CJ, Kingsbury K, Feindel CM. Modeling the cardiac surgery workforce in Canada. Ann Thorac Surg 2010; 90(2):467-473.
  • 20
    Hofer AN, Abraham JM, Moscovice I. Expansion of coverage under the patient protection and affordable care act and primary care utilization. Milbank Q 2011; 89(1):69-89.
  • 21
    Kim JSC, Cooper RA, Kennedy DW. Otolaryngology-head and neck surgery physician work force issues: An analysis for future specialty planning. Otolaryngol Neck Surg 2012; 146(2):196-202.
  • 22
    Yuji K, Imoto S, Yamaguchi R, Matsumura T, Murashige N, Kodama Y, Minayo S, Imai K, Kami M. Forecasting Japan's Physician Shortage in 2035 as the First Full-Fledged Aged Society. PLoS One 2012;7(11):e50410.
  • 23
    Lee PP, Jackson CA, Relles DA. Estimating eye care workforce supply and requirements. Ophthalmology 1995; 102(12):1964-1972.
  • 24
    Stein ML, Rudge JW, Coker R, Weijden C van der, Krumkamp R, Hanvoravongchai P, Chavez I, Putthasri W, Phommasack B, Adisasmito W, Touch S, Sat le M, Hsu YC, Kretzschmar M, Timen A. Development of a resource modelling tool to support decision makers in pandemic influenza preparedness: The AsiaFluCap Simulator. BMC Public Health 2012; 12:1-14.
  • 25
    Hooker RS, Williams JH, Papneja J. Dietetics supply and demand: 2010-2020. J Acad Nutr Diet 2012; 112(3):S75-S91.
  • 26
    Murphy GT, Birch S, MacKenzie A, Alder R, Lethbridge L, Little L. Eliminating the shortage of registered nurses in Canada: An exercise in applied needs-based planning. Health Policy (New York) 2012; 105(2-3):192-202.
  • 27
    Dall TM, Storm MV, Chakrabarti R, Drogan O, Keran CM, Donofrio PD, Henderson VW, Kaminski HJ, Stevens JC, Vidic TR. Supply and demand analysis of the current and future US neurology workforce. Neurology 2013(5); 81:470-478.
  • 28
    Gallagher JE, Lim Z, Harper PR. Workforce skill mix: Modelling the potential for dental therapists in state-funded primary dental care. Int Dent J 2013; 63(2):57-64.
  • 29
    Wu M-H, Yu J-Y, Huang C-H. Theoretical system dynamics modeling for Taiwan pediatric workforce in an era of national health insurance and low birth rates. Pediatr Neonatol 2013; 54(6):389-396.
  • 30
    Harper P, Kleinman E, Gallagher J, Knight V. Cost-effective workforce planning: Optimising the dental team skill-mix for England. J Enterp Inf Manag 2013; 26(1/2):91-108.
  • 31
    Murphy GT, MacKenzie A, Guy-Walker J, Walker C. Needs-based human resources for health planning in Jamaica: Using simulation modelling to inform policy options for pharmacists in the public sector. Hum Resour Health 2014; 12:1-11.
  • 32
    Vanderby SA, Carter MW, Latham T, Feindel C. Modelling the future of the Canadian cardiac surgery workforce using system dynamics. J Oper Res Soc 2014; 65:1325-1335.
  • 33
    Edwards JP, Datta I, Hunt JD, Stefan K, Ball CG, Dixon E, Grondin SC. A novel approach for the accurate prediction of thoracic surgery workforce requirements in Canada. J Thorac Cardiovasc Surg 2014; 148(1):7-12.
  • 34
    Holliman CJ, Wuerz RC, Hirshberg AJ. Analysis of factors affecting U.S. emergency physician workforce projections. Acad Emerg Med 1997; 4(7):731-735.
  • 35
    Taba ST, Atkinson SR, Lewis S, Chung KSK, Hossain L. A systems life cycle approach to managing the radiology profession: an Australian perspective. Aust Heal Rev 2015;3 9(2):228-239.
  • 36
    Schell GJ, Lavieri MS, Li X, Toriello A, Martyn KK. Strategic modelling of the pediatric nurse practitioner workforce. Pediatrics 2015; 135(2):298-306.
  • 37
    Brailsford S, Silva D. How many dentists does Sri Lanka need? Modelling to inform policy decisions. J Oper Res Soc 2015; 66:1566-1577.
  • 38
    Senese F, Tubertini P, Mazzocchetti A, Lodi A, Ruozi C, Grilli R. Forecasting future needs and optimal allocation of medical residency positions: The Emilia-Romagna region case study. Hum Resour Health 2015; 13(1):7.
  • 39
    Lin V, Zhang X, Dixon P. Occupational therapy workforce in the United States: Forecasting nationwide shortages. PM R 2015; 7(9):946-954.
  • 40
    Ansah JP, Korne DD, Bayer S, Pan C, Jayabas kar T, Matchar DB, Lew N, Phua A, Koh V, Lamoureux E, Quek D. Future requirements for and supply of ophthalmologists for an aging population in Singapore. Hum Resour Health 2015; 13(1):1-13.
  • 41
    Robboy SJ, Gupta S, Crawford JM, Cohen MB, Karcher DS, Leonard DG, Magnani B, Novis DA, Prystowsky MB, Powell SZ, Gross DJ, Black-Schaffer WS. The pathologist workforce in the United States: II. An interactive modeling tool for analyzing future qualitative and quantitative staffing demands for services. Arch Pathol Lab Med 2015; 139(11):1413-1430.
  • 42
    Nirel N, Grinstien-Cohen O, Eyal Y, Samuel H, Ben-Shoham A. Models for projecting supply and demand for nurses in Israel. Isr J Health Policy Res 2015; 4:1-12.
  • 43
    Hu W, Lavieri MS, Toriello A, Liu X. Strategic health workforce planning. IIE Trans 2016; 48(12):1127-1138.
  • 44
    Borracci RA, Milin E, Gelpi R. Long-term estimate of the number of doctors in Argentina. Argentine J Cardiol 2016; 84(1):25-30.
  • 45
    Lipscomb J, Parmigiani G, Hasselblad V. Combining expert judgement by hierarchical modelling: An application to physician staffing. Manage Sci 1998; 44(2):149-1461.
  • 46
    Jager R, Berg N van der, Hoffmann W, Jordan RA, Schwendicke F. Estimating future dental services' demand and supply: A model for Northern Germany. Community Dent Oral Epidemiol 2016; 44(2):169-179.
  • 47
    Murphy GT, Birch S. Simulating future supply of and requirements for human resources for health in high-income OECD countries. Hum Resour Health 2016; 14(1):1-18.
  • 48
    Laurence CO, Karnon J. Improving the planning of the GP workforce in Australia: a simulation model incorporating work transitions, health need and service usage. Hum Resour Health 2016; 14:1-14.
  • 49
    Lodi A, Tubertini P, Grilli R, Mazzocchetti A, Ruozi C, Senese F. Needs forecast and fund allocation of medical specialty positions in Emilia-Romagna (Italy) by system dynamics and integer programming. Heal Syst 2016; 5(3):213-236.
  • 50
    Hoope-Bender P ten, Nove A, Sochas L, Mattews Z, Homer C, Pozo-Martin F. The 'Dream Team' for sexual, reproductive, maternal, newborn and adolescent health: an adjusted service target model to estimate the ideal mix of health care professionals to cover population need. Hum Resour Health 2017; 15(1):46.
  • 51
    Oh Y. The future prospects of supply and demand for urologists in Korea. Investig Clin Urol 2017; 58(6):400-408.
  • 52
    Laurence CO, Karnon J. What will make a difference? Assessing the impact of policy and non-policy scenarios on estimations of the future GP workforce. Hum Resour Health 2017; 15(1):1-15.
  • 53
    Ishikawa T, Fujiwara K, Ohba H, Suzuki T, Ogasawara K. Forecasting the regional distribution and sufficiency of physicians in Japan with a coupled system dynamics-geographic information system model. Hum Resour Health 2017; 15(1):1-9.
  • 54
    Liu JX, Goryakin Y, Maeda A, Bruckner T, Scheffler R. Global health workforce labor market projections for 2030. Hum Resour Health 2017; 15:1-12.
  • 55
    Segal L, Guy S, Leach M, Groves A, Turnbull C, Furber G. A needs-based workforce model to deiver tertiary-level community menatl health care for distressed infants, children, and adolescents in South Australia: a mixed-methods study. Lancet Public Heal 2018; 3(6):e296-303.
  • 56
    Angus DC, Kelley MA, Schmitz RJ, White A, Popovich J, Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS). Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease - Can we meet the requirements of an aging population? J Am Med Assoc 2000; 284:2762-2770.
  • 57
    Cruz-Gomes S, Lopes AM, Almada-Lobo B. A labor requirements function for sizing the health workfoce. Hum Resour Health 2018; 16:1-12.
  • 58
    Laurence CO, Heywood T, Bell J, Atkinson K, Karnon J. The never ending road: improving, adapting and refining a needs-based model to estimate future general practitioner requirements in two Australian states. Fam Pract 2018; 35:193-198.
  • 59
    Lopes MA, Almeida AS, Almada-Lobo B. Forecasting the medical workforce: a stochastic agent-based simulation approach. Health Care Manag Sci 2018; 21(1):52-75.
  • 60
    Schubert A, Eckhout G, Cooperider T, Kuhel A. Evidence of a current and lasting national anesthesia personnel shortfall: Scope and implications. Mayo Clin Proc 2001; 76(10):995-1010.
  • 61
    Rizza RA, Vigersky RA, Rodbard HW, Ladenson PW, Young WFJ, Surks MI, Kahn R, Hogan PF A model to determine workforce needs for endocrinologists in the United States until 2020. J Clin Endocrinol Metab 2003; 88(5):1979-1987.
  • 62
    Vijan S, Inadomi J, Hayward RA, Hofer TP, Fendrick AM. Projections of demand and capacity for colonoscopy related to increasing rates of colorectal cancer screening in the United States. Aliment Phamacology Ther 2004; 20(5):507-5015.
  • 63
    Scheffler RM, Liu JX, Dal Poz MR. Forecasting the global shortage of physicians: An economic- and needs-based approach. Bulletin of the World Health Organization 2008; 86:516-523.
  • 64
    McNutt DR. GMENAC: its manpower forecasting framework. Am J Public Health 1981;71(10):1116-1124.
  • 65
    Banks J, Carson II JS. Discrete-Event System Simulation. In: Fabrycky WJ, Mize JH, editors. Prentice-Hall International Series in Industrial and Systems Engineering. New Jersey: Prentice-Hall, Inc. Englewood Cliffs; 1984.
  • 66
    Li J, O'Donoghue C. A survey of dynamic microsimulation models: uses, model structure and methodology. Int J Microsimulation 2013; 6(2):3-55.
  • 67
    Dussault G, Fronteira I. Human resources for health (HRH) plan component of national health plan 2011-15 (Portugal). Lisboa: Alto Comissariado da Saude; 2010.
  • 68
    Dussault G. Plano Nacional de Saúde 2012-2016: Roteiro de intervenção recursos humanos em saúde (RHS). [document on the Internet]. 2014 [cited 2017 Dez .3]. Available from: https://pns.dgs.pt/roteiros-de-intervencao-do-plano-nacional-de-saude/
    » https://pns.dgs.pt/roteiros-de-intervencao-do-plano-nacional-de-saude
  • 69
    Delamaire ML, Lafortune G. Nurses in advanced roles: a description and evaluation of experiences in 12 developed countries 2010. [document on the Internet]. 2010 [cited 2018 Jul 1]. Available from: https://www.oecd-ilibrary.org
    » https://www.oecd-ilibrary.org
  • 70
    Buchan J, Seccombe I. Overstretched. Under-resourced. The UK nursing labour market review 2012. London: Royal College of Nursing (RCN); 2012.

Appendix A. Search terms utilised in the review.

Appendix B. Spatial distribution of the selected studies per country of affiliation.

Appendix C. Journals according to their context.

Appendix D. Twenty most cited papers among the 53 selected studies.

Appendix E. Characteristics of the selected studies.

Publication Dates

  • Publication in this collection
    14 June 2021
  • Date of issue
    2021

History

  • Received
    17 Mar 2020
  • Accepted
    02 Aug 2020
  • Published
    04 Aug 2020
ABRASCO - Associação Brasileira de Saúde Coletiva Rio de Janeiro - RJ - Brazil
E-mail: revscol@fiocruz.br