ABSTRACT

Objective.

To create and validate criteria for prioritizing problems related to policies and management of the health workforce.

Methods.

This methodological study was divided into three stages. First, the criteria were elaborated by means of a systematized literature review. Second, the criteria were evaluated online by a committee of judges comprised of eight specialists. In the third stage, an evaluation was carried out by the target audience in a hybrid workshop. The participants evaluated the material using the Suitability Assessment of Materials instrument, adapted for the research.

Results.

Three prioritization criteria (relevance, window of opportunity and acceptability) and a scoring scale were developed based on the literature review. In the evaluation by the committee of judges, the approval percentage of the criteria and prioritization method was 84%. Modifications were made based on suggestions in relation to the material presented to the specialists. In the pre-test stage, the approval percentage varied by item, with six of them reaching a maximum approval of 100% (corresponding to approximately 46% of the items), four reaching 92% and three achieving 83% each, indicating positive results.

Conclusions.

The developed criteria were considered valid for use in the context of policies and management in the area of human resources for health.

Keywords
Health workforce; health management; validation study; health priority agenda; health policy

RESUMEN

Objetivo.

Crear y validar criterios para priorizar los problemas relacionados con las políticas y la gestión de los recursos humanos para la salud.

Métodos.

Este estudio metodológico se dividió en tres etapas. En la primera se elaboraron los criterios mediante una revisión sistematizada de la bibliografía. En la segunda un comité de ocho especialistas evaluó en línea los criterios. Y la tercera consistió en una evaluación por parte del público destinatario en un taller híbrido. Los participantes evaluaron el material utilizando el instrumento de evaluación de la idoneidad de los materiales, que fue adaptado para la investigación.

Resultados.

Sobre la base de la revisión de la bibliografía, se elaboraron tres criterios para la asignación de prioridades (relevancia, ventana de oportunidad y aceptabilidad) y una escala de puntuación. En la evaluación realizada por el comité de especialistas, el porcentaje de aprobación de los criterios y del método de asignación de prioridades fue del 84%. Se realizaron modificaciones basadas en sugerencias planteadas con respecto al material presentado a los especialistas. En la etapa posterior de prueba preliminar, el porcentaje de aprobación varió en los distintos puntos, de tal manera que en seis puntos (es decir, en aproximadamente el 46% de los puntos) se alcanzó una aprobación máxima del 100%, en cuatro una aprobación del 92% y en tres una aprobación del 83% en cada uno, lo que indica unos resultados positivos.

Conclusiones.

Se consideró que los criterios elaborados son válidos para su uso en el contexto de las políticas y la gestión en el ámbito de los recursos humanos para la salud.

Palabras clave
Fuerza laboral en salud; gestión de la información en salud; estudio de validación; agenda de prioridades en salud; política de salud

RESUMO

Objetivo.

Criar e validar critérios para priorizar problemas relacionados a políticas e gerenciamento da força de trabalho em saúde.

Métodos.

O presente estudo metodológico foi dividido em três fases. Primeiro, foram elaborados critérios por meio de revisão sistematizada da literatura. A seguir, os critérios foram avaliados on-line por uma comissão de juízes composta por oito especialistas. Na terceira fase, o público-alvo fez uma avaliação dos critérios em uma oficina de formato híbrido. Os participantes avaliaram o material usando o instrumento Suitability Assessment of Materials, adaptado para esta pesquisa.

Resultados.

Com base na revisão da literatura, foram elaborados três critérios de priorização (relevância, janela de oportunidade e aceitabilidade) e uma escala de pontuação. Na avaliação da comissão de juízes, a porcentagem de aprovação dos critérios e do método de priorização foi de 84%. Foram feitas alterações com base em sugestões relacionadas ao material apresentado aos especialistas. Na fase de pré-teste, a porcentagem de aprovação variou de acordo com o item. Seis deles (aproximadamente 46% dos itens) atingiram aprovação máxima de 100%, quatro atingiram 92% e três atingiram 83%, indicando resultados positivos.

Conclusões.

Os critérios desenvolvidos foram considerados válidos para uso no contexto de políticas e gerenciamento na área de recursos humanos em saúde.

Palavras-chave
Mão de obra em saúde; gestão da informação em saúde; estudo de validação; agenda de prioridades em saúde; política de saúde

An adequate workforce in terms of staffing, training, regulation, distribution and management is imperative to achieve health systems and services (HSS) effectiveness and meet the population's health objectives. It is also fundamental for strengthening primary health care, a key element in achieving universal coverage, health access and, consequently, the Sustainable Development Goals (SDGs) (¹1. World Health Organization. State of the world’s nursing 2020: investing in education, jobs and leadership [Internet]. Geneva: World Health Organization; 2020. Available from: https://iris.who.int/handle/10665/331677
https://iris.who.int/handle/10665/331677... ). From this perspective, both effective and evidence-informed health workforce planning and policy-making are decisive for building and consolidating HSS capable of meeting social health needs (²2. Portela GZ, Fehn AC, Ungerer RLS, Poz MRD. Recursos humanos em saúde: crise global e cooperação internacional. Cien Saude Colet. 2017;22(7):2237–46. https://doi.org/10.1590/1413-81232017227.02702017
https://doi.org/10.1590/1413-81232017227... ); also, it is considered a commitment to human resources for health (HRH) action plans of multilateral agencies (³3. World Health Organization. Global strategy on human resources for health: Workforce 2030 [Internet]. Geneva: World Health Organization; 2016. Available from: https://iris.who.int/handle/10665/250368
https://iris.who.int/handle/10665/250368... ,⁴4. Pan American Health Organization. Strategy on Human Resources for Universal Access to Health and Universal Health Coverage. 29th Pan American Sanitary Conference 69th Session of the Regional Committee of WHO for the Americas. Washington, D.C., USA, 25-29 September 2017 [Internet]. Washington, D.C.: PAHO; 2017. Available from: https://iris.paho.org/handle/10665.2/34198
https://iris.paho.org/handle/10665.2/341... ). This relevance has been recognized in view of the challenges faced worldwide during the COVID-19 pandemic.

Among the main challenges faced in the health planning area, the mobilization of political will and financial resources are critical components to HRH, in the long term (³3. World Health Organization. Global strategy on human resources for health: Workforce 2030 [Internet]. Geneva: World Health Organization; 2016. Available from: https://iris.who.int/handle/10665/250368
https://iris.who.int/handle/10665/250368... ). The development of adequate and quality HRH requires a critical analysis of the situation in each country and the understanding of the employment and political will dynamic, enabling the exploration of alternatives regarding the composition and competencies of appropriate HRH and aligned with the care model (⁴4. Pan American Health Organization. Strategy on Human Resources for Universal Access to Health and Universal Health Coverage. 29th Pan American Sanitary Conference 69th Session of the Regional Committee of WHO for the Americas. Washington, D.C., USA, 25-29 September 2017 [Internet]. Washington, D.C.: PAHO; 2017. Available from: https://iris.paho.org/handle/10665.2/34198
https://iris.paho.org/handle/10665.2/341... ).

Therefore, based on the definition of priorities, decision-making is inherent to policymaking and management of public policies including HSS. The literature on priorities setting describes their use in different areas of care and HSS management and identifies a series of methodologies, principles and approaches that can be used. Setting priorities involves judging between different dimensions that require understanding and application of specific values. However, in a plural society, full agreement on which decisions to make or which outcomes are preferable is rarely reached (⁵5. Ministério da Saúde (Brasil). Priorização de Políticas e Programas Sob Responsabilidade Direta da Secretaria de Atenção à Saúde [Internet]. Ministério da Saúde, Secretaria de Atenção à Saúde Coordenação-Geral de Informação e Monitoramento de Serviços e Redes de Atenção à Saúde. Brasília: Ministério da Saúde; 2019. Available from: https://abrasco.org.br/wp-content/uploads/2021/04/livreto_priorizacaoSAS_ALT6-2.pdf
https://abrasco.org.br/wp-content/upload... ).

Thus, among the potential results of a priority-setting process are harmonization of competing interests and values, conflict resolution, and collective construction of consensus and a set of priorities. However, this process is sometimes controversial and requires using negotiation measures, as opposed to technical and systematic approaches, not being derived from an evidence-based decision-making process (⁶6. Ranson MK, Bennett SC. Priority setting and health policy and systems research. Heal Res Policy Syst. 2009;7(1):27. https://doi.org/10.1186/1478-4505-7-27
https://doi.org/10.1186/1478-4505-7-27... ).

Thereby, an ethical structure is required that emphasizes the process by which priorities are defined as part of a public policy´s design cycle, to the detriment of simply outlining rules and results. The definition of criteria to be used is among the core elements of the process, which may include: benefit to the health system, individuals and economy; evidence; costs; efficiency; equity; equality; disease severity; disease prevalence; solidarity; and protection of the vulnerable (⁷7. Tomlinson M, Chopra M, Hoosain N, Rudan I. A review of selected research priority setting processes at national level in low and middle income countries: towards fair and legitimate priority setting. Heal Res Policy Syst. 2011;9(1):19. https://doi.org/10.1186/1478-4505-9-19
https://doi.org/10.1186/1478-4505-9-19... ). However, in some contexts, these prioritization criteria are sometimes not established or in force or, when they occur, they are typically disease-oriented, not including a broader and integrated view of health systems and the complexity related to HRH due to labor market dynamics in the health sector (⁶6. Ranson MK, Bennett SC. Priority setting and health policy and systems research. Heal Res Policy Syst. 2009;7(1):27. https://doi.org/10.1186/1478-4505-7-27
https://doi.org/10.1186/1478-4505-7-27... ,⁸8. World Health Organization. Health labour market analysis guidebook [Internet]. Geneva: World Health Organization; 2021. Available from: https://iris.who.int/handle/10665/348069
https://iris.who.int/handle/10665/348069... ). Governments usually develop guidelines for the formulation or evaluation of public policies (⁹9. Secretaria de Gestión y Empleo Público Argentina. Guía de planificación y seguimiento de gestión de políticas públicas [Internet]. Buenos Aires; Jefatura de Gabinete de Ministros; 2020. Available from: https://www.argentina.gob.ar/sites/default/files/final_guia_de_planificacion_y_seguimiento_de_gestion_de_politicas_publicas.pdf
https://www.argentina.gob.ar/sites/defau... ). Its implementation requires the collaboration of many stakeholders, partners, and qualified individuals. Despite following all the necessary steps, success is not always guaranteed (¹⁰10. Ahmat A, Okoroafor SC, Asamani JA, Dovlo D, Millogo JJS, Abdou Illou MM, et al. Health workforce policy and plan implementation in the context of universal health coverage in the Africa Region. BMJ Glob Heal. 2022;7(Suppl 1):e008319. https://doi.org/10.1136/bmjgh-2021-008319%0A
https://doi.org/10.1136/bmjgh-2021-00831... ). Considering the relevance of criteria for the decision-making context in the HRH area, the current study aimed at developing and validating criteria to prioritize problems related to policies and management in the health workforce context. First, we present the stages designed for developing, evaluating, and validating criteria for prioritizing issues related to policies and management in the health workforce context, the instruments used to collect the information and the data treatment and analysis. This is followed by the presentation of the results of the application of the steps, and the criteria and scale proposed for prioritization. Finally, we discuss the process and importance of developing the criteria and scale for prioritization in HRH policy and management.

This study is part of a larger project carried out within the scope of the Working Group on the State of Nursing in Brazil^**Among its members are the Brazilian Association of Nursing (ABEn), Brazilian Association of Midwives and Obstetric Nurses (Abenfo Nacional), Federal Council of Nursing (Cofen), Ministry of Education, Ministry of Health, Collaborating Center of the Pan American Health Organization PAHO/WHO for the Development of Nursing Research at EERP/USP, Working Group of the Nursing Now Campaign in Brazil, United Nations Population Fund (UNFPA) and the PAHO/WHO representation in Brazil. (¹¹11. de Oliveira APC, Ventura CAA, da Silva FV, Angotti Neto H, Mendes IAC, de Souza KV, et al. State of Nursing in Brazil. Rev Lat Am Enfermagem. 2020;28. https://doi.org/10.1590/1518-8345.0000.3404
https://doi.org/10.1590/1518-8345.0000.3... ) entitled “Challenges and coping options to improve Nursing professionals' availability and accessibility in Brazil: Evidence to timely inform policies”, which had the objective of establishing an environment conducive to dialogue and exchange between the main actors in planning and regulation of the Nursing Workforce in Brazil, with a view to supporting planning and formulation of policies based on the best available evidence. The criteria developed and validated, presented in this article, were used to establish a priority agenda based on challenges and options to address identified through a multi-method study that were considered and prioritized by the actors in a Prioritization Workshop.

METHODS

This article is a methodological study focused on the development, evaluation and improvement of an instrument or strategy (¹²12. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Heal Res Policy Syst. 2010;8(1):36. Available from: https://doi.org/10.1186/1478-4505-8-36
https://doi.org/10.1186/1478-4505-8-36... ). The methodological proposal for validation of the material was adapted based on the Pasquali model (¹³13. Pasquali L. Princípios de elaboração de escalas psicológicas. Rev Psiquiatr Clínica. 1998;25(5):206–13.). The stages carried out in the study are described in Figure 1.

In the first stage—Elaboration—a systematized literature review was carried out; this type of review provides a structured and organized overview of a topic (¹⁴14. Sataloff RT, Bush ML, Chandra R, Chepeha D, Rotenberg B, Fisher EW, et al. Systematic and other reviews: Criteria and complexities. World J Otorhinolaryngol - Head Neck Surg. 2021;7(3):236–9. https://doi.org/10.1016/j.wjorl.2021.04.007
https://doi.org/10.1016/j.wjorl.2021.04.... ). The review sought to identify specific criteria and the prioritization method used in the evaluation and management of public policies. It included an analysis of technical and political documents and scientific literature. Table 1 presents the electronic search that took place in December 2021.

Two independent reviewers performed the data extraction after reading the articles in full and applying the inclusion and exclusion criteria, using an extraction document in table format, in Excel (version 2016, Microsoft). The topic and sub-topic and diverse core information on the criteria and prioritization methods used were extracted. In the absence of a prioritization tool, the documents were used to identify which components should be included in the development of criteria and methods to develop the first proposal. After mapping and analyzing the literature found, the criteria and scoring structure to be employed in the process of prioritizing problems related to policies and management in the health workforce context were elaborated.

In the second stage—Committee of judges—from six to ten specialists is recommended for this type of validation (¹⁵15. Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? critique and recommendations. Res Nurs Health. 2006;29(5):489–97. https://doi.org/10.1002/nur.20147
https://doi.org/10.1002/nur.20147... ). To identify the experts, we started with a list of well-known names in the field (brainstorming) and followed by using snowball sampling. For both methods of identification of specialists, we used the following analysis parameters to select them: degree, scientific production, comprehension of Portuguese, and experience time in the areas of HRH policies and management and technical cooperation in health. A total of eight national and international specialists on the themes of this study were invited; all agreed to participate and received via email the final version of the criteria and methodology for prioritization built from the synthesis of evidence, for them to read, analyze and evaluate the material available on the Google Forms platform using the Suitability Assessment of Materials (SAM) scale (¹⁶16. Sousa CS, Turrini RNT, Poveda VB. Translation and adaptation of the instrument “Suitability Assessment of Materials” (SAM) into portuguese. J Nurs UFPE line. 2015;9(5):7854–61. Available from: https://repositorio.usp.br/directbitstream/3ab8a7a7-5e39-4ed5-8103-dfdfc428576f/TURRINI, R N T doc 110e.p
https://repositorio.usp.br/directbitstre... ). This was adapted for this study due to the inclusion of some elements that did not apply to the study context.

After the analysis and due to the low scores on some items (Table 2), the specialists were invited to make suggestions for changes. These suggestions were grouped, analyzed and accepted, resulting in a new version of the material.

The third stage—Pre-test—was structured as a hybrid workshop where 12 participants that met the criteria to be considered target audience were invited (managers and technicians in the areas of policies, regulation and HRH management); all individuals agreed to participate in this phase. The invitation was sent to members of the State of Nursing in Brazil Working Group, accompanied by a concept note (indicating the topic under discussion and the context of the work, among others) and the material to be read and discussed (challenges to the availability and accessibility of nursing professionals in Brazil identified through a multi-method study to be prioritized). The participants received the final version of the criteria and prioritization methodology for them to read, analyze and evaluate the material after applying it to the challenges discussed in the workshop, using the adapted SAM instrument.

Instruments used to collect the information

To determine the participants' agreement in relation to the analysis items, an adaptation of the SAM instrument was used, which consists of a list or checklist with six categories (content, text comprehension, illustration, presentation, motivation and cultural adaptation). The result of adding up the points assigned to each item of the instrument categorizes the material as to its adequacy for application (¹³13. Pasquali L. Princípios de elaboração de escalas psicológicas. Rev Psiquiatr Clínica. 1998;25(5):206–13.). The instrument was adapted, presenting three categories (content, language and presentation), not using the graphic illustrations, stimulation/motivation and cultural adequacy categories, as shown in Table 2.

To score the items, a five-point Likert scale (ranging from 0, strongly disagree to 4, strongly agree) was used. The Likert scale has shown sufficient advantages for measuring attitudes and is widely used for performance evaluation in variety of domains (¹⁷17. Kriksciuniene D, Sakalauskas V, Lewandowski R. Evaluating the Interdependent Effect for Likert Scale Items. In: Abramowicz W, Corchuelo R. (eds) Business Information Systems Workshops. BIS 2019. Lecture Notes in Business Information Processing, vol 373. Springer, Cham. Available from: https://doi.org/10.1007/978-3-030-36691-9_3
https://doi.org/10.1007/978-3-030-36691-... ). The assessment instrument was organized in the Google Forms app, and the results were extracted to a spreadsheet in Microsoft Excel.

Data treatment and analysis

To analyze the data from the assessment performed via the SAM instrument, the mean agreement between the participants was calculated using the Content Validity Index (CVI), a measure based on representativeness of the positive answers through several methods. We calculated the CVI for each item by counting the number of experts who rated each item with three or four, and dividing that number by the total number of experts, which resulted in the proportion who considered the item as valid content (¹⁸18. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: Conducting a content validity study in social work research. Soc Work Res. 2003;27(2):94–104. https://doi.org/10.1093/swr/27.2.94
https://doi.org/10.1093/swr/27.2.94... ). Items marked as “Agree” or “Totally agree” were considered representative, obtaining an index score equal to 1.00, and items with all evaluations equal to 1.00 had 100% agreement (¹⁹19. Aragão J da S, França ISX de, Coura AS, Sousa FS de, Batista JDL, Magalhães IM de O. A content validity study of signs, symptoms and diseases/health problems expressed in LIBRAS. Rev Lat Am Enfermagem. 2015;23(6):1014–23. https://doi.org/10.1590/0104-1169.0325.2644
https://doi.org/10.1590/0104-1169.0325.2... ).

To perform this calculation, the answers to the items were regrouped and the values that corresponded to 0 and 1 on the Likert scale (Totally disagree and Disagree) were changed to (–1); when the value was 2 (Neutral/Indifferent) it became (0); and when the values were 3 and 4, they became (+1). Thus, each expert's answer could vary from –1 to +1 and the closer to +1, the higher the agreement that the item was pertinent. Based on these answers, it was possible to calculate the means of agreement by the committee of judges, calculating the percentage of answers that corresponded to the +1 value (²⁰20. Oliveira MS de, Fernandes AFC, Sawada NO. Manual educativo para o autocuidado da mulher mastectomizada: um estudo de validação. Texto Context - Enferm. 2008;17(1):115–23. https://doi.org/10.1590/S0104-07072008000100013
https://doi.org/10.1590/S0104-0707200800... ). For an item to be approved, at least 80% agreement between the experts was required (¹³13. Pasquali L. Princípios de elaboração de escalas psicológicas. Rev Psiquiatr Clínica. 1998;25(5):206–13.).

According to the literature, the assessment of the instrument in its entirety does not require consensus. Furthermore, there are three ways to assess the instrument, such as: the mean of the proportions corresponding to the items considered relevant by the experts; the mean of the values of the items calculated separately, that is, all the separately calculated CVIs are added up and divided by the number of items considered in the evaluation; and to divide the “total number of items considered relevant by the specialists by the total number of items”. In the case of six or more items, an index not lower than 0.78 is recommended (¹⁵15. Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? critique and recommendations. Res Nurs Health. 2006;29(5):489–97. https://doi.org/10.1002/nur.20147
https://doi.org/10.1002/nur.20147... ). Thus, authors agreed to divide the total number of items considered relevant (with answer 1, Agree or Strongly agree) by the total number of items answered in the instrument.

The SAM tool allowed the participants to issue opinions and suggestions on each item. Consequently, the experts' suggestions were grouped and analyzed, and plausible modifications were made.

The study was approved by the Research Ethics Committee of the Ribeirao Preto Nursing School at the University of São Paulo (CAAE: 21828819.4.0000.5393).

RESULTS

The search during the systematized review yielded 110 documents (84 from PubMed and 26 from other sources), none of which referred to criteria applied within the scope of management or policy formulation in HRH, with 10 documents that included other topics in the HRH area. The documents were organized into the following areas: attribute of indicators for monitoring the Regional HRH strategy (²¹21. Pérez PN. Taller para el Desarrollo de la Metodología de Seguimiento del Plan de Acción de la Estrategia de Recursos Humanos para el Acceso y Cobertura Universal de Salud: Informe de Resultados [Internet]. Organización Panamericana de la Salud; 2018. Available from: https://www.observatoriorh.org/sites/default/files/webfiles/fulltext/2018/taller_trt/informe_t%26t_2018.pdf
https://www.observatoriorh.org/sites/def... ); HRH research issues (⁶6. Ranson MK, Bennett SC. Priority setting and health policy and systems research. Heal Res Policy Syst. 2009;7(1):27. https://doi.org/10.1186/1478-4505-7-27
https://doi.org/10.1186/1478-4505-7-27... ,²²22. Ranson K, Law TJ, Bennett S. Establishing health systems financing research priorities in developing countries using a participatory methodology. Soc Sci Med. 2010;70(12):1933–42. https://doi.org/10.1016/j.socscimed.2010.01.051
https://doi.org/10.1016/j.socscimed.2010... ); research issues on HSS, including HRH (²³23. Uneke CJ, Ezeoha AE, Ndukwe CD, Oyibo PG, Onwe F, Aulakh BK. Research priority setting for health policy and health systems strengthening in Nigeria: the policymakers’ and stakeholders’ perspective and involvement. Pan Afr Med J. 2013;16. https://doi.org/10.11604/pamj.2013.16.10.2318
https://doi.org/10.11604/pamj.2013.16.10... ); national agenda of health research priorities, including HRH (²⁴24. Ministério da Saúde. Secretaria de Ciência Tecnologia e Insumos Estratégicos (Brasil). Agenda de Prioridades de Pesquisa do Ministério da Saúde [Internet]. [Internet]. Brasilia: Ministério da Saúde; 2015. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/agenda_nacional_prioridades_2ed_4imp.pdf
https://bvsms.saude.gov.br/bvs/publicaco... –²⁶26. Bassalobre Garcia A, Cassiani SHB, Reveiz L. A systematic review of nursing research priorities on health system and services in the Americas. Rev Panam Salud Publica. 2015;37(3):162–71. Available from: https://iris.paho.org/handle/10665.2/9215
https://iris.paho.org/handle/10665.2/921... ); health technology assessment, prioritization process and topics for assessment, and recommendations on which technologies to fund in the public health sector (²⁷27. Glassman A, Chalkidou K. Priority-Setting in Health - Building institutions for smarter public spending [Internet]. Washington, DC: Center for Global Development; 2012. Available from: https://www.cgdev.org/sites/default/files/1426240_file_priority_setting_global_health_FINAL_0.pdf
https://www.cgdev.org/sites/default/file... ,²⁸28. Hernandez-Villafuerte K, Li R, Towse A, Chalkidou K. International Decision Support Initiative: Mapping of Priority-Setting in Health for 17 Low and Middle Countries Across Asia, Latin America and Africa. SSRN Electron J [Internet]. 2015. https://dx.doi.org/10.2139/ssrn.2634886
https://dx.doi.org/10.2139/ssrn.2634886... ); and prioritization of research topics to achieve universal health coverage, including HRH (²⁹29. Ssengooba F, Ssennyonjo A, Rutebemberwa E, Musila T, Namusoke Kiwanuka S, Kemari E, et al. Research for universal health coverage: setting priorities for policy and systems research in Uganda. Glob Health Action. 2021;14(1). https://doi.org/10.1080/16549716.2021.1956752
https://doi.org/10.1080/16549716.2021.19... ).

Based on the search results, the criteria for prioritization related to policies and management in the HRH to be evaluated were created, encompassing the following items: relevance of the problem, and evaluation of the existence of a window of opportunity for action and acceptability. The methodological proposal for scoring the prioritization matrix was adapted from the model presented by Brazil (⁵5. Ministério da Saúde (Brasil). Priorização de Políticas e Programas Sob Responsabilidade Direta da Secretaria de Atenção à Saúde [Internet]. Ministério da Saúde, Secretaria de Atenção à Saúde Coordenação-Geral de Informação e Monitoramento de Serviços e Redes de Atenção à Saúde. Brasília: Ministério da Saúde; 2019. Available from: https://abrasco.org.br/wp-content/uploads/2021/04/livreto_priorizacaoSAS_ALT6-2.pdf
https://abrasco.org.br/wp-content/upload... ), in which all criteria score in the range between 0 and 4, as follows: None = 0; Low = 1; Intermediate = 2; High = 3; and Very high = 4.

Priority of each item is classified according to its final score. In situations of tie between the final mean values of each challenge, the following tiebreaker criteria would be used, in this order: 1. Higher score in the window of opportunity item; 2. Higher score in the relevance item; and 3. Higher score in the acceptability item.

To calculate the score for each item and later prepare the priority ranking, the arithmetic mean of all votes assigned to each criterion was used. In relation to the “window of opportunity” criterion, its score resulted from calculating a mean, subsequently added to the other two criteria.

Finally, a conceptual description of the voting criteria and methodology was reached, defined as Version 1 (Table 3).

The committee of judges made suggestions in relation to the material, which were considered in the review of the criteria and scale proposed for prioritization. The summary of the suggestions made by the committee of judges is in a supplementary appendix which is available on request through the corresponding author.

After the changes proposed by the committee of judges, the specialists were invited to review the material again, in individual evaluation sessions and, with their suggestions, the concepts and the methodology proposed were improved until reaching the final version, with the objective of achieving an excellent acceptability index of the criteria and the prioritization methodology. The result of the suggestions and adaptation with the changes are shown in Table 3.

Regarding the evaluation by the specialists in the second stage, presented in Table 1, four of the 13 items evaluated obtained 100% approval (30.7% of the items), five reached 75%, three 88% and one of them (Q1) obtained the lowest approval percentage, scoring 50%. The total approval percentage of the criteria and prioritization method was 84%. After implementing the changes, a pre-test was carried out to test effectiveness of the material by the target audience, which presented positive results, with total approval of 94%. The approval percentage varied by item, with six having maximum approval of 100% (corresponding to 46.15% of the items), four reaching 92% (related to the presentation in the material on the Google Forms platform) and three 83% each (Table 4).

DISCUSSION

The political process is not solely technical and based on scientific evidence but can be influenced by various factors as presented by conceptual frameworks such as 3Is+E (Interests, Ideas, Institutions and External Factors) and Kingdon (the configuration of the government agenda is the result of the combination of three independent streams, namely policy problems, policy solutions and the political stream, which includes public opinion, changes in governments, etc). (³⁰30. Oliveira APC de, Poz MRD, Craveiro I, Gabriel M, Dussault G. Factors that influence human resources for health policy formulation: a multiple case study in Brazil and Portugal. Cad Saude Publica. 2018;34(2). https://doi.org/10.1590/0102-311X00220416
https://doi.org/10.1590/0102-311X0022041... ,³¹31. Hoefer R. The Multiple Streams Framework: Understanding and Applying the Problems, Policies, and Politics Approach. J Policy Pract Res. 2022;3(1):1–5. https://doi.org/10.1007/s42972-022-00049-2
https://doi.org/10.1007/s42972-022-00049... ). This does not diminish the value of having tools to facilitate the decision-making process, considering that the definition of priorities in the health management context is a complex process that implies making choices based on rational and transparent criteria. The prioritization process achieves legitimacy by using the best available scientific evidence and reaching technical-political consensus. In addition to that, this task implies gathering a group of actors to reconcile different interests and world views (⁶6. Ranson MK, Bennett SC. Priority setting and health policy and systems research. Heal Res Policy Syst. 2009;7(1):27. https://doi.org/10.1186/1478-4505-7-27
https://doi.org/10.1186/1478-4505-7-27... ).

Decision-making is the duty of any manager and policy-maker in all health systems, in which definition of priorities is inevitable, considering the imbalance between needs and available resources and the multiple actors involved in the process, as well as the different influence degrees of these actors (⁵5. Ministério da Saúde (Brasil). Priorização de Políticas e Programas Sob Responsabilidade Direta da Secretaria de Atenção à Saúde [Internet]. Ministério da Saúde, Secretaria de Atenção à Saúde Coordenação-Geral de Informação e Monitoramento de Serviços e Redes de Atenção à Saúde. Brasília: Ministério da Saúde; 2019. Available from: https://abrasco.org.br/wp-content/uploads/2021/04/livreto_priorizacaoSAS_ALT6-2.pdf
https://abrasco.org.br/wp-content/upload... ). There are different examples of well-developed policies that address all the complexities of the health labor market, yet it is difficult to prioritize issues to be tackled in the face of limited resources, social demands and the relevance of the public agenda. Also, the window of opportunity to an issue on the political agenda can be short, it can disappear, so it is important to predict and prioritize (³²32. World Health Organization, Pan American Health Organization. Impact of COVID-19 on human resources for health and policy response: the case of Plurinational State of Bolivia, Chile, Colombia, Ecuador and Peru. Overview of findings from five Latin American countries [Internet]. Geneva: World Health Organization; 2021. Available from: https://iris.who.int/handle/10665/350640
https://iris.who.int/handle/10665/350640... ,³³33. Onvlee O, Kaitelidou D, Scotter C, Dieleman M. Navigating the political and technical challenges of developing a national HRH strategy: a case study in Greece. Health Policy. 2021;125(12):1574–9. https://doi.org/10.1016/j.healthpol.2021.08.010
https://doi.org/10.1016/j.healthpol.2021... ). For example, the Plan for the Implementation of the National Policy on HRH of the Ministry of Public Health and Social Welfare of Paraguay describes strategy, indicators, activities, goals and means of verification the achievements; however, the need to advance in operational prioritization was identified as a limitation (³⁴34. Guillou-Maudet M, Caballero-García C, Candia-Campos L, Mora-Escobar G, Komjati-de Miño R. Metodología de elaboración del Plan para la Implementación y Monitoreo de la Política Nacional de Recursos Humanos en Salud del Paraguay. Salud Publica Mex. 2023;65(4, jul-ago):361–9. https://doi.org/10.21149/14700
https://doi.org/10.21149/14700... ). Countries would therefore potentially benefit from tools such as the one proposed in this article and, also, in low governance contexts, evidence-based and validated tools can support consensus building around priorities.

Planning, implementation, monitoring and evaluation of challenges and interventions are competencies assigned to managers within the Brazilian Unified Health System (SUS) and imply changes and adaptations in bureaucratic procedures for strategic actions targeted at more effective, efficient and good quality results, based on prioritization of the sectorial public policies and on using fundamental instruments provided for in the legal bases of the local health system (⁵5. Ministério da Saúde (Brasil). Priorização de Políticas e Programas Sob Responsabilidade Direta da Secretaria de Atenção à Saúde [Internet]. Ministério da Saúde, Secretaria de Atenção à Saúde Coordenação-Geral de Informação e Monitoramento de Serviços e Redes de Atenção à Saúde. Brasília: Ministério da Saúde; 2019. Available from: https://abrasco.org.br/wp-content/uploads/2021/04/livreto_priorizacaoSAS_ALT6-2.pdf
https://abrasco.org.br/wp-content/upload... ).

Different proposals and methods can be used to decide on health priorities; in general, they are divided into two groups: approaches based on consensus and approaches based on metrics. The first results in priorities decided by group consensus, which tends to improve acceptability of the practice; the latter involves metrics or algorithms that result in the grouping of individual classifications, which prevents dominance by some participants (¹⁸18. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: Conducting a content validity study in social work research. Soc Work Res. 2003;27(2):94–104. https://doi.org/10.1093/swr/27.2.94
https://doi.org/10.1093/swr/27.2.94... ).

As it was possible to verify, no specific instruments were identified to establish priorities for the HRH management agenda, with identification of the prioritization criteria in the literature, sometimes not established or generally directed to the epidemiological context for diseases, and not including a broader and more integrated view of HSS or issues related to HRH (⁶6. Ranson MK, Bennett SC. Priority setting and health policy and systems research. Heal Res Policy Syst. 2009;7(1):27. https://doi.org/10.1186/1478-4505-7-27
https://doi.org/10.1186/1478-4505-7-27... ). Thus, the creation and conceptual adaptation of the criteria in the initial stage may have been impaired, being reflected in the evaluation by the Committee of Judges, as the question with the lowest approval percentage was related to the conceptual definition of the criteria presented.

Additionally, as discussed by Zanardo and Ventura (³⁵35. Zanardo ABR, Ventura CAA. Cultural adaptation and validation of the Strategies to end seclusion restraint module of the QualityRights ToolKit. Rev Lat Am Enfermagem. 2022;30. https://doi.org/10.1590/1518-8345.5638.3520
https://doi.org/10.1590/1518-8345.5638.3... ) in material validation studies, the main suggestions for modifications are generally related to wording of the items, such as substituting some words and standardizing others, which enabled adapting the terminology used and clarifying the information conveyed, thus guaranteeing coherence of the text resources of the material.

Therefore, establishing clear, transparent, and concise criteria can contribute to monitoring the problems encountered in the HRH policy framework and identifying windows of opportunity over time for the development and implementation of appropriate policies, knowing that HRH management requires flexibility and adaptability to deal with the constant changes and challenges in the public health scenario. Hence it is outstanding the importance of the knowledge translation process and the provision of timely and context-specific information to assist the use of diverse evidence to inform and guide development and implementation of effective policies.

The approval rate of the material by the target group, after the changes made in the previous stage (evaluation by the committee of judges), suggests that the changes were positive and allowed for greater conceptual understanding of the criteria. It is therefore considered that the instrument met the purposes for which it was developed, being able to guide health knowledge construction.

This instrument is intended for stakeholders who have previously analyzed the situation involving their workforce and the need for health systems and those who intend to do it. Furthermore, these actors have developed a network for dialogue, bringing in various viewpoints on particular issues and applying the instrument to allow for positioning and agreement on the issue that will be tackled. ln this process of dialogue and co-responsibility, a method like this has the potential to facilitate and streamline decisions.

In this context, this study has significant potential to contribute to the advancement of scientific knowledge in the HRH research area, improving the level of evidence available for decision-making to achieve universal health care coverage.

The study has limitations inherent to the methods applied. Regarding the systematized review, limitations rely on potential publication bias and a limited number of databases, which were mitigated by using grey literature and obtaining document recommendations from key informants. Furthermore, limitations of content validity studies can be observed, such as the subjectivity of expert feedback; foreseeing this possible bias, the panel of judges consisted of international experts from various backgrounds (managers, technicians, and academics).

In conclusion, the prioritization criteria and methodology for the area of HRH policies and management were created and evaluated by a committee of judges, reaching total approval of 84%. The suggestions were mostly accepted and those that were not accepted were justified. Immediately after that, to certify that the material was comprehensible to the entire population, the content was evaluated by the target audience, reaching a total approval of 94%.

In view of the results, the validated prioritization matrix proves to be an effective and reliable tool to guide decision-making in the area of policy and management of the health workforce, by elaborating solid criteria for selecting and facing challenges in HRH; thus, this matrix becomes an instrument of strategic relevance.

The findings represent an advance in policies based on scientific evidence, as the implementation of prioritization criteria can strengthen the response capacity in the face of emerging challenges, fostering resilience and sustainability of health systems in an environment in constant change. In addition to that, the presentation and description of the methodology and its validation have the potential to be replicated and adapted in other countries and contexts. Therefore, in terms of future research, it may be pertinent to expand it to other languages, including the remaining countries of the Community of Portuguese Speaking Countries in the process of validating the tool.

Disclaimer.

Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the Revista Panamericana de Salud Pública/Pan American Journal of Public Health or the Pan American Health Organization.

Acknowledgements.

The authors would like to thank the experts, managers and technicians on the committee of judges and the target audience for their time and valuable information provided. The article was prepared as the result of a project developed and funded in partnership with the Pan American Health Organization/World Health Organization (PAHO/WHO) and with the PAHO/WHO Collaborating Center for the Development of Nursing Research and Teaching Support Foundation. We also thank the Teaching, Research and Assistance Support Foundation (Fundação de Apoio ao Ensino, Pesquisa e Assistência, FAEPA) of the Clinical Hospital Complex belonging to the Ribeirão Preto Medical School at the University of São Paulo (Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo, HCFMRP-USP) for the managerial/administrative support.

REFERENCES

Publication Dates

History