Abstract
The objective was to analyze the functioning of the referral and counter-referral system for integral care in the Healthcare Network. Qualitative study performed through interviews with 66 participants - managers, workers and users of a town of the state of Bahia, Brazil, and upon approval by the Ethics Committee, opinion number nº 334.737. The material was analyzed using the Content Analysis proposed by Bardin, finished in 2014. Various conceptions of referral and counter-referral were identified, as follows: referral of users, user’s broader view; non-fragmented care; and integral care. For the users, the difficulties and facilities in the flows are concentrated in the Regulation and Scheduling Center and Family Health Units. It is concluded that, to enable the establishment of the network in an integral way, it is necessary to identify important strategies provided by the Unified Health System (SUS) and strengthen these strategies, as well as to identify any drawbacks to remedy them.
Keywords:
Health Care Networks; communication; integrality in health; quality management; Unified Health System
Introduction
The Unified Health System (SUS) has come to realize the public health system in Brazil guided by principles of integrality, universality and equity (Brazil, 1990). Nevertheless, among many challenges that permeate this health system, integrality is included, representing the difficulty of treating each individual as an indivisible being that is part of a society (SILVA; MIRANDA; ANDRADE, 2017SILVA, M. V. S.; MIRANDA, G. B. N.; ANDRADE, M. A. Sentidos atribuídos à integralidade: Entre o que é preconizado e vivido na equipe multidisciplinar. Interface: Communication, Health, Education, v. 21, n. 62, p. 589-599, 2017. DOI: https://doi.org/10.1590/1807-57622016.0420.
https://doi.org/https://doi.org/10.1590/... ).
Ordinance nº 4.279/2010 from the Ministry of Health (MS) reaffirms the responsibility on the part of SUS for integral care, presents the organization strategy of Health Care Networks (HCN) and defines it as “organizational arrangements for health actions and services of different technological densities, which, integrated through technical, logistical and managerial support systems, seek to guarantee the integrality of care”, and therefore the networks enable the operationalization of SUS by promoting continuous care, thus gaining prominence in discussions (BRASIL, 2011, p.4).
In orde to hold the operationalization of HCN by the health services, the referral and counter-referral system (RCR) is required, which refers to the mechanism for establishing communication. Through this system, it is possible to perceive in health services that the user obtains continuity in the care offered, where each information about the user, coming from different health professionals and different services, is always valid for the continuity of care for this individual, being seen as a whole and receiving integral care.
The RCR system is part of the logistical systems, understood as information technologies that provide the rational organization of the traffic of health information for each assisted individual, that is, flows and counterflows of information from users among the services that compose the network, as well as as of the people and products among the services that make up the networks, making the exchange of information along the HCN points effective, establishing a communication for the constitution of the integrality of care of each user (MENDES, 2011MENDES, E. V. As redes de atenção à saúde. Organização Pan-Americana da Saúde, Brasília, 2011. Available at: <http://www.saude.sp.gov.br/resources/ses/perfil/gestor/documentos-de-planejamento-em-saude/elaboracao-do-plano-estadual-de-saude-2010-2015/textos-de-apoios/redes_de_atencao_mendes_2.pdf>. Retrieved: 22 May 2020.
http://www.saude.sp.gov.br/resources/ses... ; ALVES et al., 2015ALVES, M. L. de F. et al. Reference and counter reference network for emergency care assistance in a municipality in the countryside of Minas Gerais - Brazil. Revista Médica de Minas Gerais, v. 25, n. 4, p. 469-475, 2015. DOI: http://www.dx.doi.org/10.5935/2238-3182.20150110.
http://www.dx.doi.org/10.5935/2238-3182.... ; PERREIRA; MACHADO, 2016).
This system is an important element for communication in HCN, but within it some limitations are still seen, such as failures in communication and the lack of knowledge on the part of workers about the RCR system, and therefore being unable to guarantee assistance in a non-fragmented way, which are limitations that hinder the shock of crossing the communication barrier to produce integral health care (MELO; CRISCUOLO; VIEGAS, 2016MELO, D. F.; CRISCUOLO, M. B. R.; VIEGAS, S. M. F. Referência e contrarreferência no cotidiano da atenção à saúde de Divinópolis-MG, Brasil: o suporte às decisões da atenção primária. Rev Fund Care Online, v. 8, n. 4, p. 4986-4995, 2016. DOI: http://dx.doi.org/10.9789/2175-5361.2016.v8i4.4986-4995
https://doi.org/http://dx.doi.org/10.978... ; BRONDANI et al., 2016BRONDANI, J. E. et al. Desafios da referência e contrarreferência na atenção em saúde na perspectiva dos trabalhadores. Cogitare Enferm., v. 21, n. 1, p. 01-08, 2016. DOI: http://dx.doi.org/10.5380/ce.v21i1.43350.
https://doi.org/http://dx.doi.org/10.538... ).
Knowing this, this study is considered relevant in a town located in the state of Bahia, which is part of a HCN where a federal university with a health sciences center is located; it is also important for the training of qualified professionals and for continuing health education, since it is a town that encompasses the care of several surrounding regions, thus strengthening the principle of SUS integrality.
Accordingly, this study aims at analyzing the functioning of the referral and counter-referral system for the integrality of the Health Care Network (HCN) in a town in Bahia, having as specific objectives: to know the conception(s) of the health managers and workers on referral and counter-referral; identify the actions demanded and developed by managers and workers, respectively, for the functioning of the referral and counter-referral system; and discuss facilities and difficulties for the flow and counterflow of users in the Health Care Network.
Method
This is a study with a qualitative approach of a descriptive and exploratory nature. It constitutes a thematic section of a larger research, which integrates the Education Program for Work for FHS and HCN Health of a federal university located in the state of Bahia, entitled: “The structuring and operationalization of the Health Care Network in a town in Bahia”, completed in 2014.
The study took place in a town located in Bahia, Northeast region of Brazil, which has a SUS Health Care Network that has a large dimension with several SUS health services and in all technological areas presented in: Primary Health Care (PHC), medium technological density and high technological density.
The field of study of the current research comprised thirteen health scenarios, in order to cover all levels of technological density of the SUS Health Care Network of the town in question. Through a semi-structured interview, 6 health managers, 32 health workers, middle and higher education, and 28 users of the research health scenarios were invited to participate in the study, which make up the SUS Health Care Network of the studied town, totaling 66 participants.
Data analysis was performed using the Content Analysis technique (BARDIN, 2011BARDIN, L. Análise de conteúdo. São Paulo: Almedina Brasil, Edição 70, 2011.), following the three chronological poles: pre-analysis, exploration of the material and treatment of the results. The ethical aspects of research with human beings were preserved, where the Research Ethics Committee (CEP) appraised it under consubstantiated opinion nº 334.737, all respondents signed the Free and Informed Consent Form and received it with the code T (worker, trabalhador in Portuguese) and G (manager, gestor in Portuguese). For the different investigated scenarios, the identification of the letter C (scenario, cenário in Portuguese) is attributed, followed by increasing numbering by location and order of respondents.
Conceptions of health managers and workers about referral and counter-referral
In their testimonies, the managers and health workers commonly attributed to the referral and counter-referral some conceptions, among which we discuss the following: referral of users; enlarged view of the user. The workers explained other conceptions, discussing some as such: user feedback; user monitoring; continued care; organization in HCN and articulation in HCN.
Analyzing and discussing some of the findings, it can initially be revealed by health managers and workers that the referral and counter-referral system comprises referral of users from one point to another of HCN, in order to meet the health need, explained in the following testimonials:
[...] this referral is the way to refer what cannot be done here. We refer and have the return, with the counter-referral [...] (C7T27).
In view of this, Mendes (2010MENDES, E. V. As redes de atenção à saúde. Ciênc. saúde coletiva, Rio de Janeiro, v. 15, n. 5, ago. 2010. Available at: <http://www.scielosp.org/pdf/csc/v15n5/v15n5a05.pdf>. Retrieved: 22 May 2020.
http://www.scielosp.org/pdf/csc/v15n5/v1... ) states that such a system constitutes an information technology that guarantees the rational organization of the flows and counterflows of information and users along the points of the health care network. Thus, it is easy to recognize that RCR in fact applies to the referral of users and can thus receive such a conception, which remains present in several testimonies by both managers and workers in this research, as noted in the testimony already presented previously.
Despite being a true and important conception, is the referral of users sufficient to represent the role of RCR? It can be suggested that it is a superficial conception obtained from what is explicit in the act of referencing. It is necessary to develop a critical view, capable of understanding the referral and counter-referral system and its applicability.
According to Mendes (2011MENDES, E. V. As redes de atenção à saúde. Organização Pan-Americana da Saúde, Brasília, 2011. Available at: <http://www.saude.sp.gov.br/resources/ses/perfil/gestor/documentos-de-planejamento-em-saude/elaboracao-do-plano-estadual-de-saude-2010-2015/textos-de-apoios/redes_de_atencao_mendes_2.pdf>. Retrieved: 22 May 2020.
http://www.saude.sp.gov.br/resources/ses... ), in the act of referencing and counter-referencing, the primary care professional (PHC), in addition to referring, should define why he/she is requesting the consultation or procedure, asking questions to which he/she would like to know the answers, listing the procedures he/she is performing on the user, drugs prescribed, listing the results he/she has presented and say what he/she is requesting from the specialist at the time. In turn, the specialist will have an enlarged view of the user and must fill in the counter-referral stating what he/she has done in the individual and the achieved results, making recommendations for the continuity of care in PHC and specifying in this form when the user should return to him/her, if necessary.
Thus, it can be seen that RCR represents much more than referring users. Therefore, as the ideas were presented by the interviewees in the research, other conceptions were revealed, which can be considered important for integrality in HCN, among them the concept of the user’s broader view, elucidated in the previously context. This conception gives the professional the opportunity to understand the user beyond the disease. An important understanding from the perspective of health managers and workers, observed in the following statements:
I think it’s important [...] so it works to give a broader view of the user (C8T4).
[...] the question of referral and counter-referral is a more complete look at the user, so it’s not enough that we’re simply referencing this user [...] (C3G6).
However, in view of such testimonies placed on the importance of the user’s broader view, it is worth asking the magnitude of the definition of the user’s broader view do managers and health workers understand? Are managers and health workers in this town aware of the management of care based on a user’s broader view in health practices?
Firstly, the holistic view interwoven in the concept of the user’s broader view stands out, which broadens the perception of the sick subject as a human being who is involved in a biopsychosocial context and who brings with him/her a life story, steeped in cultural values, as well as a health situation, and that from the moment he/she enters a health institution, he/she experiences different situations, since he/she is removed from his/her daily life (SILVA; MIRANDA; ANDRADE, 2017SILVA, M. V. S.; MIRANDA, G. B. N.; ANDRADE, M. A. Sentidos atribuídos à integralidade: Entre o que é preconizado e vivido na equipe multidisciplinar. Interface: Communication, Health, Education, v. 21, n. 62, p. 589-599, 2017. DOI: https://doi.org/10.1590/1807-57622016.0420.
https://doi.org/https://doi.org/10.1590/... ). Thus, this understanding observed in the interviews of health managers and workers in the studied town implies that the health professional must know all aspects that involve the user’s daily life in addition to the disease. In order for this knowledge to be present in the health professional’s practices, he/she needs to provide him/her with an instrument that presents him/her with the entire biopsychosocial context of the patient’s life, so that, from the instrument, he/she can assess the user’s health situation and what are the real needs of the subject and, sequentially, establish care directed to this individual.
In addition to the holistic view, the concept of RCR as a broadened view of the user denotes the idea of a broadened clinic. In this perspective, in health care practice, the professional allows himself/herself to be taken by the sick person’s singularities, in addition to other theoretical knowledge already acquired, and produces a care plan that takes into account both the acquired theoretical knowledge and the user’s singularities. (SILVA et al., 2018aSILVA, K. A. B. et al. Desafios no processo de referenciamento de usuários nas redes de atenção à saúde: perspectiva multiprofissional. Cienc. Cuid. Saúde, v. 17, n. 3, p. 1-8, 2018a. DOI: 10.4025/cienccuidsaude.v17i3.43568.
https://doi.org/10.4025/cienccuidsaude.v... ; SILVA et al., 2018b). It assumes that each individual actively interferes in the health-disease phenomenon, without neglecting the interference of the social environment for the subject.
In this sense, it can be said that the disruption of therapeutic interventions that dissociate the individual constitutes a major challenge to be faced in the health area, as they are opposed to the principle of integrality. The professional look towards the user must go beyond the disease, taking over what covers the individual, establishing an integral and humanized care, that is, care that is in accordance with the users’ singularities.
There is a tendency towards the organization of health services and work processes based on a discernment of hegemonic management, in order to influence the object of work of health professionals, reducing it to procedures, techniques, diseases and body parts. Moreover, it is observed that health professionals within the services are responsible for the subjects only when they are in the physical space of the service they work for, leaving the responsibility behind as for the moment before and after the subject passes through the service (CUNHA; CAMPOS, 2011CUNHA, G. T.; CAMPOS, G. W. S. Apoio Matricial e Atenção Primária em Saúde. Saúde Soc., v. 20, n. 4, p. 961-970, 2011. DOI: https://doi.org/10.1590/S0104-12902011000400013.
https://doi.org/https://doi.org/10.1590/... ). Accordingly, it is important to consider matrix support as a complementary strategy to the RCR system.
Matrix support consists in offering assistance through technical and pedagogical support to the professionals involved in the referencing of each subject. This support works in conjunction with the RCR system with the objective of promoting dialogical integration among professionals from different specialties and consequent promotion of broadened clinic and the longitudinal responsibility of all professionals from the different specialties and services involved with the patient, that is, throughout the time necessary and regardless of which service he/she is being cared for (CAMPOS; DOMITTI, 2007CAMPOS, G. W. S.; DOMITTI, A. C. Apoio matricial e equipe de referência: uma metodologia para gestão do trabalho interdisciplinar em saúde. Cad. Saúde Pública, v. 23, n. 2, p. 399-407, 2007. DOI: http://dx.doi.org/10.1590/S0102-311X2007000200016
https://doi.org/http://dx.doi.org/10.159... ).
Other conceptions identified for RCR in the interviewees’ testimonies - feedback, monitoring and continuity of care - can be represented in a single testimony of a health worker when talking about RCR, thus demonstrating the interconnection of such conceptions:
I think that referral and counter-referral is a subsidy used for continuity of service that, from the moment I sent one, one, a patient to the Viva Mulher or any other program, from the moment I send her, I wait an answer of how it was there, so I, as a unit, will be aware of that health problem to provide a subsidy and also monitor, not abandoning that patient [...] the continuity of the health care service (C4T22).
The counter-referral, carried out by filling of the specific form, consolidates relevant information from the user, which allows the unit that referred to understand what behaviors were adopted in the unit to which he/she was referred, in order to continue the assistance in the unit of origin of the user and promote integrality (SILVA et al., 2018aSILVA, K. A. B. et al. Desafios no processo de referenciamento de usuários nas redes de atenção à saúde: perspectiva multiprofissional. Cienc. Cuid. Saúde, v. 17, n. 3, p. 1-8, 2018a. DOI: 10.4025/cienccuidsaude.v17i3.43568.
https://doi.org/10.4025/cienccuidsaude.v... ).
In addition to the conceptions discussed, many others were presented by the interviewees in their testimonies as previously mentioned. Obviously, all conceptions granted to RCR contribute to the formation of interconnected services and with interdependent practices, thus forming organized networks that communicate in order to provide health care to the population. However, all these conceptions were presented by the interviewees in a superficial way, which cannot represent the magnitude of the conception of the referral and counter-referral.
Faced with so many understandings, it is interesting to note that, at no point in the survey, the interviewees bring integral care to the user as a conception for the RCR system. Apparently, they see all these conceptions, but in a way that each interviewee highlights one or the other, dissociated conceptions, when in fact all these conceptions are involved with integral care, which is essential for the health care of individuals, imbued in the concept of integrality.
It is also perceived that the research participants do not mention the matrix support, which refers to the dialogic and integrated functioning of the teams involved in referencing the patient. In this direction, it is necessary to bring this agenda about the potential of integrated and dialogical work for HCN, having RCR as an enhancer of this process. In matrix support, teams can discuss clinical cases and health situations, management and organization in services and also contribute jointly with interventions that can add knowledge that increase the ability to solve problems experienced by patients and their families in the RCR system. (CAMPOS; DOMITTI, 2007CAMPOS, G. W. S.; DOMITTI, A. C. Apoio matricial e equipe de referência: uma metodologia para gestão do trabalho interdisciplinar em saúde. Cad. Saúde Pública, v. 23, n. 2, p. 399-407, 2007. DOI: http://dx.doi.org/10.1590/S0102-311X2007000200016
https://doi.org/http://dx.doi.org/10.159... ).
Actions of health workers and managers for the functioning of the referral and counter-referral in the Health Care Network
Health workers must pay attention to users, answering the requests and rights of service users with an incessant search for resolution, since health practices generate meetings between workers and users, where these workers consist of making an effort to provide the necessary and integral care, taking responsibility for the user’s health and implementing public health policies (BRONDANI et al., 2016BRONDANI, J. E. et al. Desafios da referência e contrarreferência na atenção em saúde na perspectiva dos trabalhadores. Cogitare Enferm., v. 21, n. 1, p. 01-08, 2016. DOI: http://dx.doi.org/10.5380/ce.v21i1.43350.
https://doi.org/http://dx.doi.org/10.538... ).
It is also known that health managers have the constant challenge of seeking to achieve the guiding principles of SUS through the regulation and planning of actions and services in their professionalism, always considering the guarantee of care according to the population’s health needs. Therefore, it is necessary that managers and health workers are always motivated to constantly seek to develop the formation of HCN (PAULA; VOLOCHKO; FIGUEIREDO, 2016PAULA, S.H.B.; VOLOCHKO, A.; FIGUEIREDO, R. Linha de cuidado de câncer de mama e de colo de útero: um estudo sobre referência e contrarreferência em cinco regiões de saúde de São Paulo, Brasil. Boletim do Instituto de Saúde, v. 17, n. 2, p. 146-166, 2016. Available at: https://pesquisa.bvsalud.org/portal/resource/pt/ses-34272. Retrieved: 20 May 2020.
https://pesquisa.bvsalud.org/portal/reso... ). Thus, as mentioned, filling out the referral and counter-referral form is an instrument that has an indispensable value for the care of SUS health users, enabling quality care to meet people’s health needs, with a continuous character without fragmenting the health care (TORRALBO; JULIANI, 2016TORRALBO, F. A. P.; JULIANI, C. M. C. M. O sistema de referência e contrarreferência no atendimento ao adolescente: realidade x integralidade. Rev enferm UFPE online, v. 10, n. 3, p. 1016-21, 2016. DOI: 10.5205/reuol.8702-76273-4-SM.1003201610.
https://doi.org/10.5205/reuol.8702-76273... ).
Filling out the referral and counter-referral form, together with matrix support, constitute work methodologies that seek to reduce the fragmentation existing among health services and among health professionals, enabling communication, interdisciplinarity and sharing longitudinal responsibility for the integral care of patients and their respective therapeutic projects, and therefore contributing to the effectiveness of health interventions (CAMPOS; DOMITTI, 2007CAMPOS, G. W. S.; DOMITTI, A. C. Apoio matricial e equipe de referência: uma metodologia para gestão do trabalho interdisciplinar em saúde. Cad. Saúde Pública, v. 23, n. 2, p. 399-407, 2007. DOI: http://dx.doi.org/10.1590/S0102-311X2007000200016
https://doi.org/http://dx.doi.org/10.159... ).
During the analysis of the interviews, it was possible to identify many actions presented by the workers developed for the functioning of RCR, among them we can present the following: Filling out the referral and counter-referral form; health education; dialogue; and continued care. As for managers, among many actions, some are discussed here: activities in the community involving the entire network; interdisciplinary and intersectoral forum.
In this sense, the research health workers reported that they fill out the referral and counter-referral form, identified in the statements below:
[...] Then, we refer the patient to, it is not, for him to go through the network, which is necessary, for his treatment, he needs a specialist. And you have to use the network to refer him to another treatment unit that has a specialist. Accordingly, we use the referral and counter-referral [...] (C7T28).
The workers point to the action of referring/directing through the referral and counter-referral instrument. It can be seen through the documents raised in the town that only the Procedures Manual of the Regulation and Scheduling Center regulates the flow of SUS users and in a unidirectional way (Family Health Unit for outpatient specialties of medium and high technological density). How can the unidirectional and vertical perspective guide the flow in front of the different entrance doors? Given this context, how does the flow of SUS users happen when they are in health services with higher technological density, that is, the counterflow?
It is understood that, no matter how different professionals are in charge of the different therapeutic projects of the patients, it is important to define, among those involved, the person responsible for conducting each case and guiding the flows and counterflows. This does not mean taking responsibility for all the specialties involved in patient care, but it does mean the ability to coordinate the dialogue to understand each case and its respective therapeutic proposals, considering the subject globally and leading the discussions to define priorities and determining guidelines for each case that passes through the RCR system (CUNHA; CAMPOS, 2011CUNHA, G. T.; CAMPOS, G. W. S. Apoio Matricial e Atenção Primária em Saúde. Saúde Soc., v. 20, n. 4, p. 961-970, 2011. DOI: https://doi.org/10.1590/S0104-12902011000400013.
https://doi.org/https://doi.org/10.1590/... ).
The managers reported that they carry out actions for the functioning of RCR, such as: activities in FHS related to other levels of technological density; community activities involving the entire network; interdisciplinary and intersectoral forum. We observe such actions in the following excerpts:
[...] we develop activities in related units such as other levels, other sectors, such as epidemiological surveillance, that is, the hospital network, besides the basic network itself, we involve all levels [...] activities with, with the community involving all networks (C5G4).
[...] in this 2014 planning, we put up some forums, these forums articulate the questions of actually going to be an interdisciplinary forum, an intersectoral forum, where there will be an articulation between all urgency and emergency services in the city with all the municipal health care network [...] (C3G6).
Nonetheless, the managers’ testimonies allow the following reflection: do these activities and interdisciplinary forum, developed for the functioning of RCR, emphasize the applicability of RCR, its importance for the integrality of care in HCN and the right that users have to receive them? Every health manager is an actor in a government situation who must fulfill his/her responsibilities and plan the offer of actions and services based on the needs of the individuals who are part of the population. Managers are also responsible for meeting the goals assumed by health services at different levels of technological density; regulation; and monitoring of the results achieved for evaluation and adjustments in the planning of the actions and services offered (PAULA; VOLOCHKO; FIGUEIREDO, 2016PAULA, S.H.B.; VOLOCHKO, A.; FIGUEIREDO, R. Linha de cuidado de câncer de mama e de colo de útero: um estudo sobre referência e contrarreferência em cinco regiões de saúde de São Paulo, Brasil. Boletim do Instituto de Saúde, v. 17, n. 2, p. 146-166, 2016. Available at: https://pesquisa.bvsalud.org/portal/resource/pt/ses-34272. Retrieved: 20 May 2020.
https://pesquisa.bvsalud.org/portal/reso... ).
Throughout health management experiences in the town in question, it is observed that managers, in the midst of so many attributions, can sometimes fail to exercise fundamental activities for SUS. As an example of this, it seems to be observed a non-prioritization of the continuing education process on HCN, care management and on referral and counter-referral, important frameworks for integrality, and, consequently, strengthening of SUS, directly linked to the strengthening of the Pact for Health in its three dimensions: Pact for life, Pact in defense of SUS and the Pact for management, released by the Ministry of Health in 2006, which are focused on seeking to overcome the fragmentation of health policies and programs, as well as qualifying the population’s access to integral health care.
Conversely, in the research, workers affirm that, in their work, they carry out health education; however, in their testimonies, they make it clear that they are actions aimed at the care of dietary health, body health, promotion and prevention of diseases, but they do not talk about educational actions aimed at the applicability of the RCR form and the right that users have to receive it, which can be demonstrated through the report listed below:
I think the main thing is the educational action, isn’t it, health education, isn’t it? Making the population aware of the importance of health care, food care, body care, prevention is about diseases and something like that, and my work goes a lot in that direction. I think it is very important to be helping the population to improve the quality of life (C7T26).
In this context, health education strengthens access to health information and empowers citizens in the execution of their rights. A feasible example is the Charter of Rights of Health Users, from SUS, which address situations that range from the right to identify the professional who pays attention to the right to orderly and organized access to the health system (BRASIL, 2007). Nevertheless, health education alone does not answer all the emerging demands for the consolidation of SUS and the operationalization of HCN.
This is intriguing because, apparently, at the same time that managers claim to develop an intersectoral and interprofessional forum and activities involving the entire network for the functioning of RCR, it is clear that many do not understand what are the actions that should be developed directed to the functioning of RCR, i.e., what actions imply the effectiveness of RCR form. This fact makes us think that these actions of health managers may not be enough to reveal to health workers the importance of RCR and the necessary activities to be implemented for its operationalization, or perhaps, it can happen due to the fact that, in their actions, managers fail to reveal to health workers their responsibility to carry out RCR in their work environments.
Health workers need to recognize which health actions are necessary to be implemented in order to achieve complete assistance and to strengthen the HCN context, where the user is the center of their work, fulfilling their tasks and building strategies to favor the technologies that enable them to hold the integrality of SUS, as this will favor the development of an effective health policy, bringing satisfaction and problem solving to the population that seeks health services.
When discussing integral care, it is necessary to reflect on its dimensions: on the one hand, it is referred to as a set of services offered for the population’s health, articulating preventive actions with care actions meeting the people’s needs, but, on the other hand, integrality includes a permanent organization and interaction of resources and professionals at different levels of health care (TORRALBO; JULIANI, 2016TORRALBO, F. A. P.; JULIANI, C. M. C. M. O sistema de referência e contrarreferência no atendimento ao adolescente: realidade x integralidade. Rev enferm UFPE online, v. 10, n. 3, p. 1016-21, 2016. DOI: 10.5205/reuol.8702-76273-4-SM.1003201610.
https://doi.org/10.5205/reuol.8702-76273... ).
The practice in health is a challenge in which the health worker has to seek commitment in the perspective of building articulated health actions and establishing the integration among the generated health information, integration among professionals with a view to enabling integral care and, the commitment of the worker to the referral of the user during his/her journey through the network services becomes important. Accordingly, the performance of health workers has established itself as an important constituent in the process of caring for users, due to the complexity that exists and the commitment that must exist in professional performance (PAULA JÚNIOR; RIZZON; MACHADO, 2018PAULA JÚNIOR, W. M. P.; RIZZON, A. M.; MACHADO, C. J. Comunicação entre profissionais como ferramenta para cuidado integral aos pacientes oncológicos por mecanismos de referência e contrarreferência. Revista Interdisciplinar Ciências Médicas, v. 2, n. 2, p. 1-5, 2018. Available at: http://revista.fcmmg.br/ojs/index.php/ricm/article/view/90. Retrieved: 22 May 2020.
http://revista.fcmmg.br/ojs/index.php/ri... ).
Therefore, it is important to consider educational actions for network users. There is in the Municipal Health Secretariat of the town in question, the Report of the waiting room of the Family Health Units that proves the existence of some educational actions that took place; however, only three FHU in the town carried out educational actions related to RCR, and which were nevertheless carried out in partnership with PET-SF and HCN through students and professors at the federal university of the town. They were intended to clarify the role of RCR and make users aware of the right to receive and require the RCR functioning. To that end, they had as themes: What is the referral and counter-referral?; Referral and counter-referral. What I have to do with it?; Referral and counter-referral and HCN; Referral and counter-referral: guaranteeing your rights.
It is seen that this record of educational actions analyzed is self-explanatory and manages to pass its objective to the reader; however, it is not a guide, and is not available online or easily accessible to people, which could contribute so that other professionals and even users could understand the relevance of these actions, and professionals are motivated to perform such activities in their services.
An action understood as essential to bring changes in health workers is continuing education, which can be suggested for the town under study, overcoming some difficulties and weaknesses, as practices cannot be changed without first changing the actors, without forming people, that is, it is necessary for the subject to recognize reality and himself/herself, in order to recreate health practices (BRONDANI et al., 2016BRONDANI, J. E. et al. Desafios da referência e contrarreferência na atenção em saúde na perspectiva dos trabalhadores. Cogitare Enferm., v. 21, n. 1, p. 01-08, 2016. DOI: http://dx.doi.org/10.5380/ce.v21i1.43350.
https://doi.org/http://dx.doi.org/10.538... ). The changes that happen in health practices require time for health workers to mature and become aware.
Knowing the Municipal Health Plan (PMS) 2014-2017, a document that must be public, but access was obtained after requesting the Municipal Health Secretariat, it is possible to observe a table of proposals and referrals made by the groups of works of the 3rd Municipal Conference in 2011, and the proposal includes the implementation of an effective RCR system throughout the health network, as well as the creation of a computerized system for scheduling appointments and guaranteeing RCR.
Among the prioritized problems, this PMS also includes the low effectiveness of the municipal referral and counter-referral system. Nevertheless, when looking at this document, the proposed actions of continuing education, planned to solve the prioritized problems, it is noticed that, in none of them, there is a focus on RCR, which seems to be a planning failure, leaving aside an important problem to be solved, managers may be suggested to observe this problem and create strategies with objectives and goals that would reverse the low effectiveness of the municipal system of RCR.
The interviews of the interviewed workers pointed out that they also put dialogue and non-fragmented care as a developed action, which can be represented through the testimonies:
[...] there must be a dialogue among these units. If you can’t do it ... At meetings, let’s try to at least have this relationship on paper or phone because if the same patient I am receiving here, unit “x” is receiving, at one time, hospital “x” is receiving, sometimes other institutions are ... Social are receiving, so it is interesting that this care is given together and not fragmented (C8T3).
Thus, they affirm the need for dialogue for the continuity of care, so that the actions do not take place in a fragmented way, but integral. This is inevitable, since there is no way of continuing care without dialogue and exchange of information between professionals and services, and this happens from the referral and counter-referral system. Therefore, if there is no commitment to RCR, there is no non-fragmented care, there is no integrality in the network.
However, the worker emphasizes the importance of non-fragmented care; and, for its existence, emphasizes the need for RCR, but says that, if RCR works, it will be a “dream fulfilled”, which leaves us to understand and perceive even more how much the act of referencing and counter-referencing is far from the real practice for workers, very weakened in the town in question. This discussion is emphasized by other workers:
[...] when a patient is referred to another unit and we don’t receive a counter-referral, we don’t really know what happened to him, we don’t have any information about what was done, the operation of the network (C7T28).
When referred to the services, the user lacks an effective communication among professionals through the referral and counter-referral system. The difficulty of individuals to be referred with all their health information to other services compromises the principle of integrality, thus preventing the continuity of the care of these individuals (BRONDANI et al., 2016BRONDANI, J. E. et al. Desafios da referência e contrarreferência na atenção em saúde na perspectiva dos trabalhadores. Cogitare Enferm., v. 21, n. 1, p. 01-08, 2016. DOI: http://dx.doi.org/10.5380/ce.v21i1.43350.
https://doi.org/http://dx.doi.org/10.538... ). On the contrary, the existence of health care through matrix support enhances interdisciplinary teams and, consequently, may result in better standards of responsibility for a broadened clinic and integration between users and professionals in a more humane and effective way (CAMPOS; DOMITTI, 2007CAMPOS, G. W. S.; DOMITTI, A. C. Apoio matricial e equipe de referência: uma metodologia para gestão do trabalho interdisciplinar em saúde. Cad. Saúde Pública, v. 23, n. 2, p. 399-407, 2007. DOI: http://dx.doi.org/10.1590/S0102-311X2007000200016
https://doi.org/http://dx.doi.org/10.159... ).
In the research, it is seen that health workers bring more actions to the functioning of RCR than the managers, also demonstrating that the managers may not be carrying out their daily experiences for the functioning of RCR in the network. Health workers and managers present dissociated actions, which are very weak, succinctly reported, many are not relevant to the effectiveness of RCR and that together are not sufficient to implement the RCR system, and not everyone is committed to carrying it out. Therefore, from the findings, it can also be inferred that health managers have few strategies and their actions targeted at workers are incipient for the functioning of the referral and counter-referral.
Facilities and difficulties for the flow and counterflow of users in the Health Care Network
In SUS, the existence of multiple networks among health services stands out, which communicate with each other, in different paths, forming lines of care based on the construction of connective flows. To that end, it is based on the formation of meetings between workers and users that it is possible to develop health work, as they build communication flows and allow the formation of a network intertwined in the practice of care. In this context, RCR is part of this communications network for articulating health services based on mutual referral of patients (TORRALBO; JULIANI, 2016TORRALBO, F. A. P.; JULIANI, C. M. C. M. O sistema de referência e contrarreferência no atendimento ao adolescente: realidade x integralidade. Rev enferm UFPE online, v. 10, n. 3, p. 1016-21, 2016. DOI: 10.5205/reuol.8702-76273-4-SM.1003201610.
https://doi.org/10.5205/reuol.8702-76273... ).
As for referrals, when questioned to SUS users in the town, through this research, if they receive any referral to another health service, most say they have already received, few say no. When asked the same users if they have already received a form to be returned to the health unit, most answered that they did not receive it, few say they have already received it. This suggests that not all users are being referred with the referral form and almost all are returning to the service of origin without the counter-referral.
When asked to the users of the health services of the town about what facilities are found when they are referred in the network, most of them send the appointments that are being made in the FHS or CAPS without the need to go to CRM; appointments without the need to face a line and the appointment that is happening faster. These results can be exemplified, respectively, in the following testimonies:
It’s easier ... It’s ... faster, it’s not too long, I don’t think it’s long. If I had to go there to the central to be making an appointment for myself, it would take longer than being sent by the health post” (C2U1).
CAPS is a facilitator of referrals for those who have a problem, we don’t have to face an Appointment Center, everything is already sent from here (C8U8).
It is observed that users mention facilitations in the appointments made in FHS, which were previously in the Regulation and Scheduling Center. In 2013, the process of decentralization of appointments was implemented in the town, where the units took the appointment of procedures and examinations of the population covered by each unit, and the professionals prioritize the appointments according to the most urgent needs and send them every two weeks via bags for transport from the municipal secretariat to the Regulation and Scheduling Center.
When dealing with difficulties indicated by users when being referred, the following were mentioned: the number of insufficient vacancies for services; the arrival time at CRM to get a vacancy; the long lines formed the day before the appointment; and transport difficulty to go to CRM. These can be seen in the excerpts below:
The difficulty that I am encountering is that these people at the appointment center just keep telling me that they’re not having a vacancy yet and that they’re waiting to finish the people who have a cancer problem, and then send me a call, call to be able to schedule mine, everything is already there, I’m just waiting to call me (C1U7).
Because many times you get there, you can’t find ... You don’t have a form anymore, you have to sleep there to get a form, even more if you have a child, you can’t go out at night to sleep outside. That’s it (C2U3).
Because it’s far, [laughs] taking transports (C7U28).
In view of the facilities and difficulties reported by users in the interviews, it is clear that the common argument for them is aimed at marking specialties and procedures for medium and high technological density, both those referring to facilities and difficulties are related to the realization of this scheduling that take place within the scope of CRM and FHS, thus constituting the “bottleneck” for these users of the city.
However, this concentration of facilities and difficulties in the flow and counterflow of users turned to CRM and FHS, in the sense of obtaining scheduling or not for appointments and procedures of specialties, suggests that users of health services in the town may not have knowledge about the referral and counter-referral form, and therefore they may also not recognize it as a facility or difficulty when being referred within the network, not knowing the importance of this form during their referral within the network and for the integrality of care, besides suggesting that, consequently, users are unaware of the right to receive it.
In the studied town, there is a Standard Operating Procedure Manual of the Municipal Regulation Center (2013), which is aimed at standardizing the procedures, examinations and appointments requested by the municipality to be carried out at the Municipal Polyclinic, Maternity Hospital of the municipality, Regional Hospital and accredited providers of the specialized network, with the objective of universalizing the dialogue and actions facilitating the appointments and enabling the population’s access and consecutive resolution of its needs.
Based on the Manual, the pre-determined routes for users in the town are described in a unidirectional way, only with the flows of Primary Care for medium and high technological density. In fact, there are no paths described as to users’ counterflows, revealing that, when the user has already been referred, he/she needs to go through other services or even return to his/her original service to continue the care, as there are no possible directions described in the user’s manual. As for the flows described in this protocol, it follows according to the flowchart (figure 2):
Flow of users in the Health Care Network according to the Standard Operating Procedure Manual of the Regulation and Scheduling Center of the studied town, 2013
In this sense, returning to the map of the municipality’s health care network, shown in Figure 1, there is exposed the different possibilities of flows and counterflows that the user can establish in his/her care process within the HCN depending on his/her health need.
Thus, in order to hold the effectiveness of integral care, it is important to adapt a municipal document that standardizes the different flows and counterflows of users in the HCN context. Document that can transcend the discourse and experience of flow and counterflow in the network to the regulation of the user being regulated in the different points of the network and that allows him/her to permeate these points of different technological densities according to his/her needs, since the description in the existing document of the flow normalized by the municipality leaves the FHS and is finished in the polyclinic, specialized network, regional hospital and maternity hospital, but there is no description of all the paths that the user can follow in HCN.
This document to be adapted must present all the possibilities of flow and counterflow in the municipality’s network and must guide workers and users in the face of referrals; therefore, it must be a guiding and self-explanatory document, which translates its objective, and easily accessible so that the user can go through the HCN paths.
Given the above and knowing that filling out the RCR form has an indispensable value for user care, with quality care that guarantees the same service in all spheres of care according to health needs, besides ensuring rational organization of flows and counterflows of information (PEREIRA; MACHADO, 2016PEREIRA, J. S.; MACHADO, W. C. A. Referência e contrarreferência entre os 1033 serviços de reabilitação física da pessoa com deficiência: a (des)articulação na microrregião Centro-Sul Fluminense, Rio de Janeiro, Brasil. Physis: Revista de Saúde Coletiva, v. 26, n. 3, p. 1033-1051, 2016. DOI: http://dx.doi.org/10.1590/S0103-73312016000300016.
https://doi.org/http://dx.doi.org/10.159... ), it is clear that users, when referred to HCN, need to be aware of their rights and the relevance of RCR for health care to be applied by professionals, which in fact is not seen in the research, as users, at no time, cite RCR as a facility or as a difficulty in referral, which suggests that they do not have an understanding of RCR.
RCR could be put into arguments by users as a facility during referral within the network, as, as already discussed here, it allows integral quality care to the individual, or perhaps it could be placed as a difficulty if it was not being effective in the town.
In order to enhance the functioning of the RCR system, matrix support is suggested, at the same time, to facilitate the process through a support specialist who must maintain contact with the referral team and make decisions for each patient’s therapeutic project. Thus, CRM would be triggered only in urgency and with the objective of monitoring and evaluating therapeutic projects according to the pertinence (CUNHA; CAMPOS, 2011CUNHA, G. T.; CAMPOS, G. W. S. Apoio Matricial e Atenção Primária em Saúde. Saúde Soc., v. 20, n. 4, p. 961-970, 2011. DOI: https://doi.org/10.1590/S0104-12902011000400013.
https://doi.org/https://doi.org/10.1590/... ).
Final considerations
In order to enable the network to be established in its entirety, it is necessary to identify important strategies instituted in the SUS context and to strengthen them, as well as to identify the obstacles to solving them and also to recognize the relevance of the referral and counter-referral system for the integrality in HCN. In this sense, this research was idealized. In compliance with their objectives, it appears that health managers and workers have different conceptions; however, They are presented in a dissociated way by the interviewees and that do not represent the importance and complexity of RCR.
It was also seen that the actions of managers and health workers for the functioning of RCR are incipiente, in addition to the fact that some actions are not relevant to the effectiveness of the referral and counter-referral.
As for users, it was identified that all difficulties and facilities addressed by them during their travels on the network are concentrated on CRM and FHS, in the sense of obtaining or not appointments for procedures and specialties. Therefore, it is suggested that they may not have knowledge about RCR, and therefore they do not recognize it as a facility or perhaps as a difficulty in their referrals within the network, thus not recognizing the RCR form as important for the construction of their integral care, as well as it can be inferred that they are unaware of the right to receive it in the act of flow and counterflow in services.
Although the collection for the development of this research was completed in 2014, it is observed from the discussion based on recent publications that the findings of the investigated reality are close to many realities in the country and that part of these realities remain problems in the functioning of HCN with persistent difficulties in the flow of users. Investments of a political and operational nature in the national and local SUS have not yet been able to overcome the difficulties now presented. It is noteworthy that this study has contributed to underlining the importance of building Health Care Networks and their communication, highlighting RCR as a potential instrument for reaching this communication, thus favoring the realization of one of the principles of SUS, i.e., integrality.
It is hoped that the results and discussions of this study will serve to enable health professionals to envision Health Care Networks, concrete and integrated, providing continuous care and contributing to the realization of existing public health policies, so that users have each more and more integral and quality care.
Finally, it is recommended that other studies be carried out to investigate RCR in other contexts and scenarios, and also the development of research-action combined with information technology to strengthen the operationalization of the flow and counterflow of users in the Health Care Network.11 C. C. R. B. Oliveira: data collection; data analysis; conceptualization; resource management; project management; investigation; methodology; writing: manuscript preparation, proofreading and editing; supervision. E. A. L. Silva: acquisition of funds; data collection; data analysis; conceptualization; resource management; project management; research; methodology; writing: manuscript preparation, proofreading and editing; supervision. M. K. B. de Souza: data collection; data analysis; conceptualization; resource management; project management; research; methodology; writing: manuscript preparation, proofreading and editing; supervision.
References
- ALVES, M. L. de F. et al. Reference and counter reference network for emergency care assistance in a municipality in the countryside of Minas Gerais - Brazil. Revista Médica de Minas Gerais, v. 25, n. 4, p. 469-475, 2015. DOI: http://www.dx.doi.org/10.5935/2238-3182.20150110.
» http://www.dx.doi.org/10.5935/2238-3182.20150110. - BARDIN, L. Análise de conteúdo. São Paulo: Almedina Brasil, Edição 70, 2011.
- BRASIL. Lei 8.080 de 19 de setembro de 1990. Lei Orgânica da Saúde. Diário Oficial [da República Federativa do Brasil], Brasília, DF, 20 dez. 1990, seção 1, p. 18055. Available at: <http://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?jornal=1&pagina=1&data=20/09/1990>. Retrieved: 22 May 2020.
» http://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?jornal=1&pagina=1&data=20/09/1990 - ______. Ministério da Saúde. Gabinete do Ministro. Portaria nº 4.279 de 30 de dezembro de 2010. Estabelece diretrizes para a organização da Rede de Atenção à Saúde no âmbito do Sistema Único de Saúde (SUS). Diário Oficial [da República Federativa do Brasil], Brasília, DF, 31 dez. 2010, seção 1:88. Available at: <ftp://ftp.saude.sp.gov.br/ftpsessp/bibliote/informe_eletronico/2011/iels.jan.11/iels02/u_pt-ms-gm-4279_301210.pdf>. Retrieved: 22 May 2020.
- ______. Ministério da Saúde. Portaria 2.488 de 21 de outubro de 2011. Aprova a Política Nacional de Atenção Básica. Diário Oficial [da República Federativa do Brasil], Brasília, DF, 21 out. 2011, seção 1:48. Available at: <http://www.fonosp.org.br/wordpress/wp-content/uploads/2011/11/portaria-248820111.pdf>. Retrieved: 22 May 2020.
» http://www.fonosp.org.br/wordpress/wp-content/uploads/2011/11/portaria-248820111.pdf - ______. Ministério da Saúde. Carta dos direitos dos usuários da saúde. Ministério da Saúde. Brasília, DF, 2007. Available at: <http://bvsms.saude.gov.br/bvs/publicacoes/carta_direito_usuarios_2ed2007.pdf>. Retrieved: 22 May 2020.
» http://bvsms.saude.gov.br/bvs/publicacoes/carta_direito_usuarios_2ed2007.pdf - BRONDANI, J. E. et al. Desafios da referência e contrarreferência na atenção em saúde na perspectiva dos trabalhadores. Cogitare Enferm., v. 21, n. 1, p. 01-08, 2016. DOI: http://dx.doi.org/10.5380/ce.v21i1.43350.
» https://doi.org/http://dx.doi.org/10.5380/ce.v21i1.43350. - CAMPOS, G. W. S.; DOMITTI, A. C. Apoio matricial e equipe de referência: uma metodologia para gestão do trabalho interdisciplinar em saúde. Cad. Saúde Pública, v. 23, n. 2, p. 399-407, 2007. DOI: http://dx.doi.org/10.1590/S0102-311X2007000200016
» https://doi.org/http://dx.doi.org/10.1590/S0102-311X2007000200016 - CUNHA, G. T.; CAMPOS, G. W. S. Apoio Matricial e Atenção Primária em Saúde. Saúde Soc., v. 20, n. 4, p. 961-970, 2011. DOI: https://doi.org/10.1590/S0104-12902011000400013.
» https://doi.org/https://doi.org/10.1590/S0104-12902011000400013. - MELO, D. F.; CRISCUOLO, M. B. R.; VIEGAS, S. M. F. Referência e contrarreferência no cotidiano da atenção à saúde de Divinópolis-MG, Brasil: o suporte às decisões da atenção primária. Rev Fund Care Online, v. 8, n. 4, p. 4986-4995, 2016. DOI: http://dx.doi.org/10.9789/2175-5361.2016.v8i4.4986-4995
» https://doi.org/http://dx.doi.org/10.9789/2175-5361.2016.v8i4.4986-4995 - MENDES, E. V. As redes de atenção à saúde. Ciênc. saúde coletiva, Rio de Janeiro, v. 15, n. 5, ago. 2010. Available at: <http://www.scielosp.org/pdf/csc/v15n5/v15n5a05.pdf>. Retrieved: 22 May 2020.
» http://www.scielosp.org/pdf/csc/v15n5/v15n5a05.pdf - MENDES, E. V. As redes de atenção à saúde. Organização Pan-Americana da Saúde, Brasília, 2011. Available at: <http://www.saude.sp.gov.br/resources/ses/perfil/gestor/documentos-de-planejamento-em-saude/elaboracao-do-plano-estadual-de-saude-2010-2015/textos-de-apoios/redes_de_atencao_mendes_2.pdf>. Retrieved: 22 May 2020.
» http://www.saude.sp.gov.br/resources/ses/perfil/gestor/documentos-de-planejamento-em-saude/elaboracao-do-plano-estadual-de-saude-2010-2015/textos-de-apoios/redes_de_atencao_mendes_2.pdf - PAULA JÚNIOR, W. M. P.; RIZZON, A. M.; MACHADO, C. J. Comunicação entre profissionais como ferramenta para cuidado integral aos pacientes oncológicos por mecanismos de referência e contrarreferência. Revista Interdisciplinar Ciências Médicas, v. 2, n. 2, p. 1-5, 2018. Available at: http://revista.fcmmg.br/ojs/index.php/ricm/article/view/90 Retrieved: 22 May 2020.
» http://revista.fcmmg.br/ojs/index.php/ricm/article/view/90 - PAULA, S.H.B.; VOLOCHKO, A.; FIGUEIREDO, R. Linha de cuidado de câncer de mama e de colo de útero: um estudo sobre referência e contrarreferência em cinco regiões de saúde de São Paulo, Brasil. Boletim do Instituto de Saúde, v. 17, n. 2, p. 146-166, 2016. Available at: https://pesquisa.bvsalud.org/portal/resource/pt/ses-34272 Retrieved: 20 May 2020.
» https://pesquisa.bvsalud.org/portal/resource/pt/ses-34272 - PEREIRA, J. S.; MACHADO, W. C. A. Referência e contrarreferência entre os 1033 serviços de reabilitação física da pessoa com deficiência: a (des)articulação na microrregião Centro-Sul Fluminense, Rio de Janeiro, Brasil. Physis: Revista de Saúde Coletiva, v. 26, n. 3, p. 1033-1051, 2016. DOI: http://dx.doi.org/10.1590/S0103-73312016000300016.
» https://doi.org/http://dx.doi.org/10.1590/S0103-73312016000300016. - SILVA, K. A. B. et al. Desafios no processo de referenciamento de usuários nas redes de atenção à saúde: perspectiva multiprofissional. Cienc. Cuid. Saúde, v. 17, n. 3, p. 1-8, 2018a. DOI: 10.4025/cienccuidsaude.v17i3.43568.
» https://doi.org/10.4025/cienccuidsaude.v17i3.43568. - SILVA, M. V. S.; MIRANDA, G. B. N.; ANDRADE, M. A. Sentidos atribuídos à integralidade: Entre o que é preconizado e vivido na equipe multidisciplinar. Interface: Communication, Health, Education, v. 21, n. 62, p. 589-599, 2017. DOI: https://doi.org/10.1590/1807-57622016.0420.
» https://doi.org/https://doi.org/10.1590/1807-57622016.0420. - SILVA, M. F. F. S. et al. Integralidade na atenção primária à saúde. Revista Família, Ciclos de Vida e Saúde no Contexto Social, v. 1, supl., p. 2318-8413, 2018b. DOI: https://doi.org/10.18554/refacs.v6i0.2925.
» https://doi.org/https://doi.org/10.18554/refacs.v6i0.2925. - TORRALBO, F. A. P.; JULIANI, C. M. C. M. O sistema de referência e contrarreferência no atendimento ao adolescente: realidade x integralidade. Rev enferm UFPE online, v. 10, n. 3, p. 1016-21, 2016. DOI: 10.5205/reuol.8702-76273-4-SM.1003201610.
» https://doi.org/10.5205/reuol.8702-76273-4-SM.1003201610.
Publication Dates
- Publication in this collection
12 Apr 2021 - Date of issue
2021
History
- Received
05 June 2020 - Accepted
07 Dec 2020 - Reviewed
20 Jan 2021