Abstract

The research aimed to assess the effectiveness of blended learning in the Medical Residency Programs in Family and Community Medicine and Multiprofessional Residency in Family Health at the SESAU/Fiocruz. A cross-sectional observational study was conducted, utilizing a mixed-methods approach with the Constructivist Online Learning Environment Survey (COLLES) among enrolled residents in the first and second years. The study included 88 residents who showed a high level of agreement regarding the relevance of the programs to their professional practices, highlighting significant challenges in the process. Implementing blended learning emerged as an innovative and effective strategy in Residency Programs, evidencing a positive connection between theory and practice. Despite challenges in interactivity, the results suggest the need for strategies to enhance collaboration among residents, streamlining the learning experience. The use of blended learning is a promising tool in Residency Programs, contributing significantly to professional development.

Key words:
Information Technology; Distance Education; Internship and Residency; Educational Technology

Introduction

Health Residency Programs are lato sensu specializations based on in-service training, emphasizing the teaching-learning process, with the primary objective of training health professionals to work effectively in the Unified Health System (SUS). Noteworthy is the difference in the workload of these Programs against other specializations since they require a 60-hour weekly dedication for developing theoretical, theoretical-practical, and practical activities, totaling 5,760 hours at the end of the specialization¹1 Brasil. Lei nº 6.932, de 7 de julho de 1981. Dispõe sobre as atividades do médico residente e dá outras providências. Diário Oficial da União 1981; 9 jul.^,22 Brasil. Lei nº 11.129, de 30 de junho de 2005. Institui o Programa Nacional de Inclusão de Jovens - ProJovem; cria o Conselho Nacional da Juventude - CNJ e a Secretaria Nacional de Juventude; altera as Leis nº s 10.683, de 28 de maio de 2003, e 10.429, de 24 de abril de 2002; e dá outras providências. Diário Oficial da União 2005; 1 jul..

Approximately 80% of the total workload of developing this postgraduate modality are practical and theoretical-practical educational strategies, and 20% comprise theoretical educational strategies³3 Brasil. Resolução nº 5, de 7 de novembro de 2014. Dispõe sobre a duração e a carga horária dos programas de Residência em Área Profissional da Saúde nas modalidades multiprofissional e uniprofissional e sobre a avaliação e a frequência dos profissionais da saúde residentes. Diário Oficial da União 2014; 10 nov.. Considering the relevance of the training process in Health Residency Programs and their significant workload, new technologies, and different teaching-learning methods have been proposed in the health field, emphasizing blended learning⁴4 Lima ACB, Santos DCM, Almeida SL, Silva EL, Pereira EBF. Ensino híbrido na formação em saúde: uma revisão sistemática. Rev Cuidarte 2022; 13(1):e2051..

Blended learning is a teaching model that uses traditional and online teaching methods, targeting a student-centered teaching-learning process⁵5 Rovai AP, Jordan H. Blended learning and sense of community: a comparative analysis with traditional and fully online graduate courses. Int Rev Res Open Dis 2004; 5(2):1-13.^,66 Leinster SJ, Pereira JH, Down S, Simpson AD. Blended Learning in Healthcare Education. Med Res Arch 2021; 9(8):1-12.. According to Avelino⁷7 Avelino WF. Ensino híbrido: uma relação entre a avaliação e a prática docente. Rev Educ Publica [periódico na Internet] 2020; 20(42) [acessado 2023 nov 3]. Disponível em: https://educacaopublica.cecierj.edu.br/artigos/20/42/ensino-hibrido-uma-relacao-entre-a-avaliacao-e-a-pratica-docente.
https://educacaopublica.cecierj.edu.br/a... , blended learning is crucial in improving pedagogical practice to improve teaching in everyday life and enabling a growing reach of students innovatively and effectively. A systematic review published by Vallée et al.⁸8 Vallée A, Blacher J, Cariou A, Sorbets E. Blended learning compared to traditional learning in medical education: systematic review and meta-analysis. J Med Internet Res 2020; 22(8):e16504. showed that blended learning produced better results in the learning process in health courses than the traditional model. Corroborating these data, in Medicine, Zhang et al.⁹9 Zhang X, Wen H, Li H, Huang Y, Lv C, Zhu H. Effectiveness of blended learning on improving medical student's learning initiative and performance in the physiology study. Cogent Educ 2023; 10(1):2192150. observed that students had better grades in a blended learning model in the physiology discipline.

In 2020, considering this scenario, the Medical Residency Programs in Family and Community Medicine and Multiprofessional Residency in Family Health developed by the Municipal Health Secretariat of Campo Grande (SESAU), Mato Grosso do Sul (MS), Brazil, in partnership with the Integrated Health Care Territories Project of the Oswaldo Cruz Foundation (TEIAS/Fiocruz), was reformulated using a blended learning model to include theoretical educational strategies in an ongoing process. In this new model, residents work in person in real environments located in different areas of Campo Grande-MS and access theoretical classes in person and remotely through the Virtual Learning Environment (AVA) developed for the Programs. In the AVA, besides video classes divided into thematic learning modules, students must perform activities and participate in discussion forums to assess the gain in technical knowledge and stimulate critical and reflective thinking.

That said, we should imperatively consider the evaluation to qualify and effectively improve the courses offered¹⁰10 Oliveira AFP, Queiroz AS, Souza Júnior FA, Silva MCT, Melo MLV, Oliveira PRF. Educação a Distância no mundo e no Brasil. Rev Educ Publica 2019; 19(17) [acessado 2023 nov 20]. Disponível em: https://educacaopublica.cecierj.edu.br/artigos/19/17/educacao-a-distancia-no-mundo-e-no-brasil.
https://educacaopublica.cecierj.edu.br/a... ^,1111 Oliveira ET, Piconez SCB. Avaliação da educação superior nas modalidades presencial e a distância: análises com base no Conceito Preliminar de Cursos (CPC). Avaliação 2017; 22(3):833-851.. In this sense, this research aimed to evaluate the effectiveness of blended learning in the Medical Residency Programs in Family and Community Medicine and Multiprofessional Residency in Family Health developed by the Municipal Health Secretariat of Campo Grande (SESAU), Mato Grosso do Sul, in partnership with the Integrated Health Care Territories Project (TEIAS/Fiocruz).

Methods

This observational cross-sectional study with a mixed approach (qualitative and quantitative) was approved by the Human Research Ethics Committee of the Federal University of Mato Grosso do Sul (UFMS) (CAAE No. 75540023.6.0000.0021). The study included first- and second-year residents enrolled in the Medical Residency Programs in Family and Community Medicine and Multiprofessional Residency in Family Health SESAU/Fiocruz. Participation was voluntary and anonymous per the recommendations of the Informed Consent Form signed by each participant.

The Program’s Moodle virtual environment was used to teach the theoretical content, with a defined program content for residents in years 1 and 2 in 2023. Each theoretical module was based on a knowledge axis and consisted of video classes, a tutoring forum, and activities to be completed within a stipulated time frame. At the end, students had a synchronous meeting with tutors to clarify doubts and complete the module. Students took a theoretical test on school year completion to assess their knowledge of the topics covered throughout the year.

Data retrieved from the Moodle platform were used to evaluate the effectiveness of the developed method, such as engagement and participation in forums, participation and grades on activities, and student’s perceptions of the teaching process through the Constructivist Online Learning Environment Survey (COLLES) in the “effective experience” version¹²12 Taylor PC, Maor D. Assessing the efficacy of online teaching with the Constructivist On-line Learning Environment Survey. In: Proceedings of the 9th Annual Teaching Learning Forum [Internet]. Perth: Curtin University of Technology; 2000 [acessado 2023 dez 12]. Disponível em: https://clt.curtin. edu.au/events/conferences/tlf/tlf2000/taylor.html.^,1313 Mallmann EM. Gestão de cursos de capacitação em EaD: potencialidade das ferramentas de pesquisa e avaliação. In: Congresso Internacional ABED de Educação a Distância [Internet]. Fortaleza: ABED; 2009 [acessado 2023 dez 12]. Disponível em: http://www.abed.org.br/congresso2009/CD/trabalhos/452009163839.pdf.. This 24-item instrument with a variable 5-point scale is a diagnostic self-assessment tool for virtual learning environments that aims to assess students’ perceptions regarding relevance (whether it is meeting expectations), critical thinking (whether it was stimulated throughout the course), interactivity (presence or absence of interaction throughout the modules), tutor support (quality of support offered by tutors), peer support (if there was such support/contributions in the learning process), and understanding (related to understanding the messages sent and received during the learning process). Moreover, questions related to sociodemographic data and platform accessibility were added to the final questionnaire.

The data were tabulated in a Microsoft Office Excel spreadsheet (2007) and recorded as a database in the R statistical package version 4 (R Foundation for Statistical Computing, Vienna, Austria). Initially, we performed descriptive analyses of the categories assigned by the residents in each question of the six topics (Learning relevance, reflection, interactivity, tutor support, peer support, and understanding). To this end, we adopted absolute and relative frequencies of the responses. Subsequently, scores were assigned to each category. We assigned a score of 1 for responses “Almost never”, 2 for “Rarely”, 3 for “Sometimes”, 4 for “Frequently”, and 5 for “Often”. Then, we calculated the medians of the responses to the four questions in each topic and proceeded with descriptive analyses of the scores using quartiles. The nonparametric Friedman and Nemenyi tests were used to compare the topics with each other regarding the residents’ perception scores. The analyses were conducted using the R statistical analysis software, with a significance level of 5%.

Results

In 2023, 155 residents attended the Medical Residency Programs in Family and Community Medicine (n=70 students) and Multiprofessional Residency in Family Health (n=85 students) at SESAU/Fiocruz. Thirty-five participants (50%) were in the first year (R1) of the Medical Residency Program in Family and Community Medicine, and 35 (50%) were in the second year (R2). Regarding the Multiprofessional Residency Program in Family Health, 46 (54.1%) were in the first year (R1) and 39 (45.9%) in the second year (R2). The sociodemographic data for each year of the Residency Programs highlight the origin of the students: Mato Grosso do Sul between 70 and 80% for the Multiprofessional Residency Program in Family Health and approximately 50% for the Medical Residency Program in Family and Community Medicine (Table 1).

Regarding the perception of the teaching process, the 24 questions that make up the COLLES instrument were answered by 88 residents from the total number of registrants from both years and residencies (56.77%), with 52 (59.1%) residents linked to the Multiprofessional Residency Program and 36 (40.9%) residents linked to the Medical Residency Program. Regarding the Medical Residency Program in Family and Community Medicine, 17 (45.95%) respondents were from year 1 and 20 (54.05%) from year 2. Regarding the Multiprofessional Residency Program in Family Health, 29 (56.86%) respondents were from year 1 and 22 (43.14%) from year 2.

The results of this study showed that, overall, all participating students agreed that the Residency Programs were relevant to their professional practice (high frequency in the area of relevance). A similar result was also observed in understanding, in which residents showed that there was often a high level of understanding regarding communication between them (resident-resident) and between tutors (resident-tutor), and the interlocution between the parties was satisfactory. Interactivity and peer support recorded the lowest satisfaction (Figure 1).

We can observe in Table 2 that, regarding learning relevance, most residents reported that their learning is frequently or often focused on subjects that interest them (60.2%), that what they are learning is vital for the practice of their profession (80.7%), that they are learning how to improve their professional performance (69.3%), and that what they are learning is very much connected with their professional activity (72.8%). There was a statistically significant difference between the topics evaluated regarding the residents’ perception score (Table 2). The topics “Interactivity” and “Peer support” recorded significantly lower scores than “Learning relevance”, “Reflective thinking”, “Tutor support”, and “Understanding” (p<0.05). Moreover, the topic “Tutor support” received significantly lower scores than “Learning relevance” (p<0.05).

Regarding the suggestions, the recommendations included greater creativity, classes recorded on video and not just with the teachers’ voices, a concise approach to the topics, modules with greater content flexibility, and avoiding simultaneous modules. Regarding the criticisms, issues such as the delay in feedback on activities and the lack of objectivity in classes were highlighted.

Regarding the residents’ assessments of the content developed in the Virtual Learning Environment (AVA), which occur in a modular manner and an annual assessment, consolidating the themes addressed throughout the year, we should highlight the results. In the Multiprofessional Residency, first-year residents (R1) obtained an overall mean of 81.4 points in the annual assessment. In contrast, they achieved a mean of 88.7 points in the modular assessments proposed in the AVA. In contrast, second-year residents (R2) obtained a mean score of 66 points in the 2023 annual assessment and a mean of 64.7 points in the modular assessments developed throughout the AVA.

In Medical Residency, R1s recorded an overall mean of 83 points in the 2023 annual assessment and a mean of 80.2 points in the modular AVA assessments. On the other hand, R2s achieved an overall mean of 70 in the annual assessment and an overall mean in AVA activities of 74.1 points.

Discussion

The Integrated Health Care Territories Project of Campo Grande-MS (TEIAS) operates as a tool to develop technologies applicable to the work of professionals focused on the Family Health Strategy to simplify the routine of teams and provide training resources and support for the management of effective and low-cost technologies in the Unified Health System (SUS). Thus, it is based on increasing PHC resolution, streamlining financing, sustainability, transparency, compliance with attributes, and strengthening this care level¹⁴14 Ranzi DVM, Nachif MCA, Soranz DR, Marcheti PM, Santos MLM, Carli ADD. Laboratório de inovação na Atenção Primária à Saúde: implementação e desdobramentos. Cien Saude Colet 2021; 26(6):1999-2011..

This research addresses the inclusion and application of a blended learning model developed for the Medical Residency Programs in Family and Community Medicine and Multiprofessional Residency in Family Health, which are linked to the Municipal Secretariat of Campo Grande/MS and the TEIAS/Fiocruz Project. This model combines traditional teaching elements with online methods, an evolution in teaching for Residency Programs, aligned with the growing incorporation of new technologies and different teaching-learning methods⁴4 Lima ACB, Santos DCM, Almeida SL, Silva EL, Pereira EBF. Ensino híbrido na formação em saúde: uma revisão sistemática. Rev Cuidarte 2022; 13(1):e2051.

5 Rovai AP, Jordan H. Blended learning and sense of community: a comparative analysis with traditional and fully online graduate courses. Int Rev Res Open Dis 2004; 5(2):1-13.^-66 Leinster SJ, Pereira JH, Down S, Simpson AD. Blended Learning in Healthcare Education. Med Res Arch 2021; 9(8):1-12.. In this study, we observed that “Interactivity” and “Peer support” had significantly lower perception intensity scores than “Learning relevance”, “Reflective thinking”, “Tutor support”, and “Understanding” (p<0.05). The topic “Tutor support” received significantly lower perception intensity scores than “Learning relevance” (p<0.05).

Adopting blended learning through a student-centered approach⁵5 Rovai AP, Jordan H. Blended learning and sense of community: a comparative analysis with traditional and fully online graduate courses. Int Rev Res Open Dis 2004; 5(2):1-13.^,66 Leinster SJ, Pereira JH, Down S, Simpson AD. Blended Learning in Healthcare Education. Med Res Arch 2021; 9(8):1-12. in Health Residency Programs is innovative since it allows residents to have practical field experience while accessing theoretical content in a virtual learning environment, mixing pedagogical practices⁷7 Avelino WF. Ensino híbrido: uma relação entre a avaliação e a prática docente. Rev Educ Publica [periódico na Internet] 2020; 20(42) [acessado 2023 nov 3]. Disponível em: https://educacaopublica.cecierj.edu.br/artigos/20/42/ensino-hibrido-uma-relacao-entre-a-avaliacao-e-a-pratica-docente.
https://educacaopublica.cecierj.edu.br/a... , allowing flexibility in the management of the professional’s study time in their training process.

Several studies address qualification, assessment, and hybridization of teaching, covering research on elementary education, higher education, and postgraduate courses⁷7 Avelino WF. Ensino híbrido: uma relação entre a avaliação e a prática docente. Rev Educ Publica [periódico na Internet] 2020; 20(42) [acessado 2023 nov 3]. Disponível em: https://educacaopublica.cecierj.edu.br/artigos/20/42/ensino-hibrido-uma-relacao-entre-a-avaliacao-e-a-pratica-docente.
https://educacaopublica.cecierj.edu.br/a... ^,1010 Oliveira AFP, Queiroz AS, Souza Júnior FA, Silva MCT, Melo MLV, Oliveira PRF. Educação a Distância no mundo e no Brasil. Rev Educ Publica 2019; 19(17) [acessado 2023 nov 20]. Disponível em: https://educacaopublica.cecierj.edu.br/artigos/19/17/educacao-a-distancia-no-mundo-e-no-brasil.
https://educacaopublica.cecierj.edu.br/a... ^,1111 Oliveira ET, Piconez SCB. Avaliação da educação superior nas modalidades presencial e a distância: análises com base no Conceito Preliminar de Cursos (CPC). Avaliação 2017; 22(3):833-851.^,1515 Pereira JG, Rodrigues AP. O ensino a distância e seus desafios. Rev Cient Multidisc Nucleo Conhecimento 2021; 7(6):5-20.

16 Albuquerque O, Conceição MH, Melis MF, Martins AM, Albuquerque FM, Berbel N. Aplicação da Tecnologia Digital de Informação e Comunicação no Ensino Superior [Internet]. 2021 [acessado 2023 nov 20]. Disponível em: https://www.publi.ludomedia.org/index.php/ntqr/article/download/335/339.
https://www.publi.ludomedia.org/index.ph... ^-1717 Rocha Filho JCO, Adeodato JM. A Implantação do Ensino a Distância na Graduação em Direito é Uma Necessidade, Uma Inovação ou Um Retrocesso? Rev Int Consinter Direito 2022; 8(15):153-171., including Health Residency Programs¹⁸18 Guerra Y, Carrara V, Martins AMC. Ensino e formação virtuais: a nova estratégia do projeto de educação a serviço do capital. Rev Katálysis 2022; 25(3):570-584.^,1919 Pereira MC, Rocha CMF. A tutoria nas residências em saúde: um novo contexto de atuação de educação na saúde. Saberes Plurais Educ Saude 2018; 2(2):148-163..

The contribution of a qualification focused on in-service teaching proposed by the Multiprofessional Residency Programs in Family Health is still perceived empirically since there is no systematic evaluation of its effectiveness. In this sense, evaluating the Residency Programs becomes relevant as it can provide subsidies for orienting the health education policy¹⁹19 Pereira MC, Rocha CMF. A tutoria nas residências em saúde: um novo contexto de atuação de educação na saúde. Saberes Plurais Educ Saude 2018; 2(2):148-163., with consequent qualification of health training following the principles and guidelines of the Unified Health System (SUS).

As explained by some researchers⁸8 Vallée A, Blacher J, Cariou A, Sorbets E. Blended learning compared to traditional learning in medical education: systematic review and meta-analysis. J Med Internet Res 2020; 22(8):e16504.^,99 Zhang X, Wen H, Li H, Huang Y, Lv C, Zhu H. Effectiveness of blended learning on improving medical student's learning initiative and performance in the physiology study. Cogent Educ 2023; 10(1):2192150., blended learning education studies support its effectiveness, suggesting that this pedagogical method can provide superior performance in the teaching-learning process against traditional approaches, reinforcing the relevance of opting for the blended learning model in the Residency Programs in question.

Regarding the sociodemographic profile of residents enrolled in the Medical Residency Programs in Family and Community Medicine and Multiprofessional Residency in Family Health SESAU/Fiocruz, most are female professionals, with a predominant age range around 30, and most residents are born in the state of Mato Grosso do Sul. These data reflect a profile of young professionals entering specializations focusing on qualification in and for the Unified Health System.

As shown in the Tables 1 and 2 and Figure 1, this research’s results indicate, in general, that the participants consider that the Residency Programs as relevant for developing their professional practices, which is added by the fact that the high level of agreement of the residents regarding their learning is a positive indication for the use of the blended learning teaching model adopted in the Residency Programs object of this study, bringing new perspectives for training in this specialization model.

Despite the above, we should underscore that “Interactivity” and “Peer support” had lower levels of satisfaction, which can be attributed to the hybrid nature, where part of the interaction occurs online, and the methodological strategies employed, such as the few forum activities developed during the research analysis period. In this sense, developing strategies toward greater engagement and collaborative attitude among residents is crucial.

We should highlight some changes in the group of tutors who monitored the residents in the virtual learning environment in 2023, and even in this scenario, tutor support throughout the year was consistent and well-evaluated by the residents. Additionally, the high agreement of the residents regarding promoting reflection, encouraging participation, and improving the discourse’s quality may indicate the relevance and satisfaction of these residents with the tutors’ performance in the process.

The recommendations and notes presented by the residents at the end of the questionnaire provide critical support for the qualification of the blended learning model. The criticism regarding tutors’ response time for providing feedback and releasing grades highlights the relevance of greater efficiency in managing the virtual learning environment and the need for constant evaluation and adaptation of the model to guarantee its effectiveness, enhancing the residents’ training process.

Research on resident evaluations in the Multidisciplinary and Medical Residency Programs reveals a distinct trend in the effectiveness of blended learning among first- and second-year residents. First-year residents in both residencies performed consistently high on annual and modular assessments conducted in the Virtual Learning Environment (AVA), suggesting a positive adaptation to the blended learning environment.

However, the results of second-year residents were more varied, with those in the Multiprofessional Residency showing lower scores in both the annual and modular assessments in the AVA. At the same time, those in the Medical Residency achieved a slightly lower overall mean in the annual assessment against the AVA activities. We should highlight that such discrepancies in the results can be attributed to several factors, such as how the content is presented and structured between the Residency Programs and individual differences in adapting students to the blended learning environment.

These discrepancies highlight the relevance of careful monitoring and specific interventions for second-year residents to improve their adaptation and performance in this blended learning methodology proposed in these two Programs. Furthermore, the findings stress the need for continuing evaluation and adjustment of AVA planning, ensuring that it meets the needs and expectations of all professionals in training. Ultimately, this research provides valuable information for improving and providing blended learning in Residency Programs to ensure an effective and satisfactory learning experience, providing flexibility for residents in their teaching-learning process.

Conclusion

The blended learning format was effective and satisfactory, and it was possible to derive valuable insights for improving the training of health professionals in this specialization model. In general, with the high level of agreement regarding the relevance of the teaching-learning process for their professional practices, we can infer that adherence to the blended learning model stood out as a promising strategy, with a positive perception of residents regarding the connection between the content learned in the virtual learning environment (theory) and their daily professional activities (practice), reinforcing the ability of this method to apply theoretical concepts to practice.

The results of this research highlight the benefits and challenges of blended learning in residency programs. Although first-year residents benefit significantly from this model, it is vital to continue to improve processes and adapt teaching practices to meet the needs of all students, thus ensuring an effective and satisfying learning experience.

This research’s results can direct future implementations and updates in the teaching-learning models of Residency Programs. These initiatives can focus on stimulating more intense interaction between residents and fostering a collaborative and enriching learning environment. This approach will contribute to the individual growth of professionals and construct a more robust community of practice in the health environment. In this context, the opportunity for blended learning in Health Residency Programs stands out.

References

Publication Dates

History