Abstract

The National Curriculum Guidelines for achieving a medical degree value active teaching methodologies and the application of new teaching skills in the current setting. In this context, we consider that evaluation of teachers by students is an important tool for the development of education. Therefore, we aimed to identify students perceptions about the skills of medical school faculty of the Federal University of Amapá (UNIFAP) through the implementation of a cross-sectional and qualitative research from four focus group sessions, attended by 28 volunteer students from the first, second and third year of the course. Sessions were recorded and the content was analyzed in two stages using the Wordle.net platform and Bardin technique. Focal groups discussed five main topics: faculty teaching skills; academic planning; faculty skills evaluation by students; theory-practice integration and the teaching-learning process. In general, it was observed that all the students had similar ideas, but those with more years in the course showed greater mastery of subjects. Worth highlighting were the need to improve essential aspects of medical training and the importance of constant evaluation of this process.

Key words
Educational evaluation; Medical education; Medical faculty; Problem-based learning

Introduction

In 1990, in Brazil, medical schools distance from social requirements and professional medical graduates’ need for higher quality were used as a pretext to establish the Interinstitutional Commission for the Evaluation of Medical Education (CINAEM), which identified deficiencies in the conventional curriculum and in the current medical graduation process. Due to the problems highlighted, in 2001, the National Curricular Guidelines (DCNs) of Medicine were created and aimed to train professionals with skills related to comprehensive care provided to patients before the new technological setting and the development of the Unified Health System (SUS), as well as recommend the use of active teaching methodologies, favoring the development of students’ skills necessary to improve medical practice¹1. Rêgo C, Batista, HS. Desenvolvimento docente nos cursos de Medicina: um campo fecundo. Rev Bras Educ Med 2012; 36(3):317-324..

In this new teaching perspective, faculty involved in the teaching-learning process assumes different positions when comparisons are made with the teaching practices required in a traditional curriculum, since DCNs establish that they should act as learning mediators and facilitators. In order to comply with their new duties, educators must develop cognitive and non-cognitive skills, among them content mastery, good communication with students, didactics, commitment, research involvement, constant updating, organization, self-criticism, ability to provide feedback to students and to embed theory and practice²2. Costa NMSC, Cardoso, CGLV, Costa DC. Concepções sobre o bom professor de medicina. Rev Bras Educ Med 2012; 36(4):499-505..

In this context, Problem-Based Learning (PBL) is characterized by the development of students’ ability to actively build their learning, relating their previous knowledge to health-disease problems submitted for the study, using critical thinking, communication skills and other competences required to solve clinical cases³3. Barrows HS, Tamblyn RM. Problem-based Learning: An Approach to Medical Education. New York: Springer Publishing Company; 1980.. Thus, PBL has been an important tool for the education of health professionals since the second half of the twentieth century, breaking with the paradigm of the educational focus, moving away from the teacher-centered to the student-centered rationale⁴4. Negarsikar PN. From student to tutor in Problem Based Learning: An unexplored avenue. British Journal of Medical Practitioners 2010; 3(2):2-4..

Regarding this theme, Franco et al.⁵5. Franco CAGS, Cubas MR, Franco RS. Currículo de medicina e as competências propostas pelas diretrizes curriculares. Rev Bras Educ Med 2014; 38(2):221-230. understand that in order to materialize the expected changes in the new medical curricula, routine evaluations should be carried out regarding the way in which the subjects involved in this process – students, teachers and society – discuss and implement the curricular reform. Related to this argument, it is accepted that educators must be evaluated by students in order to improve the pedagogical practice in active methodologies. This practice has a great impact on medical education, since it allows establishing the educational diagnosis, since it clarifies lags in the teaching practice and allows the formulation of resolutive proposals for the problems found⁶6. Almeida BC. Avaliação do docente pelo discente: análise das percepções de utilização ideal e efetiva [dissertação]. Salvador: Universidade Federal da Bahia; 2014..

Before this reality, we understand that the development of devices that can evaluate faculty teaching skills works as an important tool for educational diagnosis. Thus, this study was developed with the purpose of identifying the perceptions of students of different course years of the medical course of the Federal University of Amapá (UNIFAP) about the teaching competences of their teachers. The research in question will be used later as a source of information for the construction of faculty evaluative tools, which will allow the identification of positive and negative aspects of the teaching practice, contributing to the accomplishment of measures that can improve the teaching-learning process and provide effective feedback to university medical faculty.

Methodology

Study outline

This is a qualitative research developed with students of the medical course of the Federal University of Amapá (UNIFAP). Data was collected during the first and second semester of the 2015 academic year. The project was submitted to the Research Ethics Committee (CEP) of UNIFAP, according to Plataforma Brasil (Brazilian Platform) procedures and was approved without restrictions.

Participants

The sample consisted of six students from the first year, fifteen from the second year and seven from the third year of the Medical School, totaling 28 volunteers. Inclusion criteria were to be students enrolled in the first, second or third year of the Medical School and have signed an informed consent form.

Procedures

Initially, first through third year students were invited to participate in the research at hand. In order to obtain students’ perception about the subject under study, focus groups – a technique characterized by group integration that allows wide discussion and problem-based learning on a specific theme – were held in the Medical School building of UNIFAP.

Thus, there were four focus group sessions in the first half of 2015 with six and seven members of the first and third year, respectively, and two in the second semester with seven and eight members of the second year, in which structured questions were used according to a previously established roadmap.

Data collection tool

The questionnaire implemented in the focus group sessions was structured into three categories of questions: opening question (“What is your perception about quality medical education?”), thematic development questions (“In your student's opinion, how could the teacher improve the teaching-learning process in the different areas of the course that adopts the PBL methodology?” / “In your student's perception, how would you verify that the teacher improved or developed subjects related to the teaching-learning process in the different areas of the course that adopt the PBL methodology?” / “Could you figure out a systematized method for evaluating teachers on the teaching-learning process?” / “How would it work?”) and a final question (“What would the contributions of the evaluation tool to the development of the teaching-learning skills and professional skills necessary for good medical practice be?”).

Data analysis tool

After full transcription of data collected in the focus group sessions, the recorded content was interpreted in two stages: in the first one, we used the Wordle.net platform to identify the words most used by the members of the focal group; in the second, Bardin's⁷7. Bardin L. Análise de Conteúdo. Lisboa: Edições 70; 2009. technique was observed for content analysis and identification of emerging themes in group discussions.

According to the methodological design selected for this research, following identification of keywords obtained based on the developed discussion, we searched for sentences in which they were inserted and selected those related to the topic under study. These sentences have been grouped into thematic realms to facilitate the interpretation of the information. In addition, we followed the ethical principles of the Declaration of Helsinki⁸8. World Medical Association. Declaration of Helsinki: Recommendation guiding physicians in biomedical research involving humans subjects. JAMA 1997; 277(11):925-926., preserving the confidentiality of the sources of information and respecting the guidelines on Ethics in Research involving Human Beings in accordance with the provisions of Resolution N° 466 of December 12, 2012/Ministry of Health⁹9. Brasil. Resolução n° 466, de 12 de dezembro de 2012. Aprova normas regulamentadoras de pesquisas envolvendo seres humanos. Diário Oficial da União 2013; 13 jun..

Results and discussion

The analysis of results showed the emergence of five themes related to students’ perceptions obtained through the focus group: Faculty Teaching Skills, Academic Planning, Faculty Skills Evaluation by Students, Integration of Course Activities and Development of Education (Chart 1).

Faculty teaching skills

The freshmen focus group highlighted the need for the teacher to provide students with a feedback on their performance as a way to improve the teaching-learning process, as evidenced in the following excerpt:

I think it would be really interesting if the tutor arrived and […] not only provided the grade […] he should clarify what skills students need to develop during the tutorial session (E1.1)

Similar ideas were observed in the sophomore focus group:

I need teachers to provide me with a feedback. Where did I not perform well? Where did I go well? How will I know this if they do not speak? (E2.1)

In turn, in the third-year focus group, students emphasized that teachers need to establish effective communication with students:

There are teachers who value this contact, this conversation, they try to know more about the student and put their soul into it. (E3.1)

There's lack of maturity of them knowing how to listen to us and us how to listen to them (E3.2)

These ideas were discussed by authors such as Costa et al.²2. Costa NMSC, Cardoso, CGLV, Costa DC. Concepções sobre o bom professor de medicina. Rev Bras Educ Med 2012; 36(4):499-505., who affirm that the development of the teaching-learning process relies directly on the relationship established between faculty and students. They argue that flexible, patient and receptive teachers are able to make the academic environment more pleasant, thus allowing greater participation of students in the development of teaching. Other authors such as Dent and Harden¹⁰10. Dent J, Harden RM. A practical guide for medical teachers. 4th. Toronto: Elsevier Health Sciences; 2013. argue that teachers within the PBL methodology should be able to encourage group interaction, contribute to the discussion by instigating critical thinking of students – helping them to relate theoretical knowledge to practical aspects – and provide them with effective feedback that contributes to the improvement of group performance.

Another aspect discussed by the focus groups was addressed by authors like Borges et al.¹¹11. Borges MC, Miranda CH, Santana RC, Bolella VR. Avaliação formativa e feedback como ferramenta de aprendizado na formação de profissionais da saúde. Medicina (Ribeirão Preto. Online) 2014; 47(3):324-331. who consider that the feedback offered by teachers to students can develop as a structural practice of the educational evaluation, providing improvement of students’ learning with the improvement of their abilities. Thus, it is considered that feedback should point out positive and negative aspects in a specific way about students’ performance.

In addition, freshmen recognized the need for the tutor to constantly improve, as follows:

Some tutors are really interested in improving and realizing that maybe something is missing from their tutorial […] that is very important for our medical training (E1.2)

On the other hand, second-year considered that a teacher has a differentiated role in an active methodology:

In PBL, especially, teachers will lose some of the central figure they had in the traditional method. I particularly see teachers as mentors; they will guide you during your graduation, since they have already gone through all that. I think that this is indispensable in a quality medical education.(E2.2)

Another aspect emphasized by third-year students is the need for teachers to develop appropriate skills according to their knowledge areas within the course - Morphofunctional, Tutorial, Teaching-Service-Community Integration (IESC), Clinical Skills and Conferences –that are integrated, but require specific approaches on the part of teachers.

And for teachers to be better, they had indeed to take courses; they had to study as we do… There must be training and updating. (E3.3)

I think what each teacher can improve; it will depend on each area (E3.4)

Continuous learning, as addressed by focus groups, is an essential teaching competence according to Zhu et al.¹²12. Zhu C, Wang D, Cai Y, Engels N. What core competencies are related to teachers’ innovative teaching? Asia-Pacific Journal of Teacher Education 2013, 41(1):9-27.. This competence is related to teachers’ ability to constantly update themselves in order to improve their skills and improve the educational process. Thus, Moura and Oliveira¹³13. Moura ATMS, Oliveira AV. Aprimoramento curricular na perspectiva do desenvolvimento docente-desafios e possibilidades. Rev HUPE 2014, 13(4):73-88. point out that teachers’ development programs are important devices for raising teachers’ awareness about the need to commit to medical education and improve their teaching techniques, since in new methodologies, teachers abandon their central role of knowledge holder and begin to act as learning facilitators.

In relation to faculty's specific characteristics in the PBL method, Jusoh et al.¹⁴14. Jusoh AJ, Isa NJM, Razali MMSM. Problem-Based Learning: Mandatory Personal Qualities of Effective Facilitators. International Journey of Learning, Teaching and Educational Research 2015; 13(2):88-96. emphasize that teachers must develop several skills to be good tutors, which involve both the mastery of the content discussed and the ability to deal with the dynamics of the tutorials, adapting to the new educational reality. This aspect becomes crucial, since the same authors point out that the lack of preparation of teachers that work in the PBL methodology results in failures in the process of conducting learning and course activities, interfering in the quality of the students’ academic education and in the development of teaching.

Academic planning

All focus groups addressed the importance of academic planning for the development of teaching, considering that the way in which activities are organized and carried out directly influences this process, as observed in the following reports of first year students:

I would say … that quality medical education is characterized by providing all the necessary conditions for the individual to learn at the right moments… without overburdening or lack of activities (E1.3)

I think lessons should be better distributed […] (E1.4)

On the other hand, sophomores emphasized that the difficulties of academic planning are a reality in the course and may be related to factors such as the limited amount of teachers, inadequate workload and poor teaching-extension integration. The following excerpts discuss such information:

We need more teachers of skills … and enough workload to take a proper approach. (E2.3)

There are many problems with project development and these works contribute and bring resources to our course (E2.4)

In line with the idea provided by focus groups, Zuljan¹⁵15. Zuljan MV, Peklaj C, Pecjak S, Puklek M, Kalin J. Didactic competencies of teachers from the learner's viewpoint. Educational studies 2012; 38(1):51-62. states that PBL requires continuous planning of the activities, requiring constant interactions between faculty in charge of different fronts, so that they occur in a sequenced and integrated way. In addition, Garcia and Silva¹⁶16. Garcia MAA, Silva ALB. Um perfil do docente de medicina e sua participação na reestruturação curricular. Rev Bras Educ Med 2011; 35(1):58-68. mention that academic planning must be combined with adequate infrastructure, for the effective development of activities. In general, it is understood that planning works as an important tool that provides students with a meaningful learning, because contents, deadlines and goals that will guide the activities developed are established and coordinated from it.

Another aspect mentioned by focus groups was that the course had to include in its pedagogical plan a greater integration in teaching-research-extension activities. Within this theme, authors such as Mota et al.¹⁷17. Mota A, Carvalho B, Candido L, Lomanto R, Maia T. Exame do Cremesp como Indicador da Qualidad e do Ensino Médico. Rev Bras Educ Med 2014; 38(1):150-159. discuss the importance of extracurricular activities for the academic life of students and their professional future, emphasizing that the course should provide adequate support in order to give students the opportunity of developing in other activities that complement their training.

Faculty evaluation

Freshmen emphasized the importance of providing educational feedback to teachers as a way to contribute to the improvement of their skills. They say that this would help teachers to identify their shortcomings and solve them, as well as to increase faculty-student bond. Addressing the issue of the evaluation method, it was pointed out that the best way to evaluate faculty would be through a structured questionnaire with objective and subjective questions:

Observing him in class, in activities and evaluations … and observing grades at the end, if he improved his evaluation … if he was more judgmental and fairer. (E1.5)

I believe that the idea of the questionnaire is ideal, […] we would score good, fair or bad, and at the end we would make observations, such as where do you believe that they can improve. (E1.6)

The same idea was advocated by sophomores on this subject. However, they say that students’ perceptions should be related to the activities developed in the module:

What I believe could be done at the end of each module is that we could sit down with all the teachers of the year and the students who wanted to go or all of them. (E2.5)

Third-year students added that evaluation should occur more than once, so that it would be possible to verify changes in the teachers’ performance after the analysis and dissemination of the initial perceptions of students about their pedagogical competences. In addition, it would be possible to establish the efficacy of the evaluative method as a device to promote improvement of teachers:

Thinking about this evaluation … it would be interesting if it were done more than once. We should have an evaluation at the beginning and an evaluation at the end of the year, to know whether they have changed and whether this had any effect or not. (E3.5)

In general, the three focus groups emphasized the need for such an assessment to take place anonymously to ensure information reliability:

There should not be a student's name there, […] the tutor would feel bad and the student would feel embarrassed. (E1.7)

Evaluative methods are well described in literature by authors such as Paquay et al.¹⁸18. Paquay L, Nieuwenhoven CV, Wouters P. A avaliação como ferramenta de desenvolvimento profissional de educadores. Porto Alegre: Penso; 2012., who state that they can be used for different purposes, including controlling, judging and/or validating actions. In general, the evaluation establishes how much improvement has to be achieved to achieve an objective. Discoursing with the ideas advocated by the focus groups, Almeida⁶6. Almeida BC. Avaliação do docente pelo discente: análise das percepções de utilização ideal e efetiva [dissertação]. Salvador: Universidade Federal da Bahia; 2014. affirms that the educational feedback provided to faculty following students’ evaluation works as a reliable informative source about teachers’ pedagogical performance.

Dubi et al.¹⁹19. Dubi AY, Becker D, Tekian A. A workshop in feedback improves learning and changes the teaching culture. Medical Education 2015; 49(5):534-535. argue that the evaluation of students increases the bond between them and teachers, who become more willing to receive criticism, while at the same time being more committed to improving their teaching methods, after realizing that students recognize their efforts. Likewise, authors report in their studies that students have a greater participation in faculty evaluation when they perceive that opinions and feedbacks given to teachers were considered, contributing to improvements in their education.

Higa et al.²⁰20. Higa EFR, Hafner MLMB, Tonhom SFR, Taipeiro EF, Moreira HM, Guimarães APC, Pinheiro OL. Indicadores de avaliação em gestão e saúde coletiva na formação médica. Rev Bras Educ Med 2013; 37(1):52-59. show some characteristics that the evaluative method must have in order to work as an effective tool in the search for improved teaching competences, among which are being continuous, including all stakeholders of the teaching-learning process, generating investigations on the educational diagnosis and lead to reflections that may contribute to improved teaching-learning process.

In addition, Keane and Labhrainn²¹21. Keane E, Labhrainn IM. Obtaining student feedback on teaching & course quality. Briefing Paper 2005; 2: 1-19. argue that student assessments are generally reliable and valid, but that there is a need to ensure students’ anonymity and teachers’ absence during the provision of any feedback mechanisms to ensure a safer and more effective evaluation.

Theory-practice integration

An aspect strongly emphasized by the freshmen focus group was the need to combine theory and practice for a quality teaching-learning process. According to them, medical practice requires cognitive aspects and psychomotor skills that allow them to use the knowledge acquired during graduation in a safe way in the professional reality. In addition, students emphasized that effective practical activities contribute to the assimilation of theoretical knowledge, so it is fundamental that both develop in an integrated way:

What we really want is to learn theory content that can be safely implemented during future practice of our profession. (E1.8)

Practice is one of the fundamental things, studying theory in university and associating it with practice will make us absorb more knowledge and apply it. (E1.6)

Sophomores also advocated this idea and added that practice and theory should occur at appropriate times, suitably organized for a full approach to content and its best fixation:

[…] when we get back to it, I have the feeling that I was not able to fix it because I did not practice the subject. (E2.6)

The third-year focus group emphasized the importance of practice being included in the first years of graduation to produce a quality medical education, and that the latter should be fully linked to other fronts of the module.

If we are studying a module on women's health, we should at least see a pregnant woman. (E3.6)

This idea has been discussed by authors such as Cezar et al.²²22. Cezar PHN, Guimarães FT, Gomes AP, Rôças G, Siqueira-Batista R. Transição paradigmática na educação médica: um olhar construtivista dirigido à aprendizagem baseada em problemas. Rev Bras Educ Med 2010; 34(2):298-303., who point out that, in addition to cognitive knowledge, it is necessary to develop practice in laboratories or PHC facilities throughout academic training, since they are the “pillars of the course”. Authors point out that laboratory-acquired practice facilitates the development of knowledge of basic disciplines such as anatomy, physiology and histology, while practice in health facilities brings students closer to their functioning dynamics and patients’ biopsychosocial reality, contributing to improved education.

Likewise, Sacchetim et al.²³23. Sacchetim SC, Major CR, Mendanha DBA, Teles AA, Souza FHC, Sandes MTS. Percepção docente da aprendizagem baseada em problemas - medicina UniEVANGÉLICA. Rev Educação em Saúde 2012; 1(1):10-17. argue that early contact with practice is a fundamental aspect to the improvement of medical education, leading to the training of professionals who are critical and more prepared for the professional reality. In addition, practice also favors the learning process, allowing knowledge construction via the action-reflection-action process.

Costa et al.²2. Costa NMSC, Cardoso, CGLV, Costa DC. Concepções sobre o bom professor de medicina. Rev Bras Educ Med 2012; 36(4):499-505. report that teachers have to establish faculty negotiations and constant planning to achieve theory-practice integration.

The teaching-learning process

One of students’ most addressed aspects in relation to teaching development was the excessive emphasis given to tutorials to the detriment of other PBL teaching fronts. They say that, while tutorials are the method's backbone, it is necessary to have a greater faculty commitment to structure and integrate all the activities developed, regardless of the academic modality, so that the teaching-learning process is effective.

In addition, third-year students added that evaluations at the end of the module overestimate the contents covered in the tutorials in relation to those discussed in the other areas, revealing gaps in the importance given by faculty to the different fronts of the course, besides lacking contact between tutors and other teachers before the preparation of the tests:

As much as it is a PBL course, whose flagship is the tutorial, the course cannot be just tutorial-based. I think the other fronts of the course are unsatisfactory. (E 2.7)

It can be done. And even in the discourses, you could approach them all with a, b c and d, both from the tutorials and the other fronts… But I don't think I've ever seen such a split exam (E 3.7)

Almeida and Batista²⁴24. Almeida EG, Batista NA. Desempenho docente no contexto PBL: essência para aprendizagem e formação médica. Rev Bras Educ Med 2013; 37(2):192-201. advocate the same line of thought of the students, stating that it is necessary to have interdisciplinarity between the activities developed by the course, so that curriculum offered is comprehensive, considering that all areas are important to medical training. Thus, faculty plays a decisive role, since by means of communicative and social skills they can facilitate the establishment of this interaction between disciplines, as well as their equivalent development, avoiding that varying levels of importance are assigned to the fronts involved during academic education.

Despite all the mentioned aspects, it is worth emphasizing the importance of further studies to evaluate other aspects related to the subjects discussed, since the study in question has its own methodological limitations, as it is a qualitative research carried out through the focus group technique, in which sample size is made for convenience (students were invited to participate).

While statistical representativeness is not an adequate parameter in qualitative research, it is difficult to generalize findings for certain groups because the subject needs to be studied within a context and it is difficult to compare results with other studies, since not all have the same methodological design.

Conclusions

This study allowed the analysis of students’ perceptions from different academic levels – first, second and third-year course – on topics related to medical education, faculty's teaching skills and evaluation of these skills. While students showed varying degrees of educational maturity, it was perceived that the ideas advocated addressed similar aspects, differing, in short, from the details found in the arguments used, since participants of the more advanced series evidenced a greater capacity to discourse about more specific themes and mastered them, showing even a differentiated relation with regard to grades, for example - theme valued in the first years, but of secondary importance compared to learning over time for students of more advanced course years.

The focus group technique favored discussion in the form of dialogue, contributing to the flow of ideas, which were reformulated and improved throughout the session. In addition, the opportunity for interaction among students provided the building of critical and effective solutions to the problems indicated.

Thus, reflections of the academics geared to faculty teaching skills focused on teachers seeking constant improvement in order to adapt their methods to the new curricular requirements advocated by the DCNs. In addition, it was emphasized that the educational feedback by teachers to students is a tool to promote the students’ abilities and the need for interaction among faculty of the different areas of the graduation to implement the academic planning was considered.

Students understood that course activities must be sequenced and integrated, and planning operates as a tool to guide the development of teaching. Thus, they emphasized that practical activities should be integrated to theoretical ones, allowing students to apply the cognitive knowledge acquired in situations of the medical reality. Thus, tutorial sessions that provide most of the theoretical knowledge need to take place in an integrated way and with the same degree of pedagogical valorization in relation to the other fronts of the course, which are responsible for the practical skills necessary to a good medical education.

On the other hand, there was mention to the importance of a constant evaluation of faculty, which should allow an educational feedback to educators, indicating their weaknesses and potentialities, which would contribute to the implementation of solutions to the problems referenced in the teaching-learning process.

We considered data of the specialized literature in relation to the fact that the curricular implementation of new methodologies seeks to favor the formation of medical professionals who can meet the new social and technological requirements. Therefore, it is understood that the understanding of the perceptions of the various stakeholders involved in the teaching-learning process can contribute to the improvement of medical education, since it allows the diagnosis of the educational reality and subsidizes the future establishment of faculty evaluation platforms that can provide the skills required by the new teaching methods.

Thus, from the information collected in this study, it is possible to set, develop and follow the process of implementing a system for the evaluation of academic activities to be used by students, based on the characteristics evidenced by the focus groups, including ensuring student secrecy, the possibility of constructively criticizing faculty, identification of positive points related to academic activities and assistance in the organization of classes in the different learning settings. We presume that future studies capable of evaluating the results provided by the feedback system can contribute to validate the method, establishing the opportunity for wider implementation in the medical academic setting.

References

Publication Dates

History