ORIGINAL RESEARCH INVESTIGACIÓN ORIGINAL
A Spanish-language patient safety questionnaire to measure medical and nursing students' attitudes and knowledge
Un cuestionario en espanol sobre seguridad del paciente para medir las actitudes y los conocimientos de los estudiantes de medicina y enfermería
José J. MiraI; Isabel M. NavarroII; Mercedes GuilabertI; Rodrigo PobleteIII; Astolfo L. FrancoIV; Pilar JiménezV; Margarita AquinoVI; Francisco J. Fernández-TrujilloVII; Susana LorenzoVIII; Julián VitallerIX; Yohana Díaz de ValleX; Carlos AibarXI; Jesús M. AranazXII; José A. De PedroXIII
IDepartment of Health Psychology, Universidad Miguel Hernández de Elche, Elche, Spain. Send correspondence to: José Joaquín Mira, firstname.lastname@example.org
IIDepartment of Economic and Financial Studies,Universidad Miguel Hernández de Elche, Elche,Spain
IIIDepartment of Internal Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
IVFaculty of Public Health, Universidad del Valle, Cali, Colombia
VSchool of Law, Universidad de Salamanca, Salamanca, Spain
IVHospital de Ninos Benjamín Bloom, San Salvador, El Salvador
VIIDepartment of Human Anatomy and Embryology, Universidad de Cádiz, Cádiz, Spain
VIIHospital Universitario Fundación Alcorcón, Madrid, Spain
IXDepartment of Public Health, History of Science and Gynecology, Universidad Miguel Hernández de Elche, Elche, Spain
XPan American Health Organization, Guatemala City, Guatemala
XIDepartment of Public Health, Universidad de Zaragoza, Zaragoza, Spain
XIIHospital Universitario Ramón y Cajal, Madrid, Spain
XIIIDepartment of Surgery, Universidad de Salamanca, Salamanca, Spain
OBJECTIVE: To design and validate a questionnaire for assessing attitudes and knowledge about patient safety using a sample of medical and nursing students undergoing clinical training in Spain and four countries in Latin America.
METHODS: In this cross-sectional study, a literature review was carried out and total of 786 medical and nursing students were surveyed at eight universities from five countries (Chile, Colombia, El Salvador, Guatemala, and Spain) to develop and refine a Spanish-language questionnaire on knowledge and attitudes about patient safety. The scope of the questionnaire was based on five dimensions (factors) presented in studies related to patient safety culture found in PubMed and Scopus. Based on the five factors, 25 reactive items were developed. Composite reliability indexes and Cronbach's alpha statistics were estimatedfor each factor, and confirmatory factor analysis was conducted to assess validity. After a pilot test, the questionnaire was refined using confirmatory models, maximum-likelihood estimation, and the variance-covariance matrix (as input). Multiple linear regression models were used to confirm external validity, considering variables related to patient safety culture as dependent variables and the five factors as independent variables.
RESULTS: The final instrument was a structured five-point Likert self-administered survey (the "Latino Student Patient Safety Questionnaire") consisting of 21 items grouped into five factors. Compound reliability indexes (Cronbach's alpha statistic) calculated for the five factors were about 0.7 or higher. The results of the multiple linear regression analyses indicated good model fit (goodness-of-fit index: 0.9). Item-total correlations were higher than 0.3 in all cases. The convergent-discriminant validity was adequate.
CONCLUSIONS: The questionnaire designed and validated in this study assesses nursing and medical students' attitudes and knowledge about patient safety. This instrument could be used to indirectly evaluate whether or not students in health disciplines are acquiring and thus likely to put into practice the professional skills currently considered most appropriate for patient safety.
Key words: Health knowledge, attitudes, practice; patient safety; questionnaires; students; Chile; Colombia; El Salvador; Guatemala; Spain; Latin America.
OBJETIVO: Diseñar y validar un cuestionario de evaluación de las actitudes y los conocimientos en materia de seguridad del paciente con una muestra de estudiantes de medicina y enfermería que reciben formación médica en Espana y en cuatro países de América Latina.
MÉTODOS: En este estudio transversal se llevó a cabo una revisión bibliográfica y se encuestó a un total de 786 estudiantes de medicina y enfermería de ocho universidades de cinco países (Chile, Colombia, El Salvador, Espana y Guatemala) con objeto de elaborar y corregir un cuestionario en espanol sobre conocimientos y actitudes en materia de seguridad del paciente. El ámbito del cuestionario se basó en cinco dimensiones (factores) presentadas en estudios relacionados con la cultura de la seguridad del paciente encontrados en PubMed y Scopus. Con base en los cinco factores, se elaboraron 25 ítems reactivos. Se calcularon los índices de fiabilidad compuesta y alfa de Cronbach para cada factor, y se realizó un análisis factorial confirmatorio para evaluar la validez. Tras una prueba piloto se corrigió el cuestionario mediante modelos confirmatorios, el cálculo de la máxima probabilidad y la matriz de variancia-covariancia (como insumo). Se utilizaron modelos de regresión lineal múltiple para confirmar la validez externa, considerando las variables relacionadas con la cultura de seguridad del paciente como variables dependientes y los cinco factores como variables independientes.
RESULTADOS: El instrumento final fue una encuesta autoadministrada mediante escala de Likert estructurada en cinco puntos ("Cuestionario de Seguridad del Paciente para Estudiantes Latinos"), que consta de 21 ítems agrupados en cinco factores. Los índices de fiabilidad compuesta (prueba estadística de alfa de Cronbach) calculados para los cinco factores fueron aproximadamente de 0,7 o superiores. Los resultados de los análisis de regresión lineal múltiple indicaron un buen ajuste del modelo (índice de bondad de ajuste: 0,9). Las correlaciones ítem-total fueron superiores a 0,3 en todos los casos. La validez convergente y discriminatoria fue adecuada.
CONCLUSIONES: El cuestionario disenado y validado en este estudio evalúa las actitudes y los conocimientos de los estudiantes de enfermería y medicina en materia de seguridad del paciente. Este instrumento podría utilizarse para evaluar indirectamente si los estudiantes de disciplinas de la salud están adquiriendo, y por lo tanto, es probable que pongan en práctica, las habilidades profesionales consideradas actualmente como más apropiadas para la seguridad del paciente.
Palabras clave: Conocimientos, actitudes y práctica en salud; seguridad del paciente; cuestionarios; estudiantes; Chile; Colombia; El Salvador; Espana; Guatemala; América Latina.
Safety policies implemented since a 1999 Institute of Medicine (IOM) report ("To Err is Human") (1) and the subsequent international movement for patient safety led by the World Health Organization (WHO) have led to changes in clinical training practice (2). The key elements of this change in favor of a safer environment are health professionals' attitudes and ongoing education on patient safety (3).
Studies analyzing the culture of safety in health care organizations (4, 5) have identified various barriers to safer practices (6-8); the causes of adverse events (AEs), in the opinion of health professionals; and training needs (9).
Attitudes influenced by knowledge determine health professionals' behavior before and after the occurrence of an AE (10-12). The competencies needed by health professionals to carry out safer practices have been identified by WHO (13, 14). Halpern et al. (15) and Andermann et al. (16) have defined core competencies for good-quality practice and patient safety, and Okuyama et al. (17) have conducted a systematic review of existing tools designed to assess these competencies.
Although institutions and professional forums have conducted multiple awareness-raising and training activities on patient safety in medicine and nursing schools, they have not had the same impact as the aforementioned initiatives (18, 19). Only a few medical schools have implemented curricula addressing patient safety (20), and the majority of them are located in Canada, the United Kingdom, and the United States (20, 21). In most countries, training in patient safety typically occurs after completion of undergraduate studies. There is hardly any research on what medical and nursing students know and what their attitudes are toward AEs (22), even though, according to some authors (20, 23), up to 76% of them have witnessed a medical error (with varying levels of relevance) during their clinical practices.
Patey et al. (24) from the University of Aberdeen (Scotland) developed and tested a five-hour training module on patient safety for final-year students and analyzed training content in Scottish universities on patient safety. Their results showed that only clinical training instructors informed their students (sporadically) about the most common safety risks and the measures required to improve patient safety. Kern et al. (25) proposed curricula for medical education, and Seiden et al. (26) made recommendations about the content of medical school curricula. Fischer et al. (27) developed and proposed training content based on phone interviews with medical students and resident physicians in Massachusetts.
Flin et al. (28) assessed what medical students knew and what their attitudes were toward patient safety. The results of that study were used to design and validate the Medical Student Patient Safety Questionnaire. Additional survey instruments testing patient safety knowledge and attitudes have been developed and tested in other studies, mainly among residents and nurses (18, 29, 30), and students who have completed their clinical rotations (31). Several studies have explored nursing students' attitudes and coping styles after recognition of a medical error (28-30, 32).
The small amount of research conducted to date on medical and/or nursing students' knowledge and attitudes about medical errors has highlighted the need to strengthen the training in patient safety received by undergraduate students to enable them to address the risks inherent in clinical training practice. In Spain, the Quality Agency of the National Health System (Agencia de Calidad del Sistema Nacional de Salud, SNS) has proposed incorporating patient safety and risk management in undergraduate and postgraduate medical and nursing programs (33), and the Pan American Health Organization (PAHO) has developed an ambitious training program for health professionals in Latin American countries. However, only a few universities, such as the University of Málaga (nursing programs) and Miguel Hernández University (medical programs) in Spain, or the Pontifical Catholic University (medical programs) in Chile, have systematically addressed this issue through initiatives to fill gaps in patient safety training for undergraduate students.
To promote reflection on what has been accomplished thus far and facilitate future research, this study aimed to design and validate a questionnaire for assessing attitudes and knowledge about patient safety using a sample of medical and nursing students undergoing clinical training in Spain and four countries in Latin America. The new survey instrument (the "Latino Student Patient Safety Questionnaire") was designed and produced in Spanish for use in 1) evaluating how patient safety is being taught to Latino students in various health disciplines and 2) measuring progress.
MATERIALS AND METHODS
In this cross-sectional study a literature review was carried out and 786 medical and nursing students were surveyed at eight universities from five countries (Chile, Colombia, El Salvador, Guatemala, and Spain) to develop and refine a Spanish-language questionnaire on knowledge and attitudes about patient safety. The first version of the questionnaire was based on dimensions (factors) related to patient safety culture presented in studies found in PubMed and Scopus. Five factors were defined. The instrument was validated by administering it to a random study sample composed of medical and nursing students.
Design of the questionnaire
PubMed and Scopus databases were searched using the terms "core competencies," "patient safety," "medical error," "knowledge," and "attitudes" related to medical or nursing students. All articles published since 2000 in Spanish or English in medical journals were considered. The research by Andermann et al. on core competencies of good-quality practice and patient safety (16) and by Flin et al. on the experience in Scotland with the Medical Student Patient Safety Questionnaire (28) was considered relevant for the qualitative review. In addition, a manual search of the references of selected articles was carried out and all relevant articles reviewed. The scope of the questionnaire was defined by five factors identified a priori based on the literature review: 1) "Openness in communication" (FCP);2) "Understanding of human factor" (CFH); 3) "Proactive attitude to avoid risks in security" (AP); 4) "Awareness of error" (AWE); and 5) "System complexity and interrelationship" (CS). These five factors represented the different types of information health science students receive about AEs and patient safety in their university programs. Attitudes and knowledge were selected as the focus of the instrument because these two variables 1) had been regularly included in previous studies on patient safety and 2) interact with each other to define the behavior of future health professionals.
To explore the five factors that defined the scope of the questionnaire, 25 reactive items were developed based on the literature review and the subsequent analysis. Using successive approaches the items were refined to be more precise, easier to understand, and clearer in the description of the concepts they evaluated. A pilot test to assess survey participant understanding of the 25 items was administered to 10 Spanish students and the result was considered satisfactory. Based on the results of the pilot test, the instrument was refined into a structured five-point Likert self-administered questionnaire.
Selection of subjects
The validation study of the instrument was carried out in eight universities from five countries (Chile, Colombia, El
Salvador, Paraguay, and Spain) selected for their willingness to participate in the study and their involvement in postgraduate courses on patient safety (convenience sample inclusion criterion). The researcher who coordinated the study in each country chose the universities and invited them to participate. The research team agreed to survey a minimum sample of 385 medical and nursing students (calculated for a precision error of 5%, with p = q = 0.50). Final selection of subjects was based on participation quotas (20% for each of the five participating countries and 50% each for medical and nursing students) to ensure sample representativeness. The subjects were invited to participate in the study by completing the questionnaire after their written permission had been obtained.
Following an explanation of the objectives and scope of the study, participation was voluntary and based on the desire to contribute to collaborative research. The institutional review board (IRB) of the Department of Health of Miguel Hernández University (Elche, Spain) approved the study in April 2013.
Ceiling and floor effects were considered by analyzing the variability of the responses to the questionnaire items. For each individual factor, composite reliability indexes and Cronbach's alpha statistics were estimated to measure internal consistency.
Exploratory analyses of the principal components were conducted to ascertain the unidimensionality of the proposed factors. The eigenvalue criterion was determined to be equal to 1 (based on previous calculation of the Kaiser-Meyer-Olkin (KMO) statistic and the Bartlett test of sphericity) to verify observance of the requirements for use of these tests), and Varimax rotation was applied (34, 35). A factor loading greater than 0.5 was considered an acceptable level of missing data. The instrument's internal consistency was calculated using Cronbach's alpha statistic; acceptable values were defined as those greater than 0.7. Acceptable item-total correlations with reference values were defined as those greater than 0.3. Statistical analyses were carried out with SAS software version 20 (SAS Institute, Cary, North Carolina, United States).
Confirmatory factor analysis was conducted using EQS software version 6.1 (Multivariate Software Inc., Encino, California, United States). Maximum-likelihood estimation and the variance-covariance matrix (as input) were used. The questionnaire was refined using several confirmatory models (34). Questionnaire items were successively eliminated if they met any of the following three criteria: 1) did not present factorial regression coefficients with statistical significance (Student's t-test >2.58, P = 0.01); 2) presented standardized coefficients <0.5; or 3) contributed little to the explanatory capacity of the model (using R2 <0.30 as a cutoff) (35). Using this successive approach, four of the 25 items were eliminated. After the four items were removed, the consistency of the resulting model was verified to confirm that the structure of the factors had not changed and the statistics employed in the initial exploratory analyses remained satisfactory.
The criterion construct validity of the questionnaire was determined by examining the convergent-discriminant validity. By analyzing the weight of each item on its respective factor, the degree of relationship among the elements was assessed. By studying the standardized covariances between latent factors it was determined that each of the five factors had discriminant validity (i.e., represented a separate dimension), based on the criterion that confidence intervals (CIs) for the correlation between each pair of latent factors should not contain the value of 1 (35). Finally, the criterion construct validity was analyzed to confirm the external validity. To analyze this relationship, several multiple linear regression models were run, using variables related to the culture of patient safety ("AEs due to multiple system failures versus single system failures"; "incentives to professionals"; "expressing doubts to health professionals, instructors, or fellow students"; and "reporting errors gives health professionals a bad image") as dependent variables and the five questionnaire factors as independent variables. In the various analyses that were performed, differences were considered statistically significant when P <0.05.
A total of 786 valid questionnaires were received (reducing the expected estimation error from 5% to 3%), completed by 280 males (35.6%) and 506 females (64.4%). Of those, 464 (60.1%) were completing their final-year courses and 308 (39.9%) were in earlier stages of their medical or nursing school programs; 299 (40.3%) were in nursing programs and 443 (59.7%) were in medical programs. A more detailed sample description can be found in Table 1.
Analysis of dimensionality and initial consistency
Table 2 shows the mean scores for each questionnaire item. No floor effect or ceiling effect was found for any of the items.
The content validity of the questionnaire was derived from the adequacy of the literature review. The values for the KMO statistic (0.9) and Bartlett's test of sphericity (approximate chi-square = 5241.3, P <0.001) ensured the appropriateness of the applied technique. The factor loadings were greater than 0.5 and the explained variance was acceptable (FCP, 28.6%; AP, 8.5%; AWE, 7.2%; CS, 4.6%; and CFH, 4.48%). With respect to consistency, the Cronbach's alpha statistic was greater than 0.7 for all factors except CS, which had a value of 0.67. The Cronbach's alpha statistic was also calculated for the completed questionnaires, and its value exceeded 0.8 (an acceptable level of reliability). Item-total correlations were higher than 0.30 in all cases.
Confirmatory analysis of dimensionality and analysis of reliability
The factor analysis results are shown in Table 3. In its final version, the questionnaire consisted of 21 items grouped into five factors. The compound reliability indexes were greater than 0.7, indicating acceptable reliability (30) for all factors except CS, which had a very similar value (0.62). In the analysis of the convergent-discriminant validity, all stan-dardized loads were found to be significant for the respective factor and greater than 0.6 (Table 3). The results demonstrated good model fit (goodness-of-fit index (GFI): 0.90; adjusted GFI: 0.90) (Supplementary material, Tables 1 and 2). The optimized model is shown in Figure 1. Table 4 shows the results of the analysis of the standardized covariances between latent factors, which guaranteed discriminatory validity.
The results of the linear regression models are shown in Table 4.
The metric proprieties of the final version of the Latino Student Patient Safety Questionnaire allow for assessment of the attitudes and basic knowledge of students of health disciplines about patient safety. Because attitudes and knowledge form the basis of professional skills, this questionnaire could be used to indirectly evaluate whether or not nursing and medical students are likely to put into practice the skills considered most appropriate for patient safety.
The structure of the Latino Student Patient Safety Questionnaire is very similar to the one proposed by Flin et al. (28). The results of the validation study yielded a five-factor structure ("Knowledge of error and patient safety or Awareness of error," "Knowledge of workplace safety or Understanding of human factor," "Feelings about making errors or Openness in communication," "Attitudes toward patient safety or Proactive attitude to avoid risks in security," and "Safety at the workplace or System complexity and interrelationship"). However, the new questionnaire assesses a set of competencies (including attitudes) acquired during clinical training, whereas Flin's Medical Student Patient Safety Questionnaire mainly evaluates attitudes. The results of both the current study and Flin's study showed a positive attitude toward patient safety among the student sample. In the current study, the results suggested that students from the participating universities received information on safety procedures from their instructors more frequently and had significantly more positive attitudes (including those about maintaining open disclosure with the patient) than the student participants in other studies. According to Flin et al., the students surveyed in their study were mainly concerned about 1) the punitive attitude that still exists when a health professional makes an error and 2) learning how to conduct an open disclosure procedure appropriately to avoid a negative reaction from the patient.
The Latino Student Patient Safety Questionnaire also differs from the H-PEPSS (Health Professional Education in Patient Safety Survey), which is designed to measure health professionals' perceptions of their patient safety competence (36). The H-PEPSS was recently used in Canada to assess competence in patient safety among medical students (37). While both instruments were validated using the same procedure and refined into an instrument with fewer items, the H-PEPSS was designed to measure self-perceptions of patient safety competence whereas the Latino Student Patient Safety Questionnaire was designed to measure educational needs for future doctors and nurses. In both studies, medical students in their final year had lower scores than medical students in their final year from earlier eras, although the students from Spain and Latin America surveyed in the current study showed better understanding of the human factor in patient safety than students surveyed in the past.
The Latino Student Patient Safety Questionnaire is easy to answer and quick to administer (requiring less than 15 minutes for the entire process) and could be used by instructors and quick to administer (requiring less than 15 minutes for the entire process) and could be used by instructors and academic coordinators to determine what information students are receiving about patient satisfaction; review curricula and teaching approaches; and define training targets in hospitals or ambulatory care. This type of tool can 1) help promote curriculum changes based on the results it obtains for students at different stages of the academic programs and 2) elucidate the degree of adjustment required in certain curricula to meet consensus standards for patient safety content.
In most countries, nursing/medical students are typically trained in good-quality practices and patient safety after the completion of their undergraduate studies. This results in missed opportunities to train future professionals at a time when they are receptive and have access to all types of information and costs are uniquely low. Once undergraduate study has been completed, accessing information is more difficult, and training is costlier because it involves replacing established practices and lessons learned from past experience with new ones. Introducing patient safety in the curricula of medical and nursing programs could help ensure students acquire the expected competencies, at a lower cost. However, this is not an easy task. Despite a plethora of patient safety initiatives, there are only a few patient safety education programs for future health care professionals in Latin America and worldwide (20, 21, 37-39).
The Latino Student Patient Safety Questionnaire can be used to determine if a training course achieves its objectives, and thus may reduce the reluctance of academic institutions and educators to introduce patient safety in the curricula for health science students. The results of this study should contribute to accelerating the incorporation of patient safety teaching in Spanish-language higher education curricula.
Curricula for various health disciplines have many similarities across programs and countries (20). In Europe, directives establish competencies for health professional profiles as well as the required number of hours of clinical training. However, there are also differences, mainly in teaching approaches, organizational aspects, and available resources. Differences across countries are mostly due to the number of hours allocated to practice-oriented teaching because a significant proportion of the skills required for professional competence (including access to practical training and, later, specialized training) derive from this type of clinical training. Some previously identified barriers to including patient safety in program curricula were seen at universities in both Latin America and Spain, including 1) curricular rigidities and static pedagogy; 2) lack of resources and educational material; 3) lack of sufficient familiarity with patient safety (a new area of knowledge) among instructors and educators; 4) insufficient supply of skills in patient safety; 5) lack of recognition of educators who introduce changes in curricula; and 6) traditional views of medical errors rather than those fostered by the new culture of patient care, which encourages avoidance of risks related to health care management (40).
Educators' clinical experience (37, 41), along with experiences from other universities and/or countries (20, 21, 37, 39),and the freely available WHO Curriculum Guide (42), could serve as a guide for implementing patient safety education. Several studies have shown that patient safety education effectively shifted students' attitudes; led to their improved comprehension of the inevitability of error; increased their capacity to identify the most common sources of AEs; improved their skills in avoiding AEs (e.g., through better hand-hygiene practices); and increased their sense of responsibility (32, 37, 43, 44). However, a majority of students did not feel comfortable speaking up about patient safety issues (45) and were not confident in their skills to manage safety risks (37). The use of simulation methodology (role playing) could be an alternative means of improving the results of patient safety education in universities (increasing undergraduate nursing and medical students' comfort in reporting and speaking about AEs) based on the results of several studies in Anglo-Saxon countries (41, 46, 47).
This study had some limitations. First, because the participating universities were selected from the cooperation network for training in patient safety, the results can not be generalized to all Latin American and Spanish universities. Second, because the study methodology focused on validation of the instrument, the following could not be determined: 1) if the questionnaire is able to detect changes in knowledge and attitudes among students once new patient safety information is received, 2) the influence on the students' responses of the specific patient safety training of their teaching staff, and 3) the impact of the instrument on future training syllabuses. Future studies should assess the impact of transactional or longitudinal training models and whether patient safety training should begin in the first year of undergraduate programs or if it is more effective when introduced at the beginning of clinical training (in the final year of undergraduate studies).
The Latino Student Patient Safety Questionnaire has shown adequate metric properties for assessing attitudes and knowledge about patient safety among students of health disciplines. This was the first study in either Latin America or Spain designed to develop a questionnaire to assess the results of patient safety curricula. The results of this research could be used to help raise awareness among academic authorities about the importance of training future doctors and nurses in patient safety while at the same time providing schools and nursing/medical departments with a valuable Spanish-language tool for assessing attitudes and knowledge about patient safety. The findings of this study have implications for the design and implementation of patient safety content in medical and nursing education. A pragmatic approach would be to introduce small changes in curricula, integrating some of the ideas proposed by WHO, and subsequently modify them based on the results of evaluations conducted using the questionnaire.
Acknowledgments. The participation of a large number of students in the participating countries made this study possible, and the authors would like to express their gratitude to all of them. Ana López and Alicia Peralta assisted in the data codification.
Funding. This work was supported by the Miguel Hernández University (Elche, Spain).
Conflicts of interest. None.
Disclaimer. Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the RPSP/ PAJPH and/or PAHO.
1. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. Washington: National Academy Press; 2000.
2. Altman D, Clancy C, Blendon RJ. Improving patient safety-five years after the IOM report. N Engl J Med. 2004;351(20): 2041-2.
3. Hartmann CW, Meterko M, Rosen AK, Shibei Zhao, Shokeen P, Singer S, et el. Relationship of hospital organizational culture to patient safety climate in the Veterans Health Administration. Med Care Res Rev. 2009;66(3):320-38.
4. Singla AK, Kitch BT, Weissman JS, Campbell EG. Assessing patient safety culture: a review and synthesis of the measurement tools. J Patient Saf. 2006;2:105-15.
5. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005;14(5):364-6.
6. Pronovost PJ, Weast B, Holzmueller CG, Rosenstein BJ, Kidwell RP, Haller KB, et al. Evaluation of the culture of safety: survey of clinicians and managers in an academic medical center. Qual Saf Health Care. 2003;12(6):405-10.
7. Singer SJ, Gaba DM, Falwell A, Lin S, Hayes J, Baker L. Patient safety climate in 92 US hospitals: differences by work area and discipline. Med Care. 2009;47(1):23-31.
8. Iglesias-Alonso F, Tejero-González JM, Torijano-Casalengua ML, Mira JJ. Elementos básicos de una cultura de seguridad clínica. Rev Calid Asist. 2012;27(5):295-7.
9. Hobgood C, Peck CR, Gilbert B, Chappell K, Zou B. Medical errors-what and when: what do patients want to know? Acad Emerg Med. 2002;9(11):1156-61.
10. Robinson AR, Hohmann KB, Rifkin JI, Topp D, Gilroy CM, Pickard JA, et al. Physician and public opinions on quality of health care and the problem of medical errors. Arch Intern Med. 2002;162(19):2186-90.
11. Schoen C, Osborn R, Huynh PT, Doty M, Zapert K, Peugh J, et al. Taking the pulse of health care systems: experiences of patients with health problems in six countries. Health Aff (Millwood). 2005; 24 Suppl Web Exclusives:W5-509-25.
12. Health and Safety Commission (GB). Organising for safety: third report of the ACSNI (Advisory Committee on the Safety of Nuclear Installations) Study Group on Human Factors. Sudbury, England: HSE Books; 1993.
13. World Health Organization. Development of the core competencies for Patient Safety Research. Patient Safety Research Education and Training Working Group. Geneva: WHO; 2010.
14. World Health Organization. Patient safety research: a guide for developing training programmes. Geneva: WHO; 2012.
15. Halpern R, Lee MY, Boulter PR, Philips RR. A synthesis of nine major reports on physicians' competencies for the emerging practice environment. Acad Med. 2001; 76(6):606-15.
16. Andermann A, Ginsburg L, Norton P, Arora N, Bates D, Wu A, et al. Core competencies for patient safety research: a cornerstone for global capacity strengthening. BMJ Qual Saf. 2011;20(1):96-101.
17. Okuyama A, Martowirono K, Bijnen B. Assessing the patient safety competencies of healthcare professionals: a systematic review. BMJ Qual Saf. 20(11);20:991-1000.
18. Wakefield AB, Carlisle C, Hall AG, Attree MJ. The expectations and experiences of blended learning approaches to patient safety education. Nurse Educ Pract. 2008; 8(1):54-61.
19. Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, et al. Technologyenhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. 2011;306(9):978-88.
20. Leotsakos A, Ardolino A, Cheung R, Zheng H, Barraclough B, Walton M. Educating future leaders in patient safety. J Multidiscip Healthc. 2014;7:381-8.
21. Nie Y, Li L, Duan Y, Chen P, Barraclough BH, Zhang M, et al. Patient safety education for undergraduate medical students: a systematic review. BMC Med Educ. 2011;11:33.
22. Rodrigue Ch, Seoane L, Gala RB, Piazza J, Amedee RG. Developing a Practical and Sustainable Faculty Development Program With a Focus on Teaching Quality Improvement and Patient Safety: An Alliance for Independent Academic Medical Centers National Initiative III Project. Ochsner J. 2012;12(4):338-43.
23. Madigosky WS, Headrick LA, Nelson K, Cox KR, Anderson T. Changing and sustaining medical students' knowledge, skills, and attitudes about patient safety and medical fallibility. Acad Med. 2006; 81(1):94-101.
24. Patey R, Flin R, Cuthbertson BH, Mac-Donald L, Mearns K, Cleland J, et al. Patient safety: helping medical students understand error in healthcare. Qual Saf Health Care. 2007;16(4):256-9.
25. Kern DE, Thomas PA, Hughes MT. Curriculum development for medical education: a six-step approach. 2nd ed. Baltimore: Johns Hopkins University Press; 2009.
26. Seiden SC, Galvan C, Lamm R. Role of medical students in preventing patient harm and enhancing patient safety. Qual Saf Health Care. 2006;15(4):272-6.
27. Fischer MA, Mazor KM, Baril J, Alper E, DeMarco D, Pugnaire M. Learning from mistakes. Factors that influence how students and residents learn from medical errors. J Gen Intern Med. 2006;21(5):419-23.
28. Flin R, Patey R, Jackson J, Mearns K, Dissanayaka U. Year 1 medical undergraduates' knowledge of and attitudes to medical error. Med Educ. 2009;43(12): 1147-55.
29. Gómez O, Gutiérrez WA, González L, Garzón J, Mateus E, Soto A. Cultura de seguridad del paciente por personal de enfermería en Bogotá, Colombia. Cienc Enferm. 2011;17(3):97-111.
30. Institute for Healthcare Improvement. What students think: The surgical safety checklist and the ICU walker. Healthc Exec. 2009;24(3):76-7.
31. Liao JM, Etchegaray JM, Williams ST, Berger DH, Bell SK, Thomas EJ. Assessing medical students' perceptions of patient safety: the medical student safety attitudes and professionalism survey. Acad Med. 2014;89(2):343-51.
32. Hayes AJ, Roberts P, Figgins A, Pool R, Reilly S, Roughley C, et al. Improving awareness of patient safety in a peer-led pilot educational programme for undergraduate medical students. Educ Health (Abingdon). 2014;27(2):213-6.
33. Agencia de Calidad del Sistema Nacional de Salud (ES). Estrategia de Seguridad del Paciente: Recomendaciones del Taller de Expertos celebrado el 8 y 9 de febrero de 2005. Madrid: Ministerio de Sanidad y Consumo; 2005.
34. McDonald RP. The dimensionality of tests and items. Br J Math Stat Psychol. 1981; 34(1):100-17.
35. Joreskog K, Sorbom D. LISREL 8: Structural Equation Modeling with the SIMPLIS Command Language. Chicago: Scientific Software International; 1993.
36. Ginsburg L, Castel E, Tregunno D, Norton PG. The H-PEPSS: an instrument to measure health professionals' perceptions of patient safety competence at entry into practice. BMJ Qual Saf. 2012;21: 676-84.
37. Doyle P, VanDenKerkhof EG, Edge DS, Ginsburg L, Goldstein DH. Self-reported patient safety competence among Canadian medical students and postgraduate trainees: a cross-sectional survey. BMJ Qual Saf. 2015;24(2):135-41.
38. Dell EM, Varpio L, Petrosoniak A, Gajaria A, McMcarthy AE. The ethics and safety of medical student global health electives. Int J Med Educ. 2014;5:63-72.
39.Teigland CL, Blasiak RC, Wilson LA, Hines RE, Meyerhoff KL, Viera AJ. Patient safety and quality improvement education: a cross-sectional study of medical students' preferences and attitudes. BMC Med Educ. 2013;13:16.
40. World Health Organization. WHO Patient Safety Curriculum Guide for Medical Schools. Geneva: WHO; 2009. Available from: http://whqlibdoc.who.int/publications/2009/9789241598316_eng.pdf
41. Mariani B, Cantrell MA, Meakim C, Jenkinson A. Improving students' safety practice behaviors through a simulation-based learning experience. J Nurs Educ. 2015;54(3 Suppl):S35-8.
42. World Health Organization. Patient safety curriculum guide: multi-professional edition. Geneva: WHO; 2011.
43. Roh H, Park SJ, Kim T. Patient safety education to change medical students' attitudes and sense of responsibility. Med Teach. 2014:1-7. [Epub ahead of print]
44. Bradley SM, Chang D, Fallar R, Karani R. A patient safety and transitions of care curriculum for third-year medical students. Gerontol Geriatr Educ. 2015;36(1): 45-57.
45. Lukewich J, Edge DS, Tranmer J, Raymond J, Miron J, Ginsburg L, et al. Undergraduate baccalaureate nursing students' self-reported confidence in learning about patient safety in the classroom and clinical settings: an annual cross-sectional study (2010-2013). Int J Nurs Stud. 2015;52(5): 930-8. [Epub ahead of print]
46. Kent L, Anderson G, Ciocca R, Shanks L, Enlow M. Effects of a senior practicum course on nursing students' confidence in speaking up for patient safety. J Nurs Educ. 2015;54(3 Suppl):S12-5.
47. Thomas I, Nicol L, Regan L, Cleland J, Maliepaard D, Clark L, et al. Driven to distraction: a prospective controlled study of a simulated ward round experience to improve patient safety teaching for medical students. BMJ Qual Saf. 2015;24(2): 154-61.
Manuscript received on 31 July 2014.
Revised version accepted for publication on 9 April 2015.