Abstract:

The evaluation of the culture of patient safety in hospitals is nowadays considered as a management too, since it helps to identify problem areas and provide valuable information for planning improvements. This study explored the reliability and validity of the Brazilian version of the Hospital Survey on Patient Safety Culture, an instrument that evaluates characteristics of patient safety culture among hospital staff. The reliability of the instrument was evaluated by analyzing the internal consistency of each dimension. The validity of the tool was carried out by means of exploratory and confirmatory factor analysis. The sample was made up of 322 questionnaires that were collected in two Brazilian hospitals in 2012. Cronbach's alpha ranged from 0.52 to 0.91 for the different dimensions, with the exception of two, for which it was much lower. After excluding four items, the exploratory factor analysis presented adjusted indices that were appropriate for a 10 factor model.

Keywords:
Safety; Patient Safety; Quality of Health Care; Questionnaires

Resumo:

A avaliação da cultura de segurança do paciente no ambiente hospitalar é hoje considerada uma ferramenta de gestão, pois permite identificar áreas problemáticas e fornece informações valiosas para o planejamento de melhorias. Este estudo examinou a confiabilidade e a validade da versão brasileira da Pesquisa sobre Cultura de Segurança do Paciente em Hospitais (HSOPSC), instrumento que avalia características da cultura de segurança do paciente entre trabalhadores de hospitais. A confiabilidade do instrumento foi avaliada através da análise da consistência interna de suas dimensões. A validade de construto foi realizada por análise fatorial exploratória e confirmatória. A amostra foi composta por 322 questionários coletados em dois hospitais no Brasil, em 2012. Entre as dimensões, o alfa de Cronbach variou entre 0,52 a 0,91, exceto para duas dimensões em que ele foi muito baixo. Após exclusão de 4 itens, a Análise Fatorial Exploratória apresentou índices de ajuste apropriados a um modelo de 10 fatores.

Palavras-chave:
Segurança; Segurança do Paciente; Qualidade da Assistência à Saúde; Questionários

Resumen:

La evaluación de la cultura de seguridad del paciente en el ambiente hospitalario se considera hoy una herramienta de gestión, pues permite identificar áreas problemáticas y proporciona información valiosa para la planificación de mejoras. Este estudio examinó la confiabilidad y la validez de la versión brasileña de la Investigación sobre Cultura de Seguridad del Paciente en Hospitales (HSOPSC), instrumento que evalúa características de la cultura de seguridad del paciente entre trabajadores de hospitales. La confiabilidad del instrumento fue evaluada a través del análisis de la consistencia interna de sus dimensiones. La validez del constructo se realizó mediante un análisis factorial exploratorio y confirmatorio. La muestra estuvo compuesta por 322 cuestionarios, recogidos en dos hospitales en Brasil, en 2012. Entre las dimensiones, el alfa de Cronbach varió entre 0,52 a 0,91, excepto para dos dimensiones en la que fue muy bajo. Tras la exclusión de 4 ítems, el análisis factorial exploratorio presentó índices de ajuste apropiados para un modelo de 10 factores.

Palabras-clave:
Seguridad; Seguridad del Paciente; Calidad de la Atención de Salud; Cuestionarios

Introduction

The provision of safe, reliable health care requires that professionals participate in a robust culture of safety, particularly in hospitals. A collaborative work environment where qualified professionals treat each other with respect, leaders guide their teams efficiently and continued professional development is a priority is a core condition for reducing avoidable adverse events ¹1. Leonard M, Frankel A, Frederico F, Frush K, Haraden C. The essential guide for patient safety officers. 2nd Ed. Oakbrook Terrace: Joint Commission Resources, Institute of Healthcare Improvement; 2013..

In order to establish an effective safety culture in a health care organization, it is crucial to assess its current culture ²2. Pronovost PJ, Weast B, Bishop K, Paine L, Griffith R, Rosenstein BJ, et al. Senior executive adopt-a-work unit: a model for safety improvement. Jt Comm J Qual Saf 2004; 30:59-68.. Information from the organization's personnel about their safety-related perceptions and behavior helps to identify problem areas for planning and implementing interventions intended to strengthen the safety culture. Sorra & Nieva ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004. explain that a safety culture assessment can have multiple goals: (i) auditing the safety culture and raising employee awareness on the subject; (ii) assessing patient safety interventions implemented in the institution and tracking changes over time; (iii) making comparisons with reference data that is both internal and external to the organization; and (iv) checking compliance with regulatory requirements.

Several instruments, each with different features, are available to assess patient safety culture in health care services ⁴4. Halligan M, Zecevic A. Safety culture in healthcare: a review of concepts, dimensions, measures and progress. BMJ Qual Saf 2011; 20:338-43.. One of these, the Hospital Survey on Patient Safety Culture (HSOPSC) ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004., developed by the U. S. Agency for Healthcare Research and Quality (AHRQ), is considered a valid, reliable and efficient instrument. The HSOPSC was developed to assess the perception of hospital personnel with respect to different characteristics of the safety culture, and is one of the most commonly used instruments worldwide to measure patient safety culture ⁵5. Flin R, Burns C, Mearns K, Yule S, Robertson EM. Measuring safety climate in health care. Qual Saf Health Care 2006; 15:109-15.^{), (6}6. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety climate: a review of surveys. Qual Saf Health Care 2005; 14:364-6.^{), (7}7. Jackson J, Sarac C, Flin R. Hospital safety climate surveys: measurement issues. Curr Opin Crit Care 2010; 16:632-8.. However, in order to convert results from a cross-sectional study that uses data-collection instruments into reliable safety culture indicators, the instrument must achieve the highest possible standard of measurement ⁸8. Flin R. Measuring safety culture in healthcare: a case for accurate diagnosis. Safety Sci 2007; 45:653-67.. To that end, after an instrument has been processed to assure semantic and conceptual equivalence for use in a culture different from that of its origin, it is essential to assess its content and measurement validity ⁹9. Reichenheim ME, Hökerberg YHM, Moraes CL. Assessing construct structural validity of epidemiological measurement tools: a seven-step roadmap. Cad Saúde Pública 2014; 30:927-39..

Studies on patient safety culture in Brazil are relatively new, and a validated instrument will add potential for improving the quality of health care at Brazilian hospitals. The aim of this study was thus to examine the psychometric properties of the Brazilian version of the HSOPSC ¹⁰10. Reis CT, Laguardia J, Martins M. Adaptação transcultural da versão brasileira do Hospital Survey on Patient Safety Culture: etapa inicial. Cad Saúde Pública 2012; 28:2199-210..

Methods

The HSOPSC

This instrument is made up of 42 questions designed to assess safety culture awareness across 12 dimensions ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004.. Seven dimensions relate to the hospital department or work unit: (i) communication openness; (ii) feedback and communication about error; (iii) organizational learning - continuous improvement; (iv) supervisor/manager expectations and actions promoting patient safety; (v) non-punitive response to errors; (vi) teamwork within units; and (vii) staffing. Three dimensions assess safety culture awareness at the hospital level: (i) management support for patient safety; (ii) handoffs & transition; and (iii) teamwork across units. Two dimensions contemplate results: (i) overall perceptions of patient safety; and (ii) frequency of events reported. The items that compose the HSOPSC questionnaire are shown in Table 1, aggregated into their respective dimensions.

The HSOPSC also contains eight questions relating to respondents' characteristics (gender, professional category, level of education, how long they have worked at the hospital, weekly working hours etc.). It takes approximately 15 minutes to complete all items. The HSOPSC measures awareness among staff who are in direct or indirect contact with patients and spend most of their working hours at the hospital. Most items are answered on a five-point (Likert-type) scale reflecting level of agreement from "I strongly disagree" (1) to "I strongly agree" (5), with a neutral category "I neither agree nor disagree" (3). Other items are answered on a five-point frequency scale from "never" (1) to "always" (5).

The original instrument was submitted at the initial stages of this study for transcultural adaptation (conceptual, item and semantic equivalences) ¹⁰10. Reis CT, Laguardia J, Martins M. Adaptação transcultural da versão brasileira do Hospital Survey on Patient Safety Culture: etapa inicial. Cad Saúde Pública 2012; 28:2199-210..

Setting, population and data collection

The instrument was applied to a sample of workers at two large hospitals (Hospital 1 and Hospital 2, with 150 and 499 beds, respectively) in two states in southeastern Brazil. The study population comprised staff in direct or indirect contact with inpatients at the two participating hospitals, as well as staff (leaders, managers, supervisors and administrators) whose functions did not involve direct contact with inpatients, but directly affected inpatient care. The inclusion criteria were those suggested by the developers of the HSOPSC ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004.: staff with a minimum 20-hour working week at the hospital and on duty during the data collection period at the institution. Undergraduate students were excluded in view of the nature of their situation, i.e., still in training. Of the 718 staff invited to participate in the study, 261 were from Hospital 1 and 457 from Hospital 2.

Before data collection, the research team approached both hospital directors in order to explain the purpose of the study, highlight the importance of safety culture to improving care, and present the necessary procedures, and inclusion criteria for participation in the survey. Head nurses of several inpatient departments, heads of inpatient diagnostic support services, and staff from several departments in their work environments were identified as hospital leaders.

For data collection, in most instances, printed instruments were handed to participants to be answered and later returned to the lead researcher. Each participant received two copies of the Informed Consent Form; one copy was retained by the researchers and the other by the participant. On agreement, each participant received an envelope with a copy of the adapted instrument, without identifying information from the participant. Data were collected from March to May 2012. Researchers made five previously scheduled visits to each hospital to collect the completed instruments.

Data analysis

Reliability and construct validity were examined. Reliability testing evaluated the extent to which results obtained by the Brazilian version of HSOPSC could be replicated. In order to allow for comparison with other validation studies ¹¹11. Bodur S, Filiz E. Validity and reliability of Turkish version of "Hospital Survey on Patient Safety Culture" and perception of patient safety in public hospitals in Turkey. BMC Health Serv Res 2010; 10:28.^{), (12}12. Olsen E. Reliability and validity of the Hospital Survey on Patient Safety Culture at a Norwegian hospital. In: Øvretveit J, Sousa PJ, editors. Quality and safety improvement research: methods and research practice from the International Quality Improvement Research Network (QIRN). Lisbon: Escola Nacional de Saúde Pública, Universidade Nova de Lisboa; 2008. p. 173-86.^{), (13}13. El-Jardali F, Jaafar M, Dimassi H, Jamal D, Hamdan R. The current state of patient safety culture in Lebanese hospitals: a study at baseline. Int J Qual Health Care 2010; 22:386-95.^{), (14}14. Pfeiffer Y, Manser T. Development of the German version of the Hospital Survey on Patient Safety Culture: dimensionality and psychometric properties. Safety Sci 2010; 48:1452-62.^{), (15}15. Dolci GF, Suárez JR, Casanova HA, Fernández CR, Torres FH, Varela JS. Cultura sobre seguridad del paciente en profesionales de la salud. Cir Cir 2010; 78:527-32.^{), (16}16. Smits M, Christiaans-Dingelhoff I, Wagner C, Wal G, Groenewegen PP. The psychometric properties of the "Hospital Survey on Patient Safety Culture" in Dutch hospitals. BMC Health Serv Res 2008; 8:230.^{), (17}17. Ito S, Seto K, Kigawa M, Fujita S, Hasegawa T, Hasegawa T. Development and applicability of Hospital Survey on Patient Safety Culture (HSOPS) in Japan. BMC Health Serv Res 2011; 11:28.^{), (18}18. Wenqi L. Validation of a translated version of the Hospital Survey on Patient Culture - HSOPSC [Masters Thesis]. Hasselt: University of Hasselt; 2005.^{), (19}19. Chen IC, Li HH. Measuring patient safety culture in Taiwan using the Hospital Survey on Patient Safety Culture (HSOPSC). BMC Health Serv Res 2010; 10:152.^{), (20}20. Moghri J, Arab M, Saari AA, Nateqi E, Forooshani AR, Ghiasvand H, et al. The psychometric properties of the farsi version of "Hospital Survey on Patient Safety Culture" in Iran's Hospitals. Iran J Public Health 2012; 42:80-6.^{), (21}21. Blegen AM, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's Hospital Survey on Patient Safety Culture: psychometric analyses. J Patient Saf 2009; 5:139-44.^{), (22}22. Waterson P, Griffiths P, Stride C, Murphy J, Hignett S. Psychometric properties of the Hospital Survey on Patient Safety Culture: findings from the UK. Qual Saf Health Care 2010;19:e2., the reliability of HSOPSC was assessed through its internal consistency as estimated using Cronbach's alpha ²³23. Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol 1993; 78:98-104.^{), (24}24. Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª Ed. Porto Alegre: Bookman; 2009. for each dimension or factor, in line with the dimensions proposed by the original model. Since some questionnaire items were worded positively and others negatively, negatively worded items were reverse coded so that in all cases a higher score would indicate a positive response ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004.. The Kolmogorov Sminorv test was used to assess if the study score distribution was normal.

The construct validity was assessed by exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) ²⁵25. Brown TA. Confirmatory factor analysis for applied research. New York: The Guilford Press; 2006., and by the correlation matrix among the dimensions. These analyses were based on questionnaires with no missing data. Firstly, the dimensional structure of the original 12-dimension instrument ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004. was evaluated by CFA ²⁶26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007.. As several dependent variables in the HSOPSC are ordinal (Likert-type, five-point response options), the weighted least squares mean and variance adjusted (WLSMV) ²⁶26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007. estimation was used. In CFA, model fit was evaluated using three indices: Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI). Absolute fit was assessed using the RMSEA index, in accordance with the use of WLSMV ²⁵25. Brown TA. Confirmatory factor analysis for applied research. New York: The Guilford Press; 2006.^{), (26}26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007.. RMSEA is a poor-quality-of-fit index incorporating a penalty function to address a lack of parsimony in the model's degrees of freedom ²⁵25. Brown TA. Confirmatory factor analysis for applied research. New York: The Guilford Press; 2006.^{), (26}26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007.. The lower the RMSEA value, the better the fit: less than 0.05 indicates good fit, while values greater than 0.10 indicate poor fit, and that the model should be rejected. CFI and TLI, used as measures of incremental fit ²⁵25. Brown TA. Confirmatory factor analysis for applied research. New York: The Guilford Press; 2006.^{), (26}26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007., compare the proposed model with an independence (null) model; values range from 0 to 1, with values greater than 0.90 showing proper fit ²⁵25. Brown TA. Confirmatory factor analysis for applied research. New York: The Guilford Press; 2006.^{), (26}26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007..

Based on the results of CFA, EFA was performed to investigate a plausible alternative dimensional structure fitting the study data. The factors were initially interpreted using the 42 items by considering eigenvalues greater than 13. In addition, parallel analysis (PA) was performed to identify the number of factors to be retained. The rationale underlying PA is that nontrivial components from real data with a valid underlying factor structure should have larger eigenvalues than parallel components derived from random data having the same sample size and number of variables ²⁷27. Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organizational Research Methods 2004; 7:191-205.. Factor items were compared in terms of their factor loading behavior, and those that returned loads of less than 0.40 ²⁸28. Pett MA, Lackey NR, Sullivan JJ. Making sense of factor analysis: the use of factor analysis for instrument development in health care research. Thousand Oaks: Sage Publications; 2003. or cross-loading with differences of less than 0.2 between the two highest loads were excluded. This analysis used the weighted least squares mean and variance adjusted (WLSMV) estimation and the oblique rotation method (geomin), assuming that the dimensions that make up patient safety culture are interrelated. The indices of fit examined were RMSEA, CFI and TLI.

Missing data are expected when HSOPSC is used as a data collection instrument. The instructions read: "If you do not wish to answer a question, or if a question does not apply to you, you may leave your answer blank". Thereby, missing data was treated using the Listwise deletion method, which assumes that loss in collection can be characterized as "missing at random" (MAR) ²⁴24. Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª Ed. Porto Alegre: Bookman; 2009.. That choice was made given the difficulty of identifying the reasons leading to failure to complete items, since the instrument offers respondents that option.

The SPSS (SPSS Inc., Chicago, U.S.A.) version 17.0 was used to build the database, manage data, treat variables, perform descriptive statistical analyses and reliability calculations, and examine data factorability. Mplus version 5.0 (Muthén & Muthén, Los Angeles, U.S.A.) was used to apply Kolmogorov-Smirnov tests and calculate correlations among the dimensions, CFA and EFA ²⁶26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007.. The GAIPE package from the R statistical software (The R Foundation for Statistical Computing, Vienna, Austria; http://www.r-project.org) was applied to compute 90% confidence intervals (90%CI) for RMSEA.

Ethical considerations

This study was approved by the Ethics Research Committee of the Sergio Arouca National School of Public Health, Oswaldo Cruz Foundation (process n. 177/2011).

Results

Of the 718 workers invited to participate in the survey, 327 (149 from Hospital 1 and 178 from Hospital 2) returned the questionnaires. On reviewing all the questionnaires, two from Hospital 1 were found to have at least one section not fully completed, while one questionnaire had the same response to all items containing reverse items in one particular section. These three questionnaires were excluded from the study. Thus, Hospital 1 returned 146 valid instruments (a total response rate of 56%). At Hospital 2, meanwhile, of the 178 questionnaires completed, two did not have at least one section fully completed. These were excluded from the study. Thus, Hospital 2 returned 176 valid instruments (a 38.5% total response rate). Together, both hospitals produced 322 valid questionnaires, corresponding to a response rate of 44.8%.

The percentage of missing data per item ranged from 0.6% to 7.7%. In terms of the dimensional structure of the instrument, only the "Frequency of events reported" dimension, comprising three questions on error reporting, showed an average percentage loss greater than 5% (5.9%). Of the 322 questionnaires included in the study, 105 were missing at least one item, while 217 were fully completed. Thus, loss of statistical power due to the reduced number of cases was admitted without, however, major loss of veracity in the information.

Cronbach's alpha coefficients estimated for the twelve dimensions of the original model's dimensional structure ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004. ranged from 0.52 to 0.91, except for two dimensions, "Non-punitive response to errors" and "Staffing", where internal consistency was very low, respectively, α = 0.35 and α = 0.20 (Table 2).

Kolmogorov-Smirnov tests were significant (p < 0.001) for all 42 items proposed in the original model and pointed out that the score from the Brazilian version HSOPSC does not have a normal distribution. For that reason, we applied the weighted least squares mean and variance adjusted (WLSMV) method, which provides estimates of chi-square statistics corrected for mean and variance, in addition to robust standard error estimates for the model parameters ²⁶26. Muthén LK, Muthén BO. Mplus user's guide. 5th Ed. Los Angeles: Muthén & Muthén; 2007..

CFA was performed to examine the fit between the data obtained in this study and the dimensional structure proposed in the original 12-factor model ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004.. It showed poor fit for two indices, CFI = 0.888 and RMSEA = 0.106 (90%CI: 0.092-0.120), and a reasonably adequate fit for a third index (TLI = 0.930). Given that the original 12-factor model displayed inappropriate indices of fit, EFA was applied to explore a model that would best fit the data. Exploratory factor analysis of structures with 11 and 10 factors and the 42 original items exhibited good fit for both models among three indices. EFA of structure with 11 factors exhibited the following indices: CFI = 0.946, TLI = 0.971 and RMSEA = 0.067 (90%CI: 0.054-0.082); and the EFA with 10 factors resulted in the following indices: CFI = 0.942, TLI = 0.970 and RMSEA = 0.070 (90%CI: 0.056-0.084).

The eigenvalues obtained from EFA with the 42 items were, from factor 1 to 12, respectively, 12.38, 2.74, 2.38, 2.32, 1.98, 1.72, 1.46, 1.25, 1.19, 1.12, 1.01 and 0.95. These values supported a dimensional structure with 11 factors, although the last factor was slightly higher than 1. Although the models with 11 and 10 factors showed similar fit indices in EFA, the 10-factor model was selected based on results from parallel analysis.

After defining the number of factors to be retained, analysis of factor loading for the 10 factor model with 42 items identified problems with the following items, shown in Table 3: (i) A9 ("mistakes have led to positive changes here") and C1 ("we are given feedback about changes put into place based on event reports") had weak loadings (0.34 and 0.29, respectively); (ii) C6R ("staff are afraid to ask questions when something does not seem right") had weak and cross-loadings (0.24 on factor 3 and 0.30 on factor 9); and (iii) A16R ("staff worry that mistakes they make are kept in their personnel file") had weak loading (0.39).

These findings led us to exclude those items (A9, C1, C6R, A16R), and to perform a further EFA with 38 items. EFA using WLSMV, geomin oblique rotation, 38 items and the 10-factor model returned the following fit indices: CFI = 0.945, TLI = 0.971, RMSEA = 0.072 (90%CI: 0.059-0.087). Together, these 10 factors accounted for 70.8% of the variance.

With respect to the theoretical model proposed in the original instrument, some findings of the 10 factor model with 38 items were positive (Table 4). Items A5R, A8R, A12R and A14R from dimensions "staffing" and "non-punitive response to errors" loaded on factor 2. Three out of four items from the "supervisor/manager expectations & actions promoting patient safety" dimension (B1, B2 and B3R) and one item from the "communication openness" dimension (C4) loaded on factor 4. Items D1, D2 and D3, which compose the "frequency of events reported" dimension in the original model, loaded strongly on factor 6 (0.91, 0.96 and 0.75, respectively). Items F3R, F5R, F7R and F11R, which make up the dimension "handoffs and transition" in the original model, loaded on factor 7, as well as items F2R, F4 and F6R, which define the dimension "teamwork across units". Three items that make up the "teamwork within units" dimension (A1, A3 and A4) loaded on factor 8 (0.59, 0.63, and 0.73, respectively) along with 2 items from the "organizational learning" and "staffing" dimensions (A6 and A7R, respectively). The other two items in the "staffing" dimension in the original model (A2 and A7R) loaded on factors 10 and 8, respectively. Item A17R, under "overall perceptions of patient safety" in the original model, loaded on factor 3, as did items A13 and B4R, under "organizational learning" and "supervisor/manager expectations", respectively. Items F1, F8 and F9R, under dimension "management support for patient safety" loaded on factor 9, together with item F10 from "team work across units".

Nonetheless, a few weak points were observed in the final model: (i) items C3 and C5, under "feedback and communication about error", both loaded on factor 1, forming a factor consisting of two items; (ii) item A11, belonging to "teamwork across the units" in the original model, loaded on factor 10 (-0.69), as did item A2, under "staffing" (-0.53), forming a factor comprising only two items; and (iii) under "overall perceptions of patient safety", item A10R alone loaded on factor 5 (0.74) (Table 4). In Addition, the items A15 and A18 from "overall perceptions of patient safety" and item C2 from the "communication openness" dimension, had loadings below 0.40 (Table 4).

None of the factors presented strong correlation with each other; the correlations among the factors in the 38-item model presented values ranging from 0.007 to 0.439 (Table 5).

Discussion

Although self-administered questionnaires are widely used by researchers needing to obtain large amounts of information in a relatively short time, lengthy surveys, such as HSOPSC, can be tedious for those answering them, and in some cases respondents may simply lose interest and not answer questions accurately ¹¹11. Bodur S, Filiz E. Validity and reliability of Turkish version of "Hospital Survey on Patient Safety Culture" and perception of patient safety in public hospitals in Turkey. BMC Health Serv Res 2010; 10:28.. Despite these drawbacks, the percentage of items answered in this study was similar to those of other HSOPSC psychometric validation studies ¹⁴14. Pfeiffer Y, Manser T. Development of the German version of the Hospital Survey on Patient Safety Culture: dimensionality and psychometric properties. Safety Sci 2010; 48:1452-62.^{), (15}15. Dolci GF, Suárez JR, Casanova HA, Fernández CR, Torres FH, Varela JS. Cultura sobre seguridad del paciente en profesionales de la salud. Cir Cir 2010; 78:527-32.^{), (16}16. Smits M, Christiaans-Dingelhoff I, Wagner C, Wal G, Groenewegen PP. The psychometric properties of the "Hospital Survey on Patient Safety Culture" in Dutch hospitals. BMC Health Serv Res 2008; 8:230., which have ranged from 47% to 55%.

Regarding reliability, the Cronbach's coefficient ranged from 0.52 to 0.91, except for two dimensions, where it was less than 0.36: "staffing" (0.20) and "non-punitive response to errors" (0.35). Other studies have also found low Cronbach's alpha values for these two dimensions. Values estimated for the "staffing" dimension of the HSOPSC were 0.19 in the Turkish version, with 309 participants ¹¹11. Bodur S, Filiz E. Validity and reliability of Turkish version of "Hospital Survey on Patient Safety Culture" and perception of patient safety in public hospitals in Turkey. BMC Health Serv Res 2010; 10:28.; 0.12 in the Spanish version, with a sample of 174 healthcare professionals in Mexico ¹⁵15. Dolci GF, Suárez JR, Casanova HA, Fernández CR, Torres FH, Varela JS. Cultura sobre seguridad del paciente en profesionales de la salud. Cir Cir 2010; 78:527-32.; 0.49 in the validation of the Dutch version, with 583 professionals ¹⁶16. Smits M, Christiaans-Dingelhoff I, Wagner C, Wal G, Groenewegen PP. The psychometric properties of the "Hospital Survey on Patient Safety Culture" in Dutch hospitals. BMC Health Serv Res 2008; 8:230.; and 0.46 in the validation of the Japanese version, with 6,395 participants ²⁰20. Moghri J, Arab M, Saari AA, Nateqi E, Forooshani AR, Ghiasvand H, et al. The psychometric properties of the farsi version of "Hospital Survey on Patient Safety Culture" in Iran's Hospitals. Iran J Public Health 2012; 42:80-6.. Likewise, values in the "non-punitive responses to errors" dimension were 0.35 in the study evaluating the Turkish version ¹¹11. Bodur S, Filiz E. Validity and reliability of Turkish version of "Hospital Survey on Patient Safety Culture" and perception of patient safety in public hospitals in Turkey. BMC Health Serv Res 2010; 10:28. and 0.55 in the Mexican study ¹⁵15. Dolci GF, Suárez JR, Casanova HA, Fernández CR, Torres FH, Varela JS. Cultura sobre seguridad del paciente en profesionales de la salud. Cir Cir 2010; 78:527-32..

Although Cronbach's alpha has been chosen to investigate the questionnaire's reliability, this value should always be interpreted in light of the characteristics of the measurement it is associated with in the study population. Accordingly, only repeated use of the instrument with different samples will be able to provide substantial confirmation of its validity and reliability. Not until a scale has been used several times, and generated reliable data, can it safely be said, in fact, to be reliable ²³23. Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol 1993; 78:98-104.. It is therefore recommended to conduct further studies using the HSOPSC in Brazil to confirm the Cronbach's alpha values obtained for these dimensions.

Concerning validity, CFA did not confirm the 12-factor dimensional theoretical model proposed by the original questionnaire, as evidenced by the fit index values and eigenvalues greater than 1.

The EFA for all 42 items presented values that can be considered adequate fit indices for models with 11 and 10 factors, but parallel analysis indicated that a 10-factor model would be more appropriate in view of study data. However, low loadings and cross-loadings justified the exclusion of items A9, A16R, C1 and C6R and evaluation of a 38-item model. EFA results from 38 items resulted in a 10-factor model with acceptable fit indices. Hair et al. ²⁴24. Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª Ed. Porto Alegre: Bookman; 2009. considered that, for 30 or more variables and sample size less than 250, as in this study, good fit is demonstrated by RMSEA values lower than 0.08 and CFI of 0.95 or more. Although, the very low correlations obtained in the 10-factor model are considered to undermine the model's convergent validity, these values pointed to a satisfactory internal discriminant validity. As stated by Brown ⁽²⁵25. Brown TA. Confirmatory factor analysis for applied research. New York: The Guilford Press; 2006., factor correlations equal to or above 0.85 are often taken as a criterion for problematic discriminant validity.

The purpose of EFA, besides finding good fit, is to test the dimensional model of the construct. In that respect, the factor model obtained by EFA was satisfactory, and some items loaded in a manner similar to the structure of the dimensional model of the original questionnaire. Items belonging to two dimensions in the original model, loaded on a single factor in our study. The inclusion of items from two dimensions into the same factor, as suggested by the model produced by EFA, was acceptable when the scope of the dimensions involved was related to the same level. For example, items under "handoffs and transition" were aggregated with items under "teamwork across units", since both are at hospital level. Items under "non-punitive response to errors" and "staffing" were aggregated into a single factor, as well as "supervisor/manager expectations and actions promoting patient safety" were added with items under "communication openness", taking into consideration that both dimensions relate to the hospital department or work unit. In the 10-factor model suggested by EFA, 7 factors contained 3 items or more, 2 factors contained 2 items and one factor contained a single item, which weakens the consistency of the suggested model.

From there, continuing to explore for a sound model well fitted to the study data would depend on further exploratory factor analyses and removal of problematic items until achieving a satisfactory model. The authors chose not to do so for the following reasons: (1) this study, the first to investigate the psychometric properties of the Portuguese version of the HSOPSC for use in Brazil, is exploratory in nature and, accordingly, it is advisable to test this version in other larger Brazilian samples; (2) development of the HSOPSC theoretical model involved thorough, well-documented studies by AHRQ, indicating caution and parsimony in the use of empirical results and changes to the original model after comprehensive tests, including qualitative studies, and use of larger samples; and (3) HSOPSC has been used widely in diverse socio-economic and cultural contexts and at different levels of monitoring and concern with quality care and patient safety, at differing stages of development, with results that mostly confirm the original model ¹²12. Olsen E. Reliability and validity of the Hospital Survey on Patient Safety Culture at a Norwegian hospital. In: Øvretveit J, Sousa PJ, editors. Quality and safety improvement research: methods and research practice from the International Quality Improvement Research Network (QIRN). Lisbon: Escola Nacional de Saúde Pública, Universidade Nova de Lisboa; 2008. p. 173-86.^{), (17}17. Ito S, Seto K, Kigawa M, Fujita S, Hasegawa T, Hasegawa T. Development and applicability of Hospital Survey on Patient Safety Culture (HSOPS) in Japan. BMC Health Serv Res 2011; 11:28.^{), (18}18. Wenqi L. Validation of a translated version of the Hospital Survey on Patient Culture - HSOPSC [Masters Thesis]. Hasselt: University of Hasselt; 2005.^{), (19}19. Chen IC, Li HH. Measuring patient safety culture in Taiwan using the Hospital Survey on Patient Safety Culture (HSOPSC). BMC Health Serv Res 2010; 10:152.^{), (20}20. Moghri J, Arab M, Saari AA, Nateqi E, Forooshani AR, Ghiasvand H, et al. The psychometric properties of the farsi version of "Hospital Survey on Patient Safety Culture" in Iran's Hospitals. Iran J Public Health 2012; 42:80-6., thus suggesting that HSOPSC is an instrument which enables different levels of patient safety culture to be detected and compared.

On the other hand, some psychometric validation studies did not confirm the dimension structure of the questionnaire as proposed in the original model. Smits et al. ¹⁶16. Smits M, Christiaans-Dingelhoff I, Wagner C, Wal G, Groenewegen PP. The psychometric properties of the "Hospital Survey on Patient Safety Culture" in Dutch hospitals. BMC Health Serv Res 2008; 8:230. obtained a model with 11 dimensions, which together explained 57.1% of the variance. After excluding the items under "Staffing", Blegen et al. ²¹21. Blegen AM, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's Hospital Survey on Patient Safety Culture: psychometric analyses. J Patient Saf 2009; 5:139-44. also obtained an 11-dimension model. Bodur & Filiz ¹¹11. Bodur S, Filiz E. Validity and reliability of Turkish version of "Hospital Survey on Patient Safety Culture" and perception of patient safety in public hospitals in Turkey. BMC Health Serv Res 2010; 10:28. identified 10 dimensions of safety culture when validating the Turkish version HSOPSC, while maintaining all 42 items of the original model proposed by AHRQ. In the validation study of the English HSOPSC (n = 1,017), Waterson et al. ²²22. Waterson P, Griffiths P, Stride C, Murphy J, Hignett S. Psychometric properties of the Hospital Survey on Patient Safety Culture: findings from the UK. Qual Saf Health Care 2010;19:e2. estimated a 9-dimension model that explained 66.8% of data variance. In the Swiss study (n = 2,421), Pffeifer & Manser ¹⁴14. Pfeiffer Y, Manser T. Development of the German version of the Hospital Survey on Patient Safety Culture: dimensionality and psychometric properties. Safety Sci 2010; 48:1452-62. obtained a dimensional model with 8 factors that explained 59.8% of variance in item responses.

Study limitations

One of the limitations encountered was a high rate of missing data, i.e., questionnaires where the 42 items were only partially answered, which may possibly have impaired the estimates of the instrument's reliability and construct validity. Of the 322 questionnaires included in the study, 105 lacked at least one datum, and led those instruments to be excluded from the factor analysis stage. Although at the initial stage of data analysis, use of missing-data imputation techniques was considered, this methodological option was discarded due to the impact on the estimates, and also because the directionality resulting from the techniques examined was unknown, given the small number of cases included in the study.

Another important limitation of this study was the high rate of potential respondents refusing to participate. The response rate showed that more than 50% of those eligible refused to answer the questionnaire and various reasons were given to justify that refusal. The questionnaire is lengthy, and health professionals in hospitals are usually overloaded with work, making them less readily available. Although low, the response rate was similar to those of studies in other countries. The instrument's developers in the user manual provided mention a response rate of around 50% in situations where researchers are careful enough to maintain participants' anonymity and distribute the questionnaire on paper ³3. Sorra JS, Nieva VF. Hospital survey on patient safety culture. Rockville: Agency for Healthcare Research and Quality; 2004., as was the case in this study. The small sample affects data variability, impairing psychometric analyses and causing non-response bias. The items' behavior in the EFA analysis may be due to cultural and contextual effects that would compromise the comprehension of the questions as well as the level of concern and familiarity to safety patient issues.

We chose to collect data on a printed questionnaire with the aim of facilitating respondent participation, but in a scenario where increasing use is made of information technology, this could be replaced by data collection online or by email. However, this decision to collect data using hard copy questionnaires reflected issues relating to requirements from the research ethics committee, after respondents had filled out their consent forms agreeing to take part in the research.

Conclusions

Patient safety culture among healthcare professionals is gaining increasing attention. It is considered to be an aspect of organizational culture and is reflected in the behavior of individual members of an organization, unit or team, especially in the values and beliefs they share, and in what priority these professionals give to patient safety. Characteristically, in a strong patient safety culture, teamwork is valued, the organization's leaders engage with the challenges involved in providing safe health care, and those leaders take a pro-active attitude to errors, supported by open communication among staff and encouragement for people to report safety incidents and, consequently, the promotion of effective organizational learning from those incidents ²⁹29. Reis CT. Cultura em segurança do paciente. In: Sousa P, Mendes W, organizadores. Segurança do paciente: criando organizações de saúde seguras. Rio de Janeiro: Editora Fiocruz; 2014. p. 75-99..

Assessment of the status of safety culture in hospitals has been the subject of several studies at the global level. These have included pre- and post-intervention study designs that showed needs perceived in a pre-intervention assessment, and gains from effective implementation of safety interventions, translated into stronger safety culture assessed post-intervention.

The main aim of this study was to assess the reliability and validity of the Brazilian version of the HSOPSC, a patient safety culture assessment instrument. In that respect, it has achieved its goals. Although the dimensional structure proposed by the original model of the instrument was not confirmed with the sample of this study, given the importance of the subject and scarcity of Brazilian studies that have assessed the HSOPSC's psychometric properties, we believe that the decision to reduce this instrument would be premature before testing in larger samples of health professionals working in accredited hospitals.

It is hoped that the availability of this instrument in the sphere of hospital care in Brazil will contribute to improved strategies and measures, to the benefit of patients and of professionals involved in providing healthcare that aspires towards creating the appropriate conditions for safer, more effective, timely, patient-focused, efficient and equitable health care ³⁰30. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st Century. Washington DC: National Academy Press; 2001..

References

Publication Dates

History