ABSTRACT:

Objective:

The assessment of the degree to which health professionals, workers and organizations are ready to implement changes in health services deserves special attention, especially related to new technologies, public policies and innovation. The objectives of this study were to conduct a Brazilian Portuguese Brazil cross-cultural adaptation of the ORIC questionnaire and to initiate the study of its psychometric properties.

Methods:

Through a cross-sectional study, the Organizational Readiness for Implementing Change (ORIC) questionnaire, containing 12 questions, was translated and later applied to a sample of workers from traditional primary health care units undergoing transformation to family health units. Statistical analysis included Cronbach’s alpha, exploratory and confirmatory factor analysis by structural equation model using the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist.

Results:

Workers from ten health units participated in the study (n = 150). The analysis confirmed two main factors (Effectiveness and Commitment) with Eigenvalues > 1. Rotation by the orthogonal method showed that the instrument questions confirmed the factors analyzed by the original instrument. The total Cronbach’s Alpha of ORIC was 0.94, showing excellent reliability.

Conclusion:

The Brazilian Portuguese Brazil version of the ORIC-Br questionnaire showed good psychometric properties and can be used in health services to measure organizational readiness, considered as an indicator of the potential success in implementing change.

Keywords:
Public health; Health management; Organizational capacity; Implementation science

INTRODUCTION

Organizational readiness for change in health services is considered an important precursor to the successful implementation of changes in the various organizational health settings¹1. Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009; 4. http://doi.org/10.1186/1748-5908-4-67
https://doi.org/http://doi.org/10.1186/1... , especially regarding technological innovations, public policies and health programs²2. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... . Strategies for developing organizational readiness are based on the premise that they aim to minimize the discrepancy between current and desired performance levels in the future with the new implementations³3. Armenakis AA, Harris SG. Crafting a change message to create transformational readiness. J Organizational Change Management 2002; 15(2): 169-83. http://doi.org/10.1108/09534810210423080
https://doi.org/http://doi.org/10.1108/0... .

Initiatives to increase the responsiveness of organizations through the implementation of new programs, policies or technologies often fail because management leaders do not establish sufficient organizational readiness with their employees⁴4. Leeman J, Moore A, Teal R, Barrett N, Leighton A, Steckler A. Promoting Community Practitioners' Use of Evidence-Based Approaches to Increase Breast Cancer Screening. Public Health Nurs 2013; 30(4): 323-31. http://doi.org/10.1111/phn.12021
https://doi.org/http://doi.org/10.1111/p... ^,55. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q 2004; 82(4): 581-629. http://doi.org/10.1111/j.0887-378X.2004.00325.x
https://doi.org/http://doi.org/10.1111/j... . This refers to the level at which health organization professionals are psychologically and behaviorally prepared to implement a change¹1. Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009; 4. http://doi.org/10.1186/1748-5908-4-67
https://doi.org/http://doi.org/10.1186/1... , and it is a construct that can be assessed at the individual or supra-individual level ― that is, at the level of the team, department or organization¹1. Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009; 4. http://doi.org/10.1186/1748-5908-4-67
https://doi.org/http://doi.org/10.1186/1... ^,22. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... . At the organizational level, the construct is defined as a shared psychological state in which the organization members feel committed and confident about their collective skills for implementation²2. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... . This collective level, which transcends individuals, is often measured in two main dimensions: commitment and effectiveness¹1. Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009; 4. http://doi.org/10.1186/1748-5908-4-67
https://doi.org/http://doi.org/10.1186/1... ^,22. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... ^,66. Weiner BJ, Amick H, Lee SYD. Conceptualization and measurement of organizational readiness for change - A review of the literature in health services research and other fields. Med Car Res Rev 2008; 65(4): 379-436. http://doi.org/10.1177/1077558708317802
https://doi.org/http://doi.org/10.1177/1... ^,77. Storkholm MH, Mazzocato P, Tessma MK, Savage C. Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementat Sci 2018; 13. http://doi.org/10.1186/s13012-018-0769-y
https://doi.org/http://doi.org/10.1186/s... ^,88. Ruest M, Léonard G, Thomas A, Desrosiers J, Guay M. French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Serv Res 2019; 19: 535. http://doi.org/10.1186/s12913-019-4361-1
https://doi.org/http://doi.org/10.1186/s... .

The commitment to change can be defined as the mindset that links an individual to actions considered necessary for the successful implementation of an innovation⁹9. Meyer JP, Stanley DJ, Herscovitch L, Topolnytsky L. Affective, continuance, and normative commitment to the organization: A meta-analysis of antecedents, correlates, and consequences. J Vocat Behav 2002; 61(1): 20-52. http://doi.org/10.1006/jvbe.2001.1842
https://doi.org/http://doi.org/10.1006/j... . The effectiveness of the change is related to the shared belief of the members of the organization in their collective capacities to organize and execute the actions involved in the innovation¹⁰10. Butler G. Self-efficacy: The exercise of control by Albert Bandura. Br J Clin Psychol 1998; 37(4): 467-473. https://doi.org/10.1111/j.2044-8260.1998.tb01404.x
https://doi.org/https://doi.org/10.1111/... .

The development of valid instruments to measure the degree to which a health organization can innovate according to its professionals is key to increasing the responsiveness of organizations to the new demands of the health sector. With this, instruments such as Organizational Readiness for Implementing Change (ORIC)²2. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... and Organizational Readiness for Knowledge Translation (OR4KT)¹¹11. Grandes G, Bully P, Martinez C, Gagnon MP. Validity and reliability of the Spanish version of the Organizational Readiness for Knowledge Translation (OR4KT) questionnaire. Implement Sci 2017; 12. http://doi.org/10.1186/s13012-017-0664-y
https://doi.org/http://doi.org/10.1186/s... ^,1212. Gagnon MP, Attieh R, Dunn S, Grandes G, Bully P, Estabrooks CA, et al. Development and Content Validation of a Transcultural Instrument to Assess Organizational Readiness for Knowledge Translation in Healthcare Organizations: The OR4KT. Int J Health Policy Manag 2018; 7(9); 791-7. https://dx.doi.org/10.15171%2Fijhpm.2018.17
https://doi.org/https://dx.doi.org/10.15... , built in the form of scales designed to measure organizational readiness, have received increasing attention among researchers. Despite this, none of these instruments has been tested, adapted and validated for Brazilian Portuguese.

The ORIC instrument was validated in Denmark⁷7. Storkholm MH, Mazzocato P, Tessma MK, Savage C. Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementat Sci 2018; 13. http://doi.org/10.1186/s13012-018-0769-y
https://doi.org/http://doi.org/10.1186/s... , in the context of a managerial policy to reduce funding and the number of beds in a private hospital where care would be provided with a reduction in the nursing staff, and in France⁸8. Ruest M, Léonard G, Thomas A, Desrosiers J, Guay M. French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Serv Res 2019; 19: 535. http://doi.org/10.1186/s12913-019-4361-1
https://doi.org/http://doi.org/10.1186/s... , to test the application of a new algorithm to help therapists to care for people with hygiene problems. Despite the importance of the primary health care network to respond to the needs of this area, and to face the increasing costs of specialized procedures in health systems, no study has tested the instrument at this level of care.

Considering Organizational Readiness to assist managers and policy operators to assess the degree of commitment and confidence of workers and professionals in the process of implementing changes, this study aimed to carry out cross-cultural adaptation to Brazilian Portuguese and to assess the internal consistency and the psychometric properties of the ORIC instrument, in a context of implementing change in primary health care in Brazil.

METHODS

STUDY CONTEXT

In Brazil, primary health care guidelines in their most comprehensive version have been increasingly adopted over the past few years. During this period, the Family Health Strategy (FHS) has become one of the main initiatives of the Unified Health System (SUS) to overcome the biomedical model, focusing on assistance to the disease, in its individual and biological aspects, centered on the procedure, on the medical specialties and the hospital. The goal is to achieve a model under an interdisciplinary approach, conducted by a multi-professional team, with a view to the integrality of care focused both on the territory and on the health needs of people and families¹³13. Matuda CG, Aguiar DMD, Frazao P. Interprofessional collaboration and the Brazilian Sanitary Reform: implications for delivery of healthcare. Saúde Soc 2013; 22(1): 173-86. http://doi.org/10.1590/S0104-12902013000100016
https://doi.org/http://doi.org/10.1590/S... .

Between 2018 and 2019, the Municipal Health Department of the Municipality of Campo Grande, Mato Grosso do Sul, was planning a reorganization of traditional primary care units in the municipality to meet the guidelines of the FHS. Professionals and workers from the 10 health units to be restructured were invited to participate in the research, assuming that this context of change was very appropriate to assess the psychometric properties of the ORIC instrument.

CROSS-CULTURAL ADAPTATION

A process of cultural adaptation of the English version of ORIC²2. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... was conducted based on the six steps of Beaton’s adapted methodology¹⁴14. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000; 25(24): 3186-91. http://doi.org/10.1097/00007632-200012150-00014
https://doi.org/http://doi.org/10.1097/0... . The Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist¹⁵15. Terwee CB, Prinsen CAC, Chioratto A, Westerman MJ, Patrick DL, Alonso J, et al. Cosmim methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res 2018; 27: 1159-70. http://doi.org/10.1007/s11136-018-1829-0
https://doi.org/http://doi.org/10.1007/s... , regarding content validation, was also used as a guide to report the results of the study.

Step 1: Translation of the Organizational Readiness for Implementing Change questionnaire

The ORIC questionnaire was translated from the original language (English) into Brazilian Portuguese by a bilingual translator (English/Portuguese), herein called translator 1.

Step 2: Synthesis of the translation of the Organizational Readiness for Implementing Change questionnaire

The synthesis of the translation was then verified by a committee formed by two researchers and translator 1. As recommended by Weiner⁶6. Weiner BJ, Amick H, Lee SYD. Conceptualization and measurement of organizational readiness for change - A review of the literature in health services research and other fields. Med Car Res Rev 2008; 65(4): 379-436. http://doi.org/10.1177/1077558708317802
https://doi.org/http://doi.org/10.1177/1... , the questionnaire was modified with the inclusion of an introductory description of the implementation of the change (restructuring the Basic Health Unit - BHU - into the Basic Family Health Unit - BFHU model), in order to clarify the meaning of this change, which was linked to the restructuring of the traditional BHU into the BFHU model. The committee approved the translation without disagreement.

Step 3: Back-translation

This step involved a team of two researchers and a second independent translator. The independence between the translators in steps 1 and 3 helped to rigorously document the adequacy of the terms chosen for the formulation used in the original version of the questionnaire.

Step 4: Special review committee

This stage involved the three researchers in this study, plus the two translators (translation and back-translation), in order to compare the two versions. Disagreements were resolved based on consensus. There were three conflicting points regarding the front- and back-translation. The first was in the ORIC 1 question, which involved the replacement of the term “invested”, which, in Portuguese (“investido”), is linked to money or monetary resources, with the term “engage” (“envolver”), which improved the interpretation of question 1, according to the special review committee. In the ORIC 7 question, the word “enquanto”, translated as “while”, could give the idea that people are maintaining the normal pace of the health organization while the changes are being implemented. So, we chose to insert the words “of the change implementation” (“da implementação da mudança”) instead of “while implementing that change” (“enquanto implementam essa mudança”). The third conflicting point was the ORIC 8 question. We opted for the word manage (“lidar”) instead of overcome (“superar”), as it could give the impression that the implementation would be an obstacle to be overcome. The committee considered that the implementation would be useful to make services better, and not create obstacles to be overcome. Therefore, this step produced a final product to be tested in Step 5 - pre-test version (Appendix 1).

Step 5: Pilot/pre-test version

The pilot version of the instrument was applied to eight workers from a health unit participating in the change, to analyze the individuals' perceptions of the instrument, according to the COSMIN checklist¹⁵15. Terwee CB, Prinsen CAC, Chioratto A, Westerman MJ, Patrick DL, Alonso J, et al. Cosmim methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res 2018; 27: 1159-70. http://doi.org/10.1007/s11136-018-1829-0
https://doi.org/http://doi.org/10.1007/s... . This step involved the participation of one researcher, who collected information about the perception of the instrument by primary care workers. There were no changes to be made to the questionnaire. The average time for the individual response of each worker to the questionnaire was 5 min.

Step 6: Approval of the instrument by the special review committee

As no major changes were made to the questionnaire after the pre-test version, the special committee approved the translation of the instrument (Appendix 1) to be applied among study participants.

APPLICATION OF THE QUESTIONNAIRE IN THE PARTICIPATING HEALTH UNITS

According to information from the Municipal Health Department, 215 hired employees, including doctors, dentists, community health workers, nurses, technical professionals and managers, were allocated to the 10 primary care units participating in the study. Eight workers who tested the pilot version (step 5) were not included in the sample.

MEASURES

The ORIC questionnaire contains 12 items that correspond to the domains of commitment and effectiveness. It uses a 5-point Likert scale (1 = strongly disagree and 5 = strongly agree). To facilitate the analysis, the items were grouped according to the two organizational factors (commitment and effectiveness). The questions related to the analysis of the commitment are: 2, 4, 6, 9 and 11. The questions related to the analysis of the effectiveness are 1, 3, 5, 7, 8, 10 and 12.

APPLICATION OF THE FINAL VERSION OF THE INSTRUMENT

The final instrument, after analysis by the special advisory committee (Appendix 1), was applied personally by one of the researchers to workers from participating health units.

STATISTICAL ANALYSIS

Descriptive statistics were used to characterize the responses of respondents from participating health units. The covariables used were the participant's position, subdivided into technical personnel (community health agents, administrative assistants, nursing technicians, oral health assistants, health service assistants), university personnel (doctors, dentists, pharmacists, nurses and social workers) and managers (professionals responsible for the management of health units), age in years and sex (male and female).

The internal consistency of the questionnaire was measured by Cronbach’s alpha¹⁶16. Bland J, Altman D. Cronbach's alpha. Br Med 1997; 314: 572. https://doi.org/10.1136/bmj.314.7080.572
https://doi.org/https://doi.org/10.1136/... . The internal consistency of the effectiveness and commitment domains was also measured, separately.

Prior to the exploratory factor analysis, the correlation matrix between the questions in the questionnaire was determined and the determinant of the correlation matrix, the Barlett sphericity test, was analyzed, indicating whether or not it is an identity matrix that has a null hypothesis in the absence of correlation between variables. Thus, the rejection of the null hypothesis is indicative of the existence of correlation, justifying the exploratory factor analysis (EFA). Kaiser-Meyer-Olkin (KMO) values above 0.5 were considered adequate for EFA¹⁷17. Kline P. An easy guide to factor-analysis. Londres: Routledge; 2014. 208 p...

All tests were performed using the factortest command in the Stata software (College Station, TX, United States).

EFA was performed using principal axis analysis to assess dimensionality. Eigenvalues greater than 1 were retained as the main factors and, then, the rotation was performed by the Varimax orthogonal method, to assign weight to each item in each of the retained factors. Subsequently, the construct validity in the dimensions of the instrument was investigated thorough confirmatory factor analysis (CFA), using a structural equation model. CFA was performed using the Robust Maximum Likelihood method, due to the ordinal data of the instrument.

According to Myers et al. criteria¹⁸18. Myers ND, Ahn S, Jin Y. Sample Size and Power Estimates for a Confirmatory Factor Analytic Model in Exercise and Sport: A Monte Carlo Approach. Res Q Exerc Sport 2011; 82(3): 412-23. http://doi.org/10.1080/02701367.2011.10599773
https://doi.org/http://doi.org/10.1080/0... , a sample of around 120 respondents would be sufficient for an instrument with twelve questions. To access the fit quality of the models, comparative fit index (CFI) > 0.90, Tucker-Lewis index (TLI) > 0.90, root mean square error of approximation (RMSEA) < 0.06 and standardized root mean square residual (SRMR) < 0.05¹⁷17. Kline P. An easy guide to factor-analysis. Londres: Routledge; 2014. 208 p.. and Cronbach’s alpha above 0.7 were considered for an acceptable internal consistency¹⁹19. Bernardi RA. Validating research results when Cronbachs-Alpha is below .70 - a methodological procedure. Educ Psychol Meas 1994; 54(3): 766-75. http://doi.org/10.1177/0013164494054003023
https://doi.org/http://doi.org/10.1177/0... . All analyzes were performed using the STATA v.14 software (College Station, TX, United States).

Four models were carried out by structural equation to access the fit quality of the commitment and effectiveness domains of the ORIC questionnaire²⁰20. Nunes RD, Parma GC, de Campos AC, Locatelli P, Traebert J. Cross-cultural adaptation and psychometric properties of the Brazilian-Portuguese version of the Quality of Prenatal Care Questionnaire (QPCQ). Rev Saúde Pública 2019; 53: 1. http://doi.org/10.11606/S1518-8787.2019053000565
https://doi.org/http://doi.org/10.11606/... .

In model 1, the effectiveness domain was tested using structural equation modelling, with all items of the effectiveness factor. In model 2, the same domain was tested without including the ORIC 1 question, according to the Danish version⁷7. Storkholm MH, Mazzocato P, Tessma MK, Savage C. Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementat Sci 2018; 13. http://doi.org/10.1186/s13012-018-0769-y
https://doi.org/http://doi.org/10.1186/s... .

In model 3, the commitment domain was evaluated and, in the final model (model 4), the fit was tested with the items commitment and effectiveness correlated with each other, as recommended by Weiner’s theory¹1. Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009; 4. http://doi.org/10.1186/1748-5908-4-67
https://doi.org/http://doi.org/10.1186/1... . Asymmetry and kurtosis were evaluated²¹21. Meda SA, Stevens MC, Potenza MN, Pittman B, Gueorguieva R, Andrews MM, et al. Investigating the behavioral and self-report constructs of impulsivity domains using principal component analysis. Behav Pharmacol 2009; 20(5-6): 390-9. http://doi.org/10.1097/FBP.0b013e32833113a3
https://doi.org/http://doi.org/10.1097/F... .

ETHICAL ASPECTS

The study followed the precepts of Resolution No. 466/2012 of the National Health Council and was approved by the Research Ethics Committee of the Universidade Federal do Mato Grosso do Sul (Opinion No. 3.101.971). All participants signed an Informed Consent.

RESULTS

The instrument was answered by a sample of 150 workers from the 10 participating health units, corresponding to the response rate of 72.5%. Women represented 76.7% of the sample and, among the participants, 58.5% were aged between 35 and 65. As for the position/function, 76% of the respondents were auxiliary and technical level workers, 21.6% had higher-level functions (doctors, nurses, dentists, etc.) and 3.4% were managers of the primary care units (Table 1).

Cronbach’s alpha values were 0.91, 0.90 and 0.94, respectively, for the domains of effectiveness, commitment and for the total instrument, showing excellent internal consistency.

Barlett’s sphericity test (p = 0.000) showed that the variables were not interrelated, which was favorable for EFA. The KMO value was 0.91, confirming the findings.

The EFA indicated two main factors (commitment and effectiveness) with eigenvalues greater than 1. After rotation by the Varimax orthogonal method, the two retained factors explained 71.9% of the variability of the findings.

According to Table 2, all questions regarding the commitment domain - ORIC 2 (weight = 0.71), ORIC 4 (weight = 0.75), ORIC 6 (weight = 0.85), ORIC 11 (weight = 0, 65) and ORIC 9 (weight = 0.64) presented adequate values to explain this factor. The questions regarding the effectiveness domain, ORIC 3 (weight = 0.50), ORIC 5 (weight = 0.68), ORIC 7 (weight = 0.81), ORIC 8 (weight = 0.82), ORIC 10 (weight = 0.80), ORIC 12 (weight = 0.80) presented weights above 0.4, except for item 1 (weight = 0.37), which was excluded from the final version of the ORIC-Br instrument (Appendix 2).

Table 3 shows the values of the fit quality of confirmatory analyzes by structural equation modelling. Among the four models carried out, all indicated good fit qualities for the measurement of domains, according to Weiner’s theory¹1. Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009; 4. http://doi.org/10.1186/1748-5908-4-67
https://doi.org/http://doi.org/10.1186/1... .

Figure 1 shows the values of the adjusted coefficients for each question of the instrument in relation to the measurement of each domain without the presence of item 1. All values were significant (p < 0.001), indicating the importance of each question for the respective domains (commitment and effectiveness).

The possible relationship between the respondents’ characteristics and the domains was investigated. Characteristics such as age, function and sex were not significant (p > 0.05) in relation to the analyzed domains (impairment and effectiveness). The asymmetry and kurtosis of all items were also evaluated and presented values between 3 and 7, which were considered adequate.

DISCUSSION

The present study showed an important finding. The ORIC-Br instrument showed adequate internal consistency and good psychometric properties to measure, in the form of scales, the organizational readiness for implementing changes in primary health care services in Brazil.

The results were similar to those observed in other countries, since both the Danish⁷7. Storkholm MH, Mazzocato P, Tessma MK, Savage C. Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementat Sci 2018; 13. http://doi.org/10.1186/s13012-018-0769-y
https://doi.org/http://doi.org/10.1186/s... and the French⁸8. Ruest M, Léonard G, Thomas A, Desrosiers J, Guay M. French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Serv Res 2019; 19: 535. http://doi.org/10.1186/s12913-019-4361-1
https://doi.org/http://doi.org/10.1186/s... versions showed good psychometric properties for measuring organizational readiness through CFA. In Denmark, the instrument was tested in the context of a new management policy, aimed at reducing the use of beds (36%) and financing (10%) in a private hospital, where care would be performed with a reduced number of nursing professionals. In France⁸8. Ruest M, Léonard G, Thomas A, Desrosiers J, Guay M. French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Serv Res 2019; 19: 535. http://doi.org/10.1186/s12913-019-4361-1
https://doi.org/http://doi.org/10.1186/s... , the instrument was tested before applying a new algorithm to help occupational therapists to care for people with hygiene problems.

In the present research, the ORIC-Br instrument was tested prior to a change implementation process in primary health care, which aimed to introduce principles related to the FHS. In all contexts, the importance and capacity of the instrument to measure the degree to which workers were able to implement changes in health services was observed.

Similar to other studies²2. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... ^,77. Storkholm MH, Mazzocato P, Tessma MK, Savage C. Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementat Sci 2018; 13. http://doi.org/10.1186/s13012-018-0769-y
https://doi.org/http://doi.org/10.1186/s... ^,88. Ruest M, Léonard G, Thomas A, Desrosiers J, Guay M. French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Serv Res 2019; 19: 535. http://doi.org/10.1186/s12913-019-4361-1
https://doi.org/http://doi.org/10.1186/s... , item 1 was excluded from the ORIC-Br instrument because it did not present adequate compliance with the EFA²²22. Dias JCR, Maroco J, Campos J. Weight Concerns Scale Applied to College Students: Comparison Between Pencil-and-Paper and Online Formats. Cyberpsychol Behav Soc Netw 2015; 18(3): 188-92. http://doi.org/10.1089/cyber.2014.0392
https://doi.org/http://doi.org/10.1089/c... (Appendix 2). In the final version with 11 items, after the exclusion of item 1 from the pre-test version, the new arrangement of questions in the ORIC-Br instrument showed that items 1, 3, 5, 8 and 10 measured commitment and items 2, 4, 6, 7, 9 and 11 measured effectiveness. In contrast, in the instrument validated in France⁸8. Ruest M, Léonard G, Thomas A, Desrosiers J, Guay M. French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Serv Res 2019; 19: 535. http://doi.org/10.1186/s12913-019-4361-1
https://doi.org/http://doi.org/10.1186/s... , the cross-cultural adaptation process removed two items from the instrument, questions ORIC 1 and ORIC 7, even without going through EFA. The authors and the French executive committee found that the two items were not in conformity in the translation and back-translation process. Thus, the French version was validated with 10 items on the instrument, five measuring commitment and five measuring effectiveness. It should be noted that the authors of the original instrument left question 1 on the instrument to allow it to be subjected to tests and verifications in later studies, such as the one presented in this manuscript and in the Danish version. Regarding the internal reliability of the instrument, measured by Cronbach's alpha, the ORIC-Br version presented satisfactory values¹⁶16. Bland J, Altman D. Cronbach's alpha. Br Med 1997; 314: 572. https://doi.org/10.1136/bmj.314.7080.572
https://doi.org/https://doi.org/10.1136/... ^,1919. Bernardi RA. Validating research results when Cronbachs-Alpha is below .70 - a methodological procedure. Educ Psychol Meas 1994; 54(3): 766-75. http://doi.org/10.1177/0013164494054003023
https://doi.org/http://doi.org/10.1177/0... , similarly to the American²2. Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci 2014; 9: 7. http://doi.org/10.1186/1748-5908-9-7
https://doi.org/http://doi.org/10.1186/1... , Danish⁷7. Storkholm MH, Mazzocato P, Tessma MK, Savage C. Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementat Sci 2018; 13. http://doi.org/10.1186/s13012-018-0769-y
https://doi.org/http://doi.org/10.1186/s... and French⁸8. Ruest M, Léonard G, Thomas A, Desrosiers J, Guay M. French cross-cultural adaptation of the Organizational Readiness for Implementing Change (ORIC). BMC Health Serv Res 2019; 19: 535. http://doi.org/10.1186/s12913-019-4361-1
https://doi.org/http://doi.org/10.1186/s... instruments.

In Brazil, comprehensive services that allow individuals and families to obtain the care they need have been a constant concern in the reorientation of health care systems. This process implies changes that demand new strategies to strengthen the response capacity of both hospitals and specialized services and primary care, in order to improve the commitment in organizations and the cooperative relations between professionals and workers¹³13. Matuda CG, Aguiar DMD, Frazao P. Interprofessional collaboration and the Brazilian Sanitary Reform: implications for delivery of healthcare. Saúde Soc 2013; 22(1): 173-86. http://doi.org/10.1590/S0104-12902013000100016
https://doi.org/http://doi.org/10.1590/S... . In this sense, due to its ease of application, the ORIC-Br instrument can be very useful to quickly access (average time of 5 min for its application) the organizational readiness in health services, and can offer an important contribution to the production of knowledge related to the implementation of new work processes, technologies and innovations in health. Its application is wide and can be used in different contexts and organizations in the sector and encompassing processes for changing remote and/or rural health services²³23. Hossain N, Yokota F, Sultana N, Ahmed A. Factors Influencing Rural End-Users' Acceptance of e-Health in Developing Countries: A Study on Portable Health Clinic in Bangladesh. Telemed J E Health 2019; 25(3): 221-9. http://doi.org/10.1089/tmj.2018.0039
https://doi.org/http://doi.org/10.1089/t... , the use of artificial intelligence²⁴24. Bertoncelli CM, Altamura P, Vieira ER, Bertoncelli D, Solla F. Using Artificial Intelligence to Identify Factors Associated with Autism Spectrum Disorder in Adolescents with Cerebral Palsy. Neuropediatrics 2019; 50(3): 178-87. http://doi.org/10.1055/s-0039-1685525
https://doi.org/http://doi.org/10.1055/s... , among other aspects.

Other instruments have been developed for the translation of knowledge by health professionals, with excellent psychometric properties, such as OR4KT¹¹11. Grandes G, Bully P, Martinez C, Gagnon MP. Validity and reliability of the Spanish version of the Organizational Readiness for Knowledge Translation (OR4KT) questionnaire. Implement Sci 2017; 12. http://doi.org/10.1186/s13012-017-0664-y
https://doi.org/http://doi.org/10.1186/s... ^,1212. Gagnon MP, Attieh R, Dunn S, Grandes G, Bully P, Estabrooks CA, et al. Development and Content Validation of a Transcultural Instrument to Assess Organizational Readiness for Knowledge Translation in Healthcare Organizations: The OR4KT. Int J Health Policy Manag 2018; 7(9); 791-7. https://dx.doi.org/10.15171%2Fijhpm.2018.17
https://doi.org/https://dx.doi.org/10.15... . This instrument has the disadvantage of having 39 items and requires longer application time, but it offers a construct with six dimensions:

The R = MC2²⁵25. Scaccia JP, Cook BS, Lamont A, Wandersman A, Castellow J, Katz J, et al. A practical implementation science heuristic for organizational readiness: R = MC2. J Community Psychol 2015; 43(4): 484-501. http://doi.org/10.1002/jcop.21698
https://doi.org/http://doi.org/10.1002/j... approach, which also accesses organizational readiness, has the disadvantage of not being an instrument organized in scales, but a more qualitative assessment that involves three dimensions of organizational readiness.

This study has some strengths and limitations. It is the first study, to the knowledge of the authors, that tested the ORIC questionnaire in health services in Brazil, at a time prior to the process of implementing a change that included the reorientation of traditional primary care units to the FHS guidelines. It proved to be a reliable, valid and easy to apply instrument in the Brazilian context. As limitations, concurrent validation with another instrument could contribute to the evaluation of the instrument, however, as implementation science is a field that is still advancing in Brazil, there are no valid instruments in Portuguese for its use and subsequent validation. Another limitation is that the instrument's retest was not evaluated, a limitation also mentioned in the validation of the Danish version of the instrument.⁷7. Storkholm MH, Mazzocato P, Tessma MK, Savage C. Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC). Implementat Sci 2018; 13. http://doi.org/10.1186/s13012-018-0769-y
https://doi.org/http://doi.org/10.1186/s... . The validation of the instrument in different contexts also deserves to be considered, considering that the Brazilian territory has great socioeconomic and human development variation among its 5,570 municipalities.

For health managers interested in implementing new policies, innovations or new programs in health services, the ORIC-Br questionnaire can be a useful and appropriate tool for measuring organizational readiness, and can be an important predictor for the success of implementing changes in health services.

ACKNOWLEDGEMENTS

This study was partially funded by Universidade Federal do Mato Grosso do Sul (UFMS). We would like to thank all the volunteers and professional assistants who participated in this study. Funders had no role in the study design, data collection or analysis, publication decision or preparation of the manuscript.

References

Publication Dates

History