ABSTRACT

OBJECTIVE

To assess the construct validity and reliability of the Portuguese version of the European Task Force on Patient Evaluation of General Practice Care questionnaire.

METHODS

We applied the Portuguese version of the European Task Force on Patient Evaluation of General Practice Care to 392 users of 20 Family Health Units from the North of Portugal. The validity of the construct was evaluated by exploratory factor analysis, with the Principal Axis Factoring method, by orthogonal rotation (varimax procedure), by the Kaiser normalization criteria (eigenvalue ≥ 1). The factorability of the data matrix was verified by the Kaiser-Meyer-Olkin and Bartlett’s sphericity test. We estimated the reliability by the indicator of internal consistency Cronbach’s alpha. To analyze the correlations between satisfaction and loyalty, we used the Pearson correlations. The predictor effect of satisfaction on loyalty was analyzed by simple linear regression.

RESULTS

Satisfaction presented five robust and well individualized dimensions – medical care, nursing care, clinical secretariat services, accessibility, and organization of services – with alpha values between 0.86 and 0.97, good levels of internal consistency. The loyalty showed alpha value of 0.72, considered a reasonable internal consistency. The satisfaction was predictive of loyalty.

CONCLUSIONS

The Portuguese European Task Force on Patient Evaluation of General Practice Care questionnaire is a robust and reliable instrument to measure the satisfaction and loyalty of users of the Family Health Units.

Surveys and Questionnaires; Translations; Psychometrics; Self Care; Quality of Health Care; Patient Satisfaction; Reproducibility of Results; Validation Studies

INTRODUCTION

Quality in health is currently a requirement for all those involved in health care. The quality indicators have been developed in the hospital context and their use quickly spread to primary health care. From an initial focus primarily on scientific and technical aspects, they evolved into the consideration of relational and satisfaction aspects of users²⁰20. Mendonça KMPP, Guerra RO. Desenvolvimento e validação de um instrumento de medida da satsifação do paciente com a fisioterapia. Rev Bras Fisioter. 2007;11(5):369-76. DOI:10.1590/S1413-35552007000500007
https://doi.org/10.1590/S1413-3555200700... . Their usefulness can extend to the own goals of health services. In fact, the received feedback allows to introduce actions of correction and improvement to increase the levels of satisfaction of users¹³13. Giraldes MR. Eficiência versus qualidade no Serviço Nacional de Saúde: metodologias de avaliação. Acta Med Port. 2008;21(5):397-410.^,2222. Pisco L, Biscaia J. Qualidade de cuidados de saúde primários. Rev Port Saude Publica. 2001;(2):43-51..

Satisfaction is an individual perception linked to the achievement of expectations or satisfaction of a need, by the individual or with intervention of others, whose evaluation is carried out from the person’s perspective. Regarding primary health care, the satisfaction is equivalent to the well-being of users, manifested in their opinion about the quality of the obtained services²⁴24. Ribeiro AL. O percurso da construção e a validação de um instrumento para avaliação da satisfação dos utentes em relação aos cuidados de enfermagem. Rev Ordem Enferm. 2005;(16):53-60.. It is a dynamic process influenced by individual, psychological, and sociocultural factors, among others. Because it is a multidimensional concept, the satisfaction concerning primary health care can feature an assessment of several aspects, such as accessibility, user-health professional relationship, infrastructure, and health service results⁷7. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess. 2002;6(32):1-244. DOI:10.3310/hta6320
https://doi.org/10.3310/hta6320... ^,99. Esperidião M, Trad LA Bonfim. Avaliação de satisfação de usuários. Cienc Saude Coletiva. 2005;10 Supl:303-12. DOI:10.1590/S1413-81232005000500031
https://doi.org/10.1590/S1413-8123200500... . Thus, to improve the provided services, it is essential to approach the maximum satisfaction of the requirements and needs of users. For this, it is necessary to listen to the views of users on how the services contributed to improve or solve their health state²²22. Pisco L, Biscaia J. Qualidade de cuidados de saúde primários. Rev Port Saude Publica. 2001;(2):43-51.^,2424. Ribeiro AL. O percurso da construção e a validação de um instrumento para avaliação da satisfação dos utentes em relação aos cuidados de enfermagem. Rev Ordem Enferm. 2005;(16):53-60.. This concept of involvement and active participation of users in the provision of health care came from the USA, being later adopted in Europe. Health professionals and users perceive differently the results of the services provided in health. For this reason, the contribution of the user on the quality of services provided is essential. As active agents, the users share their negative and positive experiences and their needs and expectations, contributing to a response system that controls, identifies, and corrects the health system deficits⁴4. Cabete J, Lencastre A, Apetato M, Paiva L. Porque reclamam os nossos doentes? Casuística de uma consulta de dermatologia. Acta Med Port. 2012;25(5):265-70.^,2727. Wensing M, Elwyn G. Methods for incorporating patients’ views in health care. BMJ. 2003;326(7394):877-9. DOI:10.1136/bmj.326.7394.877
https://doi.org/10.1136/bmj.326.7394.877... . Additionally, users’ satisfaction increasingly occupies an important place in assessing the quality, aware of their valorization as active agents and consumers, because it was found that their satisfaction is related to therapeutic adherence and with results achieved and effectiveness of interventions, contributing to change health behaviors¹1. Albuquerque AB, Deveza M. Adesão ao tratamento na prática do médico de família e comunidade e na Atenção Primária à Saúde. In: Programa de Atualização em Medicina de Família e Comunidade - PROMEF. Porto Alegre: Artmed; 2009. v. 3, p.41-71.^,33. Brandão ALRBS, Giovanella L, Campos CEA. Avaliação da atenção básica pela perspectiva dos usuários: adaptação do instrumento EUROPEP para grandes centros urbanos brasileiro. Cienc Saude Coletiva. 2013;18(1):103-14. DOI:10.1590/S1413-81232013000100012
https://doi.org/10.1590/S1413-8123201300... ^,1919. Kantorski LP, Jardim VR, Wetzel C, Olschowsky A, Schneider JF, Heck RM, et al. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saude Publica. 2009;43 Supl 1:29-35. DOI:10.1590/S0034-89102009000800006
https://doi.org/10.1590/S0034-8910200900... ^,2323. Portela MC, Lima SML, Brito C, Ferreira VMB, Escosteguy CC, Vasconcellos MTLD. Programa de Controle da Tuberculose e satisfação dos usuários, Rio de Janeiro. Rev Saude Publica. 2014;48(3):497-507. DOI:10.1590/S0034-8910.2014048004793
https://doi.org/10.1590/S0034-8910.20140... .

Thus, it is necessary to engage the population in the construction of instruments for measuring satisfaction, which will allow understanding the expectations of users. The evaluation of services must be multidimensional, considering specific aspects that influence the overall satisfaction. It is necessary to appeal to strict qualitative studies and methods that allow the inclusion of the users’ perspective in the construction processes and their results (questionnaires, tests, interview guides etc.). And the possible negative consequences must also be contemplated, as negative rating or expression of negative expectations, i.e., the respondents must have the opportunity and own space to express their ideas freely¹⁸18. Henriques J, Alexandra D. Medicina geral e familiar: desafios hoje e no futuro. Acta Med Port. 2014;27(3):287-8.^,2121. Moura AC, Ferreira MA, Barbosa J, Mourão J. Satisfação com os cuidados anestésicos num hospital central. Acta Med Port. 2013;27(1):33-41.^,2424. Ribeiro AL. O percurso da construção e a validação de um instrumento para avaliação da satisfação dos utentes em relação aos cuidados de enfermagem. Rev Ordem Enferm. 2005;(16):53-60.^,2727. Wensing M, Elwyn G. Methods for incorporating patients’ views in health care. BMJ. 2003;326(7394):877-9. DOI:10.1136/bmj.326.7394.877
https://doi.org/10.1136/bmj.326.7394.877... .

The European Task Force on Patient Evaluation of General Practice Care (EUROPEP) questionnaire is widely used to evaluate the satisfaction of users with the primary health care, focusing on their perspective of final consumers and active agents in the therapeutic process. This instrument has been prepared using a comprehensive approach, which includes analyses of other investigations and theoretical concepts (literature analysis), interviews with users, and validation studies. The questionnaire aims to evaluate five dimensions: (1) doctor-patient relationship (e.g.: ease with which patients feel comfortable to tell their problems); (2) medical care (e.g.: clinical examination done by the doctor); (3) information and support (e.g.: explanation of the medicines, treatments, and examinations prescribed); (4) organization of services (e.g.: preparation on what to expect of hospital care, other experts, or other care workers); (5) accessibility (e.g.: ease to make an appointment in the family health units (FHU))²2. Bjertnaes O, Lyngstad I, Malterud K, Garrat A. The Norwegian EUROPEP questionnaire for patient evaluation of general practice: data quality, reliability and construct validity. Fam Pract. 2011;28(3):342-9. DOI:10.1093/fampra/cmq098
https://doi.org/10.1093/fampra/cmq098... ^,88. Dagdeviren N, Akturk Z. An evaluation of patient satisfaction in Turkey with the EUROPEP instrument. Yonsei Med J. 2004;45(1):23-8. DOI:10.3349/ymj.2004.45.1.23
https://doi.org/10.3349/ymj.2004.45.1.23... ^,1010. Ferreira PL, Raposo V. A governação em saúde e a utilização de indicadores de satisfação. Rev Port Clin Geral. 2006;22(3):285-96.^,1111. Ferreira PL, Antunes P, Portugal S. O valor dos cuidados primários: perspectiva dos utilizadores das USF - 2009. Lisboa: Ministério da Saúde; 2010.^,1414. Grol R, Wensing M, Mainz J, Ferrreira P, Hearnshaw H, Hjortdahl P, et al. Patients’ priorities with respect to general practice care: an international comparison. Fam Pract. 1999;16(1):4-11. DOI:10.1093/fampra/16.1.4
https://doi.org/10.1093/fampra/16.1.4...

15. Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, et al. Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract. 2000;50(460):882-7.^-1616. Grol R, Wensing M, Kersnik J. Patients evaluate general/family practice: the EUROPEP instrument. Nijmegen, NL: University Medical Center; 2000.^,2626. Wensing M, Mainz J, Grol R. A standardised instrument for patient evaluations of general practice care in Europe. Eur J Gen Pract. 2000;6(3):82-7. DOI:10.3109/13814780009069953
https://doi.org/10.3109/1381478000906995... .

In this context, the aim of this study was to assess the construct validity and reliability of the Portuguese EUROPEP questionnaire.

METHODS

The study was conducted at the FHU from the districts of Braga, Porto, and Viana do Castelo, which have 172 of the 353 FHU operating in Portugal²⁵25. USF-AN.pt [Internet]. Porto: Associação Nacional de Unidades de Saúde Familiar. [citado 2014 Nov 9]. Available from: http://www.usf-an.pt/index.
http://www.usf-an.pt/index... . From the invited FHU, 20 agreed to participate in this study.

Participants filled out the questionnaires in person in the waiting rooms of the FHU, while waiting for the appointment or after they have been met. The filling occurred in the self and hetero-report format, the latter only on users with low level of education or no education. The collection took place between October 2013 and January 2014. 432 questionnaires were collected, from which 40 were eliminated because they had blank answers or maximum quotation on all items, totaling 392 valid questionnaires.

The 392 users evaluated were aged between 14 and 84 years old, with average of 41.7 years (SD = 15.9). The majority was female (77.3%), married (57.4%), and has already used the FHU services more than once (98.2%). Last year, the participants used the FHU services, on average, 4.7 times (SD = 5.0). The evaluated users were enrolled at the FHU between zero and seven years, with average of 3.9 years (SD = 1.6). Only 13.0% of participants responded in the hetero-report format.

We used the Portuguese version of the EUROPEP, adapted by Ferreira et al.¹¹11. Ferreira PL, Antunes P, Portugal S. O valor dos cuidados primários: perspectiva dos utilizadores das USF - 2009. Lisboa: Ministério da Saúde; 2010. This instrument features 37 items that assess the level of satisfaction from the perspective of users. The items are rated on likert type scale of five points (0 = bad; 4 = excellent). The EUROPEP questionnaire contains, still, three items related to the loyalty of the user toward the FHU⁵5. Ciavolino E, Dahlgaard JJ. ECSI – Customer Satisfaction Modelling and Analysis: a case study. Total Qual Manag Bus Excell. 2007;18(5):545-54. DOI:10.1080/14783360701240337.
https://doi.org/10.1080/1478336070124033... ^,1717. Gronholdt L, Martensen A, Kristensen K. The relationship between customer satisfaction and loyalty: cross-industry differences. Total Qual Manag. 2000;11(4-6):509-14. DOI:10.1080/09544120050007823
https://doi.org/10.1080/0954412005000782... , rated from “0 = disagree a lot” to “4 = agree a lot”.

The construct validity was evaluated by exploratory factor analysis, with the Principal Axis Factoring method, by orthogonal rotation (varimax procedure), by the Kaiser normalization criteria (eigenvalue ≥ 1). The factorability of the data matrix was verified by the Kaiser-Meyer-Olkin and Bartlett’s sphericity test. We estimated the reliability by the indicator of internal consistency Cronbach’s alpha. We used the Pearson correlations to analyze the correlations between satisfaction and loyalty, and simple linear regression to verify the predictive effect of satisfaction on loyalty.

This project was approved by the Ethics Committee of the Regional Administration of North Health, Portugal. All participants signed the informed consent form in accordance with the Helsinki Declaration and the Convention of Oviedo.

RESULTS

Satisfaction

The correlation matrix analysis showed significant correlations (p < 0.001) with values between r = 0.24 and r = 0.78. There were no items excessively or perfectly correlated, and it may be affirmed that there are no problems of multicollinearity and singularity, respectively¹²12. Field A. Discovering statistics using SPSS. 2.ed. London: Sage Publications; 2005.. The analysis of commonalities showed values between 0.46 and 0.87, with average of 0.65, reflecting that 65.0% of the variance associated to items is common or shared. We verified factorability in the data matrix, because the value of Kaiser-Meyer-Olkin (KMO) was greater than 0.60 (KMO = 0.96) and the Bartlett’s sphericity test was significant (p < 0.001).

The internal consistency, estimated by the Cronbach’ alpha, presented high value (0.97) to the full scale, and no items changed, significantly, this value.

The exploratory factor analysis extracted five factors that explain 65.4% of the variance: medical care (31.4%; α = 0.97), organization of services (10.6%; α = 0.89), accessibility (9.9%; α = 0.86), nursing care (7.1%; α = 0.87), and clinical secretariat services (6.4%; α = 0.92).

In Table 1, we present the values of the factor saturation of each item, percentage of total variance, internal consistency of each factor, and the commonalities (h²) regarding the satisfaction of users.

For the retention of items on each scale, the following criteria have been fixed: (1) saturation ≥ 0.40 of each item in the hypothetical factor and only in one single factor; (2) the final factor solution explaining at least 50.0% of the total variance; (3) consistency between the factor solution and items that constitute each factor; and (4) each factor being represented by at least three items⁶6. Costello A, Osborne JW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10(7):1-9.^,1212. Field A. Discovering statistics using SPSS. 2.ed. London: Sage Publications; 2005.. We removed two items in the sequence of the retention criteria: “The respect with which you were treated and how your privacy was maintained” and “The quickness with which the urgent health problems have been resolved”, because they had a factor loading < 0.40 and similar factor loadings in more than one factor.

In addition to the above procedure, we also performed the exploratory factor analysis by principal component method and maximum likelihood with oblique rotation. These showed similar results, with differences only in the order of the factors.

Loyalty

The correlation matrix analysis showed significant correlations (p < 0.001), with values between r = 0.35 and r = 0.62. There were no items excessively or perfectly correlated, and it may be affirmed that there are no problems of multicollinearity and singularity, respectively¹²12. Field A. Discovering statistics using SPSS. 2.ed. London: Sage Publications; 2005.. The analysis of commonalities showed values between 0.25 and 0.76, with average of 0.50, reflecting that 50.0% of the variance associated to items is common or shared. We verified factorability in the data matrix, because the value of KMO was greater than 0.60 (KMO = 0.63) and the Bartlett’s sphericity test was significant (p < 0.001).

The exploratory factor analysis extracted one factor – loyalty – that explains 50.2% of the variance with α = 0.72.

In Table 2, we present the values of the factor saturation of each item, percentage of total variance, internal consistency of the factor, and the commonalities (h²) regarding the loyalty. For the retention of the items, we fixed the same criteria used in satisfaction. All items fulfilled the criteria.

Relation Satisfaction/Loyalty

The Pearson correlations show significant positive associations (p < 0.01) between satisfaction and its dimensions and loyalty. The highest value was in overall satisfaction, showing that high values of satisfaction are associated with high values of loyalty (r = 0.40; p < 0.01).

The simple linear regression showed that the overall satisfaction of users explains 16.0% of the variance of loyalty (R²aj = 0.16; p < 0.001) (F_1;390 = 74.07; p < 0.001), and, the higher the levels of overall satisfaction, the higher the levels of loyalty with the FHU.

DISCUSSION

The results show that the items are good indicators of the constructs intended to measure – satisfaction and loyalty – and that the factors are properly individualized. Regarding satisfaction, the results show five robust dimensions, which allow a fairly broad understanding of the users’ satisfaction. Some of these dimensions – medical care, accessibility, and organization of services – find parallels in other validations of the EUROPEP questionnaire²2. Bjertnaes O, Lyngstad I, Malterud K, Garrat A. The Norwegian EUROPEP questionnaire for patient evaluation of general practice: data quality, reliability and construct validity. Fam Pract. 2011;28(3):342-9. DOI:10.1093/fampra/cmq098
https://doi.org/10.1093/fampra/cmq098... ^,88. Dagdeviren N, Akturk Z. An evaluation of patient satisfaction in Turkey with the EUROPEP instrument. Yonsei Med J. 2004;45(1):23-8. DOI:10.3349/ymj.2004.45.1.23
https://doi.org/10.3349/ymj.2004.45.1.23... . However, the remaining dimensions are not found in other studies, which can be attributed to the fact that the Portuguese version include more items than the original one. This increase of items meets the reality of the Portuguese primary health care. As for reliability, this instrument features good internal consistency. And, still, there were no problems about the singularity and multicollinearity.

Regarding loyalty, this dimension presented acceptable internal consistency and the items that constituted it compose a robust measure of the construct. Loyalty was positively associated with overall satisfaction and all its dimensions. And, as verified in other studies⁵5. Ciavolino E, Dahlgaard JJ. ECSI – Customer Satisfaction Modelling and Analysis: a case study. Total Qual Manag Bus Excell. 2007;18(5):545-54. DOI:10.1080/14783360701240337.
https://doi.org/10.1080/1478336070124033... ^,1717. Gronholdt L, Martensen A, Kristensen K. The relationship between customer satisfaction and loyalty: cross-industry differences. Total Qual Manag. 2000;11(4-6):509-14. DOI:10.1080/09544120050007823
https://doi.org/10.1080/0954412005000782... , satisfaction is a predictor of loyalty and explains 16.0% of its variance.

In short, the Portuguese EUROPEP questionnaire is a robust and reliable instrument to measure the satisfaction of users by their own perspective.

This study had as its main limitation the apparent sample homogeneity – all FHU belong to the North of the Country – and future studies could appeal to a more heterogeneous sample, involving FHU from the North to the South of the Country. The results show that this is a robust instrument; but it contains many items, being a long questionnaire to apply. A shortened version would be more practical and economical to use in the FHU.

REFERENCES

Publication Dates

History