Abstract

This study proposes an integrative review of the literature based on articles and publications on the use of the Primary Care Assessment Tool (PCAT) as a tool for evaluating these services, discussing the results found in Brazil and in other countries of the world, from the initial matrix conceived by Professor Barbara Starfield’s team and proposing updates for the Brazilian version. We identified 124 studies, of which 42 were selected after full reading and according to the established inclusion criteria. Of this subtotal, 17 (40.5%) were Brazilian studies. There is a need to update items of each tool’s attribute, in particular “access – first contact”, including new forms of doctor-patient communication such as: email, mobile application messages, use of videoconference software for communication and even use of telemedicine, among others. PCAT’s use, application and calculation of scores is simple, which makes it useful and suitable for use in the local management of services, especially in its short version.

Primary Health Care; Health evaluation; Health care; Health services; Outcome assessment

Introduction

In the late 1990s and early 2000s, tools were developed in several countries to assess the quality of Primary Health Care through the experience of users, professionals and/or managers in the daily services. Among these resources are the Components of Primary Care Instrument (CPCI)¹1. Flocke SA. Measuring attributes of primary care: development of a new instrument. J Fam Pract 1997; 45(1):64-74., Primary Care Assessment Survey (PCAS)²2. Safran DG, Kosinski M, Tarlov AR, Rogers WH, Taira DH, Lieberman N, Ware JE. The primary care assessment survey: tests of data quality and measurement performance. Med Care 1998; 36(5):728-739., EUROPEP questionnaire³3. Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, Hjortdahl P, Olesen F, Reis S, Ribacke M, Szecsenyi J; European Task Force on Patient Evaluations of General Practice Care (EUROPEP). Patient in Europe evaluate general practice care: an international comparison. Br J Gen Pract 2000; 50(460):882-887., Primary Care Assessment Tool (PCAT)⁴4. Shi L, Starfield B, Xu J. Validating the adult primary care assessment tool. J Fam Pract 2001; 50(2):161-164., Interpersonal Processes of Care⁵5. Stewart AL, Nápoles-Springer AM, Gregorich SE, Santoyo-Olsson J. Interpersonal processes of care survey: patient-reported measures for diverse groups. Health Serv Res 2007; 42(3 Pt 1):1235-1256. and Qualicopc⁶6. Schäfer WL, Boerma WG, Kringos DS, De Maeseneer J, Gress S, Heinemann S, Rotar-Pavlic D, Seghieri C, Svab I, Van den Berg MJ, Vainieri M, Westert GP, Willems S, Groenewegen PP. QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care. BMC Fam Pract 2011; 12:115.. PCATool was proposed and validated in the USA by Cassady et al.⁷7. Cassady CE, Starfield B, Hurtado MP, Berk RA, Nanda JP, Friedenberg LA. Measuring consumer experiences with Primary Care. Pediatrics 2000; 105(4 Pt 2):998-1003., led by Professor Barbara Starfield, as a psychometric scale that covers scores for all PHC attributes, as well as two summary measures. Authors compared two national PHC models with samples conducted by phone and e-mail interviews, calculating statistics called “scores” for each of the characteristics that formed a search group of items: (i) extension of affiliation with a service; (ii) first contact access – use; (iii) first contact access – accessibility; (iv) longitudinality; (v) coordination – integration of care; (vi) comprehensiveness – available services, (vii) comprehensiveness – services provided; (viii) family orientation, (ix) community orientation. Initially, the attribute “coordination” contemplated only the perspective of the integration of care, leaving aside the measurement of information systems. This tool has a version for adult and children users, health professionals and managers.

In Brazil, Harzheim et al.⁸8. Harzheim E, Starfield B, Rajmil L, Álvarez-Dardet C, Stein AT. Consistência interna e confiabilidade da versão em português do Instrumento de Avaliação da Atenção Primária (PCATool-Brasil) para serviços de saúde infantil. Cad Saude Publica 2006; 22(8):1649-1659. were the first to adapt the PCATool – children’s version, analyzing their validity and reliability by means of a cross-sectional study in the city of Porto Alegre, performing reverse translation, adaptation, pre-test, construct validity, internal consistency and reliability analysis. Oliveira⁹9. Oliveira MMC. Presença e extensão dos atributos da atenção primária entre os serviços de atenção primária em Porto Alegre: uma análise agregada [dissertação]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2007. and Trindade¹⁰10. Trindade TG. Associação entre a extensão dos atributos de atenção primária e qualidade do manejo da hipertensão arterial em adultos adscritos à rede de atenção primária à saúde de Porto Alegre [dissertação]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2007. used the same databases of Porto Alegre to develop comparative evaluations between different health care models, associating with the quality of management of hypertension in adults. Simultaneously, in Petrópolis, a highland city in the state of Rio de Janeiro, Macinko et al.¹¹11. Macinko J, Almeida C, Sá PK. A rapid assessment methodology for the evaluation of primary care organization and performance in Brazil. Health Policy Plan. Health Policy Plan 2007; 22(3):167-177. compared facilities with family health teams x facilities with traditional care, validating a version for adult users with small differences in the composition of the items and in the response scale. In 2010, some of these versions were endorsed by the Ministry of Health with the publication of a Manual of said tool¹²12. Harzheim E, Gonçalves MR, Oliveira MMC, Trindade TG, Agostinho MR, Hauser L. Manual do instrumento de avaliação da atenção primária à saúde: primary care assessment tool PCATool – Brasil. Brasília: Ministério da Saúde; 2010.. Subsequently, a team of researchers linked to the Epidemiology Graduate Program, Faculty of Medicine, Federal University of Rio Grande do Sul (UFRGS) validated the versions for adult users¹³13. Harzheim E, Oliveira MMC, Agostinho MR, Hauser L, Stein AT, Gonçalves MR, Trindade TG, Berra S, Duncan BB, Starfield B. Validação do instrumento de avaliação da atenção primária à saúde: PCATool-Brasil adultos. Rev Bras Med Fam Comunidade 2013; 8(29):274-284., professionals¹⁴14. Hauser L, Castro RCL, Vigo A, Trindade TG, Gonçalves MR, Stein AT, Dunca BB, Harzheim E. Tradução, adaptação, validade e medidas de fidedignidade do Instrumento de Avaliação da Atenção Primária à Saúde (Pcatool) no Brasil: versão profissionais de saúde. Rev Bras Med Fam Comunidade 2013; 8(29):244-255. and the short versions for adults¹⁵15. Oliveira MMC, Harzheim E, Riboldi J, Duncan BB. PCATool-adulto-BRASIL: uma versão reduzida. Rev Bras Med Fam Comunidade. 2013; 8(29):256-263. and children¹⁶16. Hauser L. Aprimoramento do Instrumento de Avaliação da Atenção Primária à Saúde (PCATool-Brasil) [tese]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2016..

Since then, several Brazilian authors have started to use PCATool as a resource for the evaluation of Primary Health Care from the perspective of users responsible for children, adolescents and also adult users, in municipalities and cities of different population sizes, combining and complementing, sometimes, with clinical outcomes and use of other questionnaires / protocols in the health area and adapting the local culture to its items. At the global level, researchers from several countries worked on the adaptation and validation of PCATool versions appropriate to their social and health contexts, with increasing use of the tool in several parts of the world.

This study aims to present an integrative review of papers, theses and dissertations available on the internet on the use of PCAT as a tool for evaluating primary health care services through users’ perceptions, discussing the results found in Brazil and in the versions adapted and validated in other countries, from the initial matrix conceived by professor Barbara Starfield’s team, as well as to propose updates for the Brazilian version.

Methodology

The integrative literature review synthesizes information over a specific period on a specific topic. Its main advantage over the revision of the traditional bibliography is that there is a strict method in the selection of papers, with well-defined inclusion and exclusion criteria, period, pre-established languages, bibliographic databases of public domain used and, sometimes, book repositories or theses and dissertations consulted. Its elaboration presupposes detailing to the reader all these choice criteria.

For the review, we consulted Pubmed databases of the National Library of Medicine of the United States, Lilacs (Latin American and Caribbean Literature in Health Sciences), SciELO (Scientific Electronic Library Online), as well as the institutional repositories of scientific production The Sergio Arouca National School of Public Health / Oswaldo Cruz Foundation (ENSP/Fiocruz), the Federal University of Rio Grande do Sul (UFRGS) and the University of São Paulo (USP), as well as research reports with random samples of users, available on the internet. We selected studies published between January 1, 2000 and June 1, 2016. For the search, keywords used were ‘PCATool’ and associations between “PCAT” and “Primary Care Assessment”.

The collection identified 124 papers on the subject. However, after reading them, 42 were selected according to the inclusion criteria: articles published in the period; languages: English, Portuguese and Spanish; types of study: cross-sectional study with results from samples of children or adults users. In a second reading, we searched for publications that listed more than one Primary Health Care attribute and PCAT scores, both for the essential and the derivative characteristics, totaling the same 42 studies. At the end of literature review, scores whose articles were not shown on a scale of 0 to 10 were transformed into this metric in order to standardize the analysis of the observed results. The included studies were characterized by author (s), country / region / city, target audience, data collection period and PCAT calculated scores.

Results

Chart 1 shows the distribution of the 42 national and international studies, all with a cross-sectional design and with the respective attribute scores, retrieved in databases or repositories. Of this total, 17 (40.5%) are from Brazil, four each from Canada and China, three from Argentina, two each from the United States and Hong Kong and one each from South Africa, Colombia, South Korea, Spain, Japan, Paraguay, New Zealand, Thailand, Tibet and Uruguay (Figure 1).

The geographic distribution of the works shows concentration of studies in the American and Asian continents. The target audience consisted mostly of adults and/or children living in urban areas of cities of countries studied. Only one was conducted in rural cities. Of the 42 surveys, 35 (83.3%) had a cross-sectional study design. Two were before-and-after type and six more were tool validation in the country or region. Among the works geared to the evaluation of services from population samples, the study developed in the city of Rio de Janeiro using PCAT user version¹⁷17. Harzheim E, Hauser L, Pinto LF. Avaliação do grau de orientação para Atenção Primária em Saúde: a experiência dos usuários das Clínicas da Família e Centros Municipais de Saúde na cidade do Rio de Janeiro. Porto Alegre: UFRGS; 2015. (Relatório Final da Pesquisa PCATool -Rio-2014). was the one with the largest sample recorded in a single city in the world, both for children (n = 3,145) and for adults (n = 3,530).

Discussion

As can be seen in Figure 1, PCAT is a tool for assessing PHC services used in various locations worldwide. In order to meet the objectives of measuring different realms of PHC in services with heterogeneous characteristics, it was adapted and validated in different regions, always achieving acceptable psychometric properties⁴4. Shi L, Starfield B, Xu J. Validating the adult primary care assessment tool. J Fam Pract 2001; 50(2):161-164.^,88. Harzheim E, Starfield B, Rajmil L, Álvarez-Dardet C, Stein AT. Consistência interna e confiabilidade da versão em português do Instrumento de Avaliação da Atenção Primária (PCATool-Brasil) para serviços de saúde infantil. Cad Saude Publica 2006; 22(8):1649-1659.^,1717. Harzheim E, Hauser L, Pinto LF. Avaliação do grau de orientação para Atenção Primária em Saúde: a experiência dos usuários das Clínicas da Família e Centros Municipais de Saúde na cidade do Rio de Janeiro. Porto Alegre: UFRGS; 2015. (Relatório Final da Pesquisa PCATool -Rio-2014).

18. Lee JH, Choi YJ, Sung NJ, Kim SY, Chung SH, Kim J, Jeon TH, Park HK. Development of the Korean primary care assessment tool--measuring user experience: tests of data quality and measurement performance. Int J Qual Health Care 2009; 21(2):103-11

19. Berra S, Audisio Y, Mántaras J, Nicora V, Mamondi V, Starfield B. Adaptación cultural y al sistema de salud argentino del conjunto de instrumentos para la evaluación de la Atención Primaria de la Salud. Rev Argent Salud Pública 2011; 2(8):6-14.

20. Berra S, Hauser L, Audisio Y, Mántaras J, Nicora, V, Oliveira MMC, Starfield B, Harzheim E. Validez y fiabilidad de la versión argentina del PCAT-AE para evaluar la atención primaria de salud. Rev Panam Salud Publica 2013; 33(1):30-39.^-2121. Aoki T, Inoue M, Nakayama T. Development and validation of the Japanese version of Primary Care Assessment Tool. Fam Pract 2016; 33(1):112-117.. This gives PCAT an advantageous international comparability feature.

Within the observed period, Brazil was the country that most published studies evaluating services using PCAT. Studies showed as essential scores of PHC – first contact access, longitudinality, comprehensiveness and coordination of care – low values ranging from 3.86 in Ilhéus, Bahia, to 7.37 in the city of Rio de Janeiro. Regarding the general score, which includes the already described family and community orientation attributes, we observed a similar range of values: 3.66 in Ilhéus and 7.01 in Rio de Janeiro. The result may be related to the choice made by the municipality of Rio de Janeiro of scaling-up expansion of PHC services, which increased population coverage from 3.5% in 2008 to 70% in 2016²²22. Soranz D, Pinto LF, Penna GO. Eixos e a Reforma dos Cuidados em Atenção Primária em Saúde (RCAPS) na cidade do Rio de Janeiro, Brasil. Cien Saude Colet 2016; 21(5):1327-1338.. It also established a family and community medicine residency program, which increased municipal PHC capacity¹⁷17. Harzheim E, Hauser L, Pinto LF. Avaliação do grau de orientação para Atenção Primária em Saúde: a experiência dos usuários das Clínicas da Família e Centros Municipais de Saúde na cidade do Rio de Janeiro. Porto Alegre: UFRGS; 2015. (Relatório Final da Pesquisa PCATool -Rio-2014).^,2323. Justino ALA, Oliver LL, Melo TP. Implantação do Programa de Residência em Medicina de Família e Comunidade da Secretaria Municipal de Saúde do Rio de Janeiro, Brasil. Cien Saude Colet 2016; 21(5):1471-1480..

International studies have shown that, in relation to the essential / general attributes with a history of investment in PHC, the following locations performed well: Montevideo (7.51 / 6.93), Seoul and metropolitan region (7.63 / 7.45), Department of Santander in Colombia (7.84 / 6.99), Shigatse and Linzi in Tibet (7.36 / 7.41) and Columbia in the USA (6.99 / 6.63). The first study to analyze a city in South Africa also showed essential and general scores close to 6,6²⁴24. Bresick G, Sayed A, le Grange C, Bhagwan S, Manga N, Hellenberg D. Western Cape Primary Care Assessment Tool (PCAT) study: Measuring primary care organisation and performance in the Western Cape Province, South Africa (2013). Afr J Prm Health Care Fam Med 2016; 8(1):a1057..

Despite being culturally and organizationally different models, these locations, together with the municipality of Rio de Janeiro, Brazil, evidenced scores that demonstrate that their own health services are PHC-oriented. However, scores enable us to affirm that services listed above are organized from a structured health care network with established flows. The portfolio of services meets the needs of the population and care continuity and facilitated access is in place. As shown in Chart 1, study by Harzheim et al.¹⁷17. Harzheim E, Hauser L, Pinto LF. Avaliação do grau de orientação para Atenção Primária em Saúde: a experiência dos usuários das Clínicas da Família e Centros Municipais de Saúde na cidade do Rio de Janeiro. Porto Alegre: UFRGS; 2015. (Relatório Final da Pesquisa PCATool -Rio-2014). obtained a sample for the city of Rio de Janeiro, of 3,145 children and 3,530 adults, totaling 6,675 individuals interviewed in field work in the first half of 2014, with a sub-municipal representative for the so-called “health planning area”, and also for the two types of facilities that provided primary health care at the time. This was the largest sample ever performed in a single city that we located in our research until 2015.

Nevertheless, the large sample, for example, enabled authors of this study to stratify the results of attribute scores in subsamples, according to some complementary variables searched in the tool, such as: “administrative areas of the city”, “social class” , “team implantation time”, “elderly users - people over 60 years”. Authors found higher scores in health facilities with a longer time of implantation of their family health teams (primary care teams in Brazil) and did not find differences between social classes and the subgroup of adult users older than 60 years.

Some studies in Canada and Spain have used part of the PCAT or short version adapted and validated to their reality⁴³43. Rocha KB, Rodríguez-Sanz M, Pasarín MI, Berra S, Gotsens M, Borrell C. Assessment of primary care in health surveys: a population perspective. Eur J Public Health 2012; 22(1):14-19.^,4949. Tourigny A, Aubin M, Haggerty J, Bonin L, Morin D, Reinharz D, Leduc Y, St-Pierre M, Houle N, Giguère A, Benounissa Z, Carmichael PH. Patients’ perceptions of the quality of care after primary care reform: Family medicine groups in Quebec. Can Fam Physician 2010; 56(7):e273-e282., especially in the realm of “comprehensiveness”, in which the list of items that compose it is very specific in each country. This data collection tool allows the researcher to use part of the tool as specific items to use as a proxy for a particular outcome, or only items that make up the realms for an attribute of interest. In addition, short versions have been developed with the objective of optimizing the collection of information and pointing out ways from evaluations that can be routine³²32. Oliveira VBCA. Avaliação da atenção primária à saúde da criança no município de Colombo – Paraná [dissertação]. Ribeirão Preto: Universidade de São Paulo; 2012.^,4343. Rocha KB, Rodríguez-Sanz M, Pasarín MI, Berra S, Gotsens M, Borrell C. Assessment of primary care in health surveys: a population perspective. Eur J Public Health 2012; 22(1):14-19..

Issues to be incorporated into an upcoming version of the tool

After reviewing the literature and fifteen years after its initial proposal, the application and analysis of the observed results suggest the need to update items of each attribute of the tool. This applies in particular to the attribute “First contact access”, which could be tested to evaluate the possibility of including / adapting new items, such as the inclusion of new forms of doctor-patient communication, such as e-mail messages, mobile device applications messages, use of video broadcasting software for communication and even use of telemedicine, among others. In addition, the use of electronic information systems replacing paper medical records has brought to care not only a technological substitute, but also several possibilities for improving the coordination of care that must be incorporated into new versions of the tools. Likewise, change in the epidemiological context raises the need to include new items in the realm of comprehensiveness.

Final considerations

PCAT enables the evaluation of health services from the user’s perspective, observing the extent of PHC attributes in the evaluated services. This tool has been used around the world from different versions validated for local contexts that allow the comparability of findings. In addition, it is simple to use / apply and calculate scores, even when there is a need to impute data¹⁶16. Hauser L. Aprimoramento do Instrumento de Avaliação da Atenção Primária à Saúde (PCATool-Brasil) [tese]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2016., which makes it useful and suitable for use in the local management of services. Short PCAT versions have been shown to be competent to evaluate particular aspects of PHC attributes and are yet another important tool for local management.

References

Publication Dates

History