Abstract

This study verified the internal consistency and reliability of an instrument to evaluate dental services in Primary Health Care (PHC). In order to verify the factor validity, a factor analysis with principal component extraction and varimax orthogonal rotation method was used. Factors with three or more items with factor loadings greater than 0,35 were selected. This instrument’s reliability was verified using internal consistency (total item correlation >0,30 and Cronbach alpha = or >0,70)). 562 dentists participated in the study. In the factor analysis, ten factors were kept, which explain 40,95% of the total variation. Regarding the internal consistency, only 3 items presented insufficient correlation. Also on internal consistency, using Cronbach’s alpha, the following values of the coefficients were identified: Access (0.55), Continuity (0,74), Care Coordination (0,55), Coordination - Information System (0.21), Comprehensiveness of Services Available (0,91), Comprehensiveness of Services Provided (0,79), Family Orientation (0.66), Community Orientation (0,87), Cultural Competence (0,81). For the success ratio of the scale, all results were higher than 88%, less the “Information Systems” component (21%).

Key words
Primary Health Care; Oral Health; Health Services Evaluation; Monitoring

Resumo

Este estudo verificou a consistência interna e a confiabilidade de um instrumento para avaliar os serviços odontológicos na Atenção Primária à Saúde (APS). Para verificar a validade dos fatores, foi utilizada uma análise fatorial com extração de componentes principais e método de rotação ortogonal varimax. Foram selecionados fatores com três ou mais itens com cargas fatoriais maiores que 0,35. A confiabilidade deste instrumento foi verificada usando consistência interna (correlação total de itens >0,30 e alfa de Cronbach = ou >0,70). 562 dentistas participaram do estudo. Na análise fatorial, foram mantidos dez fatores, o que explica 40,95% da variação total. Quanto à consistência interna, apenas 3 itens apresentaram correlação insuficiente. Ainda na consistência interna, utilizando o alfa de Cronbach, foram identificados os seguintes valores dos coeficientes: Acesso (0,55), Continuidade (0,74), Coordenação de Atenção (0,55), Coordenação - Sistema de Informação (0,21), Abrangência dos Serviços Disponíveis (0,91), Abrangência dos serviços prestados (0,79), Orientação familiar (0,66), Orientação comunitária (0,87), Competência cultural (0,81). Para a taxa de sucesso da escala, todos os resultados foram superiores a 88%, menos o componente “Sistemas de Informação” (21%).

Palavras-chave
Atenção Primária à Saúde; Saúde Bucal; Avaliação de Serviços de Saúde; Monitoramento

Introduction

Primary Health Care (PHC) has established it self as one of the most equitable and efficient ways of organizing a health system¹1 Starfield B, Shi L, Macinko J. Contribution of Primary Care to Health Systems and Health. Milbank Q 2005; 83(3):457-502.. It can be defined by its essential attributes: 1) first-contact accessibility, 2) continuity, 3) comprehensiveness, 4) care coordination; and derived attributes, 5) family and community-centered attention, and 6) cultural competence²2 Starfield B. Atenção Primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasília: Unesco; 2002.

3 Starfield B, Shi L. Policy relevant determinants of health: an international perspective. Health Policy 2002; 60(3):201-218.^-44 Macinko J, Starfield B, Shi L. The contribution of primary care systems to health outcomes within Organization for Economic Cooperation and Development (OECD) countries, 1970-1998. Health Serv Res 2003; 38(3):831-865..

In Brazil, the Family Health Strategy (FHS) is an effort conducted by the Ministry of Health (MOH) to develop a public health policy able to establish a sturdy PHC model. In April 2003, 19,068 FHS teams, or 62,339,523 million of individuals, were covered by the FHS. In April 2020, 44,716 FHS teams or 137,360,577 million of individuals, started to be covered⁵5 Brasil. Ministério da Saúde (MS) [Internet]. Brasília: MS; 2020 [acessado 2020 jun 10]. Disponível em: https://egestorab.saude.gov.br/paginas/acessoPublico/relatorios/relatoriosPublicos.
https://egestorab.saude.gov.br/paginas/a... . Oral health in Brazil started to be supported by the PHC model in 2003, with the launch of the National Oral Health Program, also known as Brasil Sorridente. Initially, the program expanded the coverage of oral health services in PHC, with the objective of expanding access and establishing guidelines to reorient oral health services, taking as a central axis the offer of services and actions associated with the attributes of PHC⁶6 Pimentel FC, Albuquerque PC, Martelli PJL, Souza WV, Acioli RML. Caracterização do processo de trabalho das equipes de saúde bucal em municípios de Pernambuco, Brasil, segundo porte populacional: da articulação comunitária à organização do atendimento clínico. Cad Saude Publica 2012; 28(Sup. l):s146-s157..

Different conceptual models and instruments have been unfurled in recent years to evaluate the quality of health care, some of which include the PHC2 attributes²2 Starfield B. Atenção Primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasília: Unesco; 2002.^,77 Campbell S, Roland M, Buetow S. Defining quality of care. Soc Sci Med 2000; 51:1611-1625.^,88 Haggerty J, Burge F, Levesque JF, Gass D, Pineault R, Beaulieu MD, Santor D. Operational definitions of attributes of primary health care: consensus among Canadian experts. Ann Fam Med 2007; 5(4):336-344.. The Primary Care Assessment (PCATool) was one of those, a set of instruments capable of measuring the presence and extension of PHC attributes, developed by Starfield and collaborators at The Johns Hopkins Populations Care Policy Center for the Underserved Populations, in Baltimore (USA)²2 Starfield B. Atenção Primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasília: Unesco; 2002.^,99 Shi L, Starfield B, Xu J. Validating the adult Primary Care Assessment Tool. J Fam Pract 2001; 50(2):161-175.

10 Cassady CE, Starfield B, Hurtado MP, Berk RA, Nanda JP, Friedenberg LA. Measuring consumer experiences with primary care. Pediatrics 2000; 105(4 Pt 2):998-1003.^-1111 Shi L, Starfield B, Politzer R, Regan J. Primary care, self-rated health, and reductions in social disparities in health. Health Serv Res 2002; 37(3):529-550..

PCATool is an instrument able to assess the quality of primary care provided, based on the measurement of the extent of PHC attributes, producing a score for each attribute, an essential score and an overall score on the quality of care. It allows an opportunity to comparatively assess the degree of success of services when it comes to achieving excellence in PHC practice¹¹11 Shi L, Starfield B, Politzer R, Regan J. Primary care, self-rated health, and reductions in social disparities in health. Health Serv Res 2002; 37(3):529-550.^,1212 Harzheim E, Duncan BB, Stein AT, Cunha CR, Goncalves MR, Trindade TG, MMC Oliveira, MEB Pinto. Quality and effectiveness of different approaches to primary care delivery in Brazil. BMC Health Serv Res 2006; 6:156.. In Brazil, the versions for adult users¹³13 Harzheim E, Oliveira MMC, Agostinho MR, Hauser L, Stein AT, Gonçalves MR, Trindade TG, Berra S, Duncan BB, Starfield B. Validação do instrumento de avaliação da atenção primária à saúde: PCATool-Brasil adultos. Rev Bras Med Fam Comunidade 2014; 8(29):274-284., children¹⁴14 Harzheim E, Starfield B, Rajmil L, Álvarez-Dardet C, Stein AT. Consistência interna e confiabilidade da versão em português do Instrumento de Avaliação da Atenção Primária (PCATool-Brasil) para serviços de saúde infantil. Cad Saude Publica 2006; 22(8):1649-1659., health professionals¹⁵15 Castro RCL, Knauth DR, Harzheim E, Hauser L, Duncan BB. Avaliação da qualidade da atenção primária pelos profissionais de saúde: comparação entre diferentes tipos de serviços. Cad Saude Publica 2012; 28(9):1772-1784.^,1616 Hauser L, Castro RCL, Vigo A, Trindade TG, Gonçalves MR, Stein AT, Duncan BB, Harzheim E. Tradução, adaptação, validade e medidas de fidedignidade do Instrumento de avaliação da Atenção Primária à Saúde (PCATool) no Brasil: versão profissionais de saúde. Rev Bras Med Fam Comunidade 2013; 8(29):244-255. and also oral health version from the perspective of the adult user have already been translated, adapted and validated¹⁷17 Cardozo DD. Validação e aplicação PCATool-SB para avaliação da qualidade da Atenção Primária à Saúde Bucal - versão usuários [tese]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2015.. It is one of the most frequently used PHC assessment instruments in the country¹⁸18 Fracolli LA, Gomes MFP, Nabão FRZ, Santos MS, Cappellini VK, Almeida ACC. Instrumentos de avaliação da Atenção Primária à Saúde: revisão de literatura e metassíntese. Cien Saude Colet 2014; 19(12):4851-4860.. It has recently been cited as one of the instruments that will compose the National Home-based Health Research that will enable the collection of relevant information about the health of the population and the capacity of the Unified Health System (SUS), focusing on Primary Health Care (PHC). It encompasses the National Continuous Household Sample Survey (Continuous PNAD), the National Demographic and Health Survey (PNDS), and the Health-Medical Assistance Survey (AMS)¹⁹19 Agência Brasil. Ministério da Saúde e IBGE coletarão dados sobre população infantil [Internet]. [acessado 2019 dez 13]. Disponível em: http://agenciabrasil.ebc.com.br/saude/noticia/2019-12/ministerio-da-saude-e-ibge-coletarao-dados-sobre-populacao-infantil.
http://agenciabrasil.ebc.com.br/saude/no... . In addition, PCATool Brazil started to compose an extension of payment for performance of the new financing model for APS do Brazil²⁰20 Harzheim E, D'Avila OP, Ribeiro DC, Ramos LG, Silva LE, Santos CMJ, Costa LGM, Cunha CRH, Pedebos LA. Novo financiamento para uma nova Atenção Primária à Saúde no Brasil. Cien Saude Colet 2020; 25(4):1361-1374. and a new PCATool Brazil manual was released by the Ministry of Health, with guidance on its use in estimates²¹21 Brasil. Ministério da Saúde (MS). Secretaria de Atenção Primária à Saúde. Departamento de Saúde da Família. Manual do Instrumento de Avaliação da Atenção Primária à Saúde: PCATool-Brasil. Brasília: MS; 2020..

Within the scope of evaluation of oral health services in PHC from the user’s perspective, in addition to the PCATool Brazil - Oral Health Adult User, there is also the QASSAB instrument (Questionnaire for Evaluation of Quality of Oral Health Services) and the instrument of External Evaluation of the National Program for Improving Access and Quality of Basic Care (PMAQ - AB)²²22 Brasil. Ministério da Saúde (MS). Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ): manual instrutivo. Brasília: MS; 2012.. The evaluation from the user’s perspective is important because they are the target audience of the services and the opportunity to know their satisfaction in terms of structures, processes and results presented by the health service that they use is important information for improving the actions and services provided.

On the other hand, the assessment of health services encompasses both those who use and those who provide and produce the services. Therefore, it is necessary to acknowledge that users and professionals have different and complementary positions within the evaluative context, and both are important for the analysis of the services researched. The PMAQ-AB presented an evaluation instrument from the perspective of the professional, whose main objective is to provide management subsidies. Nevertheless, no instruments were found to assess oral health services in PHC from a professional perspective that had validated its psychometric characteristics.

PCATool is a set of instruments developed from PHC’s sturdy theoretical framework²2 Starfield B. Atenção Primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasília: Unesco; 2002.^,99 Shi L, Starfield B, Xu J. Validating the adult Primary Care Assessment Tool. J Fam Pract 2001; 50(2):161-175.^,1010 Cassady CE, Starfield B, Hurtado MP, Berk RA, Nanda JP, Friedenberg LA. Measuring consumer experiences with primary care. Pediatrics 2000; 105(4 Pt 2):998-1003. and has been used in different countries, not only for academic research purposes but also for decision making of managers²³23 D'Avila OP, Pinto LFS, Hauser L, Gonçalves MR, Harzheim E. O uso do Primary Care Assessment Tool (PCAT): uma revisão integrativa e proposta de atualização. Cien Saude Colet 2017; 22(3):855-865.. Given the breakthroughs registered by PHC and the expansion of oral health in the FHS in Brazil, it is relevant, at this point, to assess the results achieved in relation to the organization and provision of oral health services in the health care network. Therefore, it is essential to provide instruments for assessing oral health services in PHC that allow for international comparisons, capable of reflecting instruments that assess PHC attributes from the medical-nursing work. Therefore, the objective of this study was to adapt the instrument and verify the reliability of the PCATool Brazil Oral Health - Professional.

Methods

Outlining

Cross-sectional validation study of PCATool Brazil Oral Health - Professional.

Sample

This instrument has presented 98 evaluation items after its adaptation. In order to meet the study’s objectives, data were collected using a convenience sample composed of 562 participants, complying with the minimum threshold of 5 interviews for each item of the instrument necessary for the application of factor analysis, a methodology used to assess the validity of the instrument²³23 D'Avila OP, Pinto LFS, Hauser L, Gonçalves MR, Harzheim E. O uso do Primary Care Assessment Tool (PCAT): uma revisão integrativa e proposta de atualização. Cien Saude Colet 2017; 22(3):855-865.. The participants were dental surgeons (DS) of the Primary Health Care (PHC) of municipalities in the State of Rio Grande do Sul (Alvorada, Anta Gorda, Arroio do Meio, Arvorezinha, Bom Retiro do Sul, Canudos do Vale, Capitão, Colinas, Coqueiro Baixo, Cruzeiro do Sul, Doutor Ricardo, Encantado, Estrela, Fazenda Vilanova, Forquetinha, Gravataí, Ilópolis, Imigrante, Lajeado, Marques de Souza, Muçum, Nova Bréscia, Novo Hamburgo, Paverama, Pelotas, Porto Alegre, Poço das Antas, Pouso Novo, Progresso, Putinga, Relvado, Roca Sales, Santa Clara do Sul, Sapucaia do Sul, Sério, Tabaí, Taquari, Teutônia, Travesseiro, Venâncio Aires, Vespasiano Corrêa, Viamão and Westfália) and from the municipality of Rio de Janeiro.

All dental surgeons who were working in the same Basic Health Unit for at least six months before the date of the interview were included in the study.

Adaptation

PCATool Brazil Oral Health - Professional is a “mirror” questionnaire for PCATool Brasil - Professional¹⁶16 Hauser L, Castro RCL, Vigo A, Trindade TG, Gonçalves MR, Stein AT, Duncan BB, Harzheim E. Tradução, adaptação, validade e medidas de fidedignidade do Instrumento de avaliação da Atenção Primária à Saúde (PCATool) no Brasil: versão profissionais de saúde. Rev Bras Med Fam Comunidade 2013; 8(29):244-255. and PCATool Brasil Oral Health - Adult User¹⁷17 Cardozo DD. Validação e aplicação PCATool-SB para avaliação da qualidade da Atenção Primária à Saúde Bucal - versão usuários [tese]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2015.. Therefore, to adapt the professional version to an oral health approach, the instrument was adapted from PCATool Brazil Oral Health - Adult User. The adaptation consisted of replacing the expression “your health service” with “your oral health service”, except for the dimension “Comprehensiveness”. For this dimension in specific, the instrument PCATool Brasil Oral Health - Adult User¹⁷17 Cardozo DD. Validação e aplicação PCATool-SB para avaliação da qualidade da Atenção Primária à Saúde Bucal - versão usuários [tese]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2015. was used as a mirror.

A previous validation study identified that among, the seven attributes that make up the questionnaire, only the comprehensiveness dimension (services available/services provided) did not properly assess this dimension in relation to oral health. To develop the “comprehensiveness” dimension, this study used the Delphi Technique to build a consensus among experts (university professors and dentists with extensive experience in PHC) for the instrument PCATool Brasil Oral Health - Adult User²⁴24 Fontanive LT. Adaptação do instrumento Primary Care Assessment Tool - Brasil versão usuários dirigido à Saúde Bucal [dissertação]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2011.. Therefore, the comprehensiveness dimension, developed and validated for oral health services of PHC in Brazil, was used for the instrument PCATool Brazil Oral Health - Professional.

PCATool Brazil Oral Health - Professional

The PCATool Brazil Oral Health - Professional version instrument encompasses 98 items distributed among the essential attributes – access, continuity, comprehensiveness and coordination – and their derivatives – family and community orientation and cultural competence. It was originally developed to assess the presence and extent of PHC attributes in health services. The item’s responses are presented in Likert-type scales, with values ranging from 1= “certainly not” to 4= “certainly yes”, with the additional option 9= “don’t know/can’t remember”. From the answers, it is possible to calculate a score for each PHC attribute and also an essential score and an overall score. At least 50% of valid answers (4= “certainly yes”, 3= “probably yes”, 2= “probably not” or 1= “certainly not”) is a condition to calculate the score. If the condition is met, the code responses 9= “I don’t know/can’t remember” should be transformed to code 2= “probably not” according to guidelines in the Instrument Manual²¹21 Brasil. Ministério da Saúde (MS). Secretaria de Atenção Primária à Saúde. Departamento de Saúde da Família. Manual do Instrumento de Avaliação da Atenção Primária à Saúde: PCATool-Brasil. Brasília: MS; 2020., as statistically demonstrated by Hauser²⁵25 Hauser L. Aprimoramento do Instrumento de Avaliação da Atenção Primária à Saúde (PCATool-Brasil) [tese]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2016..

The essential score is obtained by averaging the scores of the essential attributes – first contact, continuity, comprehensiveness and coordination. In addition to these, the overall score includes the derived attributes – family orientation, community orientation and cultural competence. The score values are turned into a scale ranging from 0 to 10. A value equal to or greater than 6,6 is considered a high score.

Data collection

The professionals were identified, contacted and invited to participate in the study by telephone. The contacts were passed on by the Municipal Health Departments. Prior consent was requested from the departments in order to carry out the research and provide the contact data of the Basic Health Units (BHU). The professionals who did not agree to participate were regarded as refusals.

The research instrument was used during the oral health coordination meetings of the municipalities. Also, information was collected on sex, postgraduate education and the BHU model in which the DSs worked. A researcher, previously trained, presented the research project and each item of the instrument PCATool Brasil SB - Professional. The doubts were clarified and the instrument was handed to the professionals for them to answer. The interviews were conducted between March 2014 and December 2015 and the researcher stayed in the room to address any doubts that might arise. The average time to complete the research instrument was 25 minutes.

Statistical Analysis

In order to verify the factor validity, a factor analysis with principal component extraction and varimax orthogonal rotation method was used. Factors with three or more items with factor loadings greater than 0,35²⁶26 Stewart DW. The Application and Misapplication of Factor Analysis in Marketing Research. J Marketing Res 1981; 18:51-62. were selected.

This instrument’s reliability was verified using the internal consistency, the success ratio of the scale and the stability over time. Regarding the assessment of internal consistency, the item-total correlation was used, taking into consideration appropriate items with a value higher than 0,30²⁶26 Stewart DW. The Application and Misapplication of Factor Analysis in Marketing Research. J Marketing Res 1981; 18:51-62. and the Cronbach’s alpha coefficient ideally equal or higher than 0,70²⁶26 Stewart DW. The Application and Misapplication of Factor Analysis in Marketing Research. J Marketing Res 1981; 18:51-62.. For the scale’s success ratio, it was verified the quotient between the number of times that correlations between the items of an attribute were higher than the correlations of each of these items with other attributes and the total number of correlations found in that attribute. High values of that measure suggest greater attribute discrimination.

The instrument’s stability over time was assessed by comparing the scores of PHC attributes in ten-day intervals between test and retest. We randomly selected 10% of the sample for a new phone interview.

Wilcoxon’s statistical test was applied to two paired samples. All analyses were conducted using the SAS (Statistical Analysis Software) version 9.4 and the 5% significance level was considered.

Ethical Aspects

The research project “Evaluation of Oral Health Services in Primary Health Care - A conceptual, psychometric, exploratory and structural analysis” was submitted to and approved by the Research Ethics Committees of the Municipal Administration of Porto Alegre and the Federal University of Rio Grande do Sul. The interviews with service professionals were carried out after reading and signing the Informed Consent Form (ICF), in tune with the ethical precepts established in the Declaration of Helsinki.

Results

The sample consisted of 562 dental surgeons interviewed. During the data collection period, there was an attempt to contact 613 DSs. Among them, 24 were not found, 15 refused to participate in the survey and 12 had less than six months of experience in the current BHU. Therefore, the loss of the original sample was 9%.

Of the interviewees, 76,7% were women, 81,2% had some specialty and for, 44,29%, the declared specialty was family health/public health/collective health. The family health strategy was the PHC model declared by 81,5% of professionals.

Factorial Analysis

The exploratory factor analysis kept 10 factors that explain 40,95% of the total variation. These factors represented, in a more homogeneous manner, the attributes that allow measuring the presence and extension of PHC and its essential and derived attributes. Regarding the distribution of the items in the factors, the First-Contact Access - Accessibility was captured by factor 5. Continuity was kept in factor 4 (Table 1). The care coordination was captured by factor 8 (Table 2).

The Comprehensiveness of services available presented items in factor 1, and the same is true for the Comprehensiveness of services provided (Table 3). The attributes derived from family orientation, community orientation and cultural competence were captured by factors 3, 2 and 2, respectively (Table 4). The coordination of Information Systems was the only dimension that did not present a factor load greater than 0,35 in the items assessed (Table 2). This dimension was kept due to its conceptual importance.

The instrument PCATool Brazil Oral Health - Professional adapted for primary health care services in Brazil was initially formed by 98 items. After validation, 85 items were kept in the final instrument.

Reliability

Regarding the internal consistency, taking into account the item-total correlation, for access (Table 1), items A3 (When the patient has a mouth issue, does someone in the service see that person the same day?), A7 (When the patient has a mouth issue at night, does someone at the service see that person the same day?) and A8 (Is it easy to make an appointment at your oral health service?) presented moderate correlation values. Item A9 (On average, do your patients have to wait more than 30 minutes to be seen?) presented insufficient item-total correlation values. For continuity (Table 1), B3 (Do you think your patients understand what you say or ask?) and B4 (Can your patients call and talk to the dentist who knows them best?) presented moderate correlation values. Item B1 (Are patients always seen by the same dentist at your health service?) presented insufficient item-total correlation values. Regarding care coordination (Table 2), items C4 (Does someone in the service help the patient make a referral appointment?) and C5 (When your patients are referred, do you provide written information to inform the specialist or service?) presented moderate correlation values. In Cultural Competence (Table 4), only item I1 (Do you make home visits?) presented a value regarded as insufficient.

Also on internal consistency, using Cronbach’s alpha, the following coefficient values were observed (Table 5): Access (0,55), Continuity (0,74), Care Coordination (0,55), Coordination - Information System (0,21), Comprehensiveness of Services Available (0,91), Comprehensiveness of Services Provided (0.79), Family Orientation (0,66), Community Orientation (0,87), Cultural Competence (0,81).

For the scale’s success ratio (SSR), all results were higher than 88%, except for the “Information Systems” component (SSR=21%), (Table 5).

Discussion

This study adapted and assessed the validity and reliability of the PCATool Brazil Oral Health - Professional version in the public PHC services, indicating that the instrument is valid and reliable for the measurement of the quality of PHC health services based on the experience of the interviewed Dental Surgeons. In the factor analysis, the ten-factor model was capable of capturing the four essential attributes and three attributes derived from the PHC, according to the Starfield proposition¹1 Starfield B, Shi L, Macinko J. Contribution of Primary Care to Health Systems and Health. Milbank Q 2005; 83(3):457-502., therefore identifying the construct’s multidimensional nature. This phenomenon is also found in the adult user, child user and health professional versions of PCATool validated in Brazil¹³13 Harzheim E, Oliveira MMC, Agostinho MR, Hauser L, Stein AT, Gonçalves MR, Trindade TG, Berra S, Duncan BB, Starfield B. Validação do instrumento de avaliação da atenção primária à saúde: PCATool-Brasil adultos. Rev Bras Med Fam Comunidade 2014; 8(29):274-284.^,1414 Harzheim E, Starfield B, Rajmil L, Álvarez-Dardet C, Stein AT. Consistência interna e confiabilidade da versão em português do Instrumento de Avaliação da Atenção Primária (PCATool-Brasil) para serviços de saúde infantil. Cad Saude Publica 2006; 22(8):1649-1659.^,1616 Hauser L, Castro RCL, Vigo A, Trindade TG, Gonçalves MR, Stein AT, Duncan BB, Harzheim E. Tradução, adaptação, validade e medidas de fidedignidade do Instrumento de avaliação da Atenção Primária à Saúde (PCATool) no Brasil: versão profissionais de saúde. Rev Bras Med Fam Comunidade 2013; 8(29):244-255..

The first-contact access attribute was captured by factors 5 and 9. The continuity attribute had its items kept in factor 4. Family orientation was captured by factor 3, which, in turn, kept items of Community Orientation. The interrelation of items of different attributes kept by common factors indicates that the complex dynamic nature of PHC services is represented by the instrument, since, according to the concept of PHC, the attributes can be didactically divided, but are closely related in the practice of services¹⁶16 Hauser L, Castro RCL, Vigo A, Trindade TG, Gonçalves MR, Stein AT, Duncan BB, Harzheim E. Tradução, adaptação, validade e medidas de fidedignidade do Instrumento de avaliação da Atenção Primária à Saúde (PCATool) no Brasil: versão profissionais de saúde. Rev Bras Med Fam Comunidade 2013; 8(29):244-255..

The instrument PCATool Brazil Oral Health - Professional comprised 98 items. After an exploratory analysis, 13 of these items were excluded for not presenting sufficient factor loading or because they assessed the same domain. The items suggested for evaluating the information system did not have sufficient factor loadings. We may speculate that this is a limitation related to the precariousness of information systems in Brazil, and also due to the fact that the national e-SUS Basic Attention system was only implemented in 2016. Nevertheless, due to its relevance in terms of planning, organization and evaluation of services²⁷27 Carreno I, Moreschi C, Marina B, Hendges DJB, Rem-pel C, Oliveira MMC. Análise da utilização das informações do Sistema de Informação de Atenção Básica (SIAB): uma revisão integrativa. Cien Saude Colet 2015; 20(3):947-956., and to establish a comparison with other PCATool versions validated in Brazil, the authors decided to maintain it in the final version of the instrument.

The excluded access items are related to the provision of OH services in PHC on weekends or after 8 p.m. This practice is not common in the country and no reports were found in other studies regarding the availability of dental care in PHC after 8 p.m. The removed Care Coordination items are related to the delivery of exams in PHC and the provision of different specialized consultation services. Nonetheless, the Oral Health Team does not have the habit to request laboratory tests for PHC patients and more complex situations are referred to reference services, which are still insufficient in the country, not allowing the choice between different services²⁸28 Pimentel FC, Martelli PJL, Araújo Junior JLA, Acioli RML, Macedo CLV. Análise da atenção à saúde bucal na Estratégia de Saúde da Família do Distrito Sanitário VI, Recife (PE). Cien Saude Colet 2010; 15(4):2189-2196..

For the comprehensiveness attribute, the items related to the use of fluoride “Application of fluoride on teeth and guidance for its use” and fluorosis-related orientation “Guidance on excess fluoride on teeth/fluorosis” were also excluded from the final instrument. It may be pondered that this practice is not part of the routine of the professional who, given the high demand repressed, has its focus on recovery actions such as fillings and exodontia or pain relief measures. In addition, this study was conducted in urban areas with access to running water, which restricts the recommendations for fluoride use. Another important aspect is that access to fluoridated toothpaste is widespread in the country. On the other hand, identifying and monitoring the occurrence, distribution and aggravation of fluorosis is an epidemiological surveillance action, which is part of the tasks of PHC oral health teams and, still, they were not perceived by dentists as available services.

Regarding family orientation, the excluded items are related to the use of genograms and risk factor analysis. The use of genograms is an uncommon practice in PHC by oral health teams²⁹29 Reis CMR. Describing the primary care Actions of oral health teams in Brazil. Int J Environ Res Public Health 2015; 12(1):667-678.. The items regarding the discussion of family risk factors, social risk factors and living conditions were represented by three different questions. According to the authors, all of them explain the same evaluative dimension and, therefore, the option made was to maintain the highest loading in terms of discussions on social risk factors.

The results of the reliability analysis point out the consistency of all attributes, except for Care Coordination - Information Systems, which was assessed as insufficient. Nevertheless, the option was to maintain the items because they represent Starfield’s concept of PHC¹1 Starfield B, Shi L, Macinko J. Contribution of Primary Care to Health Systems and Health. Milbank Q 2005; 83(3):457-502. and to maintain comparability with other PCATool versions validated in Brazil. The results of the scale success ratio show that each item is allocated in its conceptual attribute, because they are more strongly related to the items of its source component than to those of the other attributes, therefore consubstantiating the reliability of the instrument.

The use of service valuation instruments is an important resource, which is nowadays used in many countries. To avoid measurement bias in these instruments, it is extremely important that the process of factor validation is performed for each one. Stein³⁰30 Stein AT. A avaliação dos serviços de saúde deve ser realizada com instrumentos validados. Epidemiol Serv Saude 2013; 22(1):179-181. points out that it is also necessary to take into account the concepts established in the literature, as well as cultural factors, particularly in epidemiological and clinical studies, and methodological rigor in the execution of the process.

The comparison between the test and the retest identified the stability of the scale over time. The comprehensiveness attribute – Services Available – showed low agreement on retesting and it is possible the rotativity of professionals in services may have influenced these results. Another possible limitation of the instrument is its large number of items (85), which may result in the interviewee not having the same willingness to answer all items. The possibility of having a reduced version of this instrument is suggested, which was already developed for another version of PCATool Brazil, one that presented reliable psychometric properties for the instrument³¹31 Oliveira MMC, Harzheim E, Riboldi J, Duncan BB. PCATool-ADULTO-BRASIL: uma versão reduzida. Rev Bras Med Fam Comunidade 2013; 8(29):256-263..

PCATool Brazil Oral Health - Professional captured the main attributes of PHC and generally presented acceptable measures of validity and reliability. The use of scale may account for a valuable instrument for evaluative research efforts and for the organization of oral health services in PHC in the different management areas, since it allows the analysis of the effect of interventions, policies and programs. According to a survey carried out by the authors, this is the only PCATool Brazil Oral Health - Professional described in the international literature with the derived attribute Cultural Competence validated. Moreover, the instrument has essential features for comparability, since it was designed as a mirror version of PCATool Brazil Oral Health - Professional for doctors/nurses – and is also used in other countries³²32 Berterretche R, Sollazzo A. El abordaje de la Atención Primária de Salud, modelos organizativos y prácticas: caso de un centro de salud público urbano de Montevideo, Uruguay 2011. Saude Debate 2012; 36(94):461-472.

33 Berra S, Hauser L, Audisio Y, Mántaras J, Nicora V, Oliveira MMC, Starfield B, Harzheim E. Validez y fiabilidad de la versión argentina del PCAT-AE para evaluar la atención primaria de salud. Rev Panam Salud Publica 2013; 33(1):30-39.

34 Lee JH. Development of the Korean primary care assessment tool - measuring user experience: tests of data quality and measurement performance. Int J Qual Health Care 2009; 21(2):103-111.^-3535 Yang H, Shi L, Lebrun LA, Zhou X, Liu J, Wang H. Development of the Chinese primary care assessment tool: data quality and measurement properties. Int J Qual Health Care 2013; 25(1):92-105..

References

Publication Dates

History