ARTIGO ARTICLE
User satisfaction with pharmacy services in the Brazilian National STD/AIDS Program: validity and reliability issues
Satisfação de usuários do Programa Nacional de DST/AIDS com a dispensação de medicamentos: aspectos de validade e confiabilidade
Thiago Botelho Azeredo; Maria Auxiliadora Oliveira; Vera Lucia Luiza; Ângela Esher; Mônica Rodrigues Campos
Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz, Rio de Janeiro, Brasil
ABSTRACT
The objective of this study is to evaluate the psychometric properties of a user satisfaction scale regarding the Brazilian National STD/AIDS Program, specifically related to dispensing AIDS medicines. The scale was developed and applied in a study covering 10 Brazilian States that evaluated the quality of medicine dispensing. The questionnaire was answered by 1,412 people living with HIV and undergoing antiretroviral therapy. Construct validation involved two stages of factor analysis. The item-total correlation matrix was analyzed, and tests for associations between the target variable, socio-demographic variables, and related constructs were performed. Reliability was studied by means of the sub-scales' internal consistency, estimated by Cronbach's alpha. Five relevant satisfaction dimensions were identified. A moderate level of internal consistency was found for these dimensions, suggesting they were adequate. The results of the association tests agreed with other studies reported in the literature. We conclude that the instrument is appropriate for application in similar populations with adequate psychometric characteristics and serves to measure users' assessments of the pharmaceutical services received and helps to orient improvements in such services.
Consumer Satisfaction; Patient Satisfaction; Questionnaires; Psychometrics
RESUMO
O objetivo do presente artigo é avaliar propriedades psicométricas de uma escala de satisfação de usuários do Programa Nacional de DST/AIDS com a dispensação de medicamentos. A escala foi desenvolvida e aplicada num estudo abrangendo dez estados brasileiros para avaliar a qualidade da dispensação de medicamentos. O questionário foi respondido por 1.412 pessoas vivendo com HIV em tratamento com anti-retrovirais. Para validação de constructo, foram realizadas duas etapas de análise fatorial. A matriz de correlação item-escala corrigida foi analisada; testes de associação entre a variável de interesse, variáveis sócio-demográficas e constructos relacionados foram realizados. A confiabilidade foi estudada por meio da consistência interna das subescalas, estimada pelo alpha de Cronbach. Cinco dimensões da satisfação foram identificadas. Encontrou-se consistência interna moderada para essas dimensões, sugerindo adequações. Houve concordância entre os resultados dos testes de associação realizados e os achados de outros estudos descritos na literatura. O instrumento mostrou-se apropriado para ser aplicado em populações semelhantes, servindo para aferir a avaliação dos usuários sobre os serviços recebidos e assim orientar melhorias nos serviços avaliados.
Satisfação do Usuário; Satisfação do Paciente; Questionários; Psicometria
Introduction
Current approaches for evaluating the quality of health services include outcome measurements, which can be performed by measuring the opinions of users concerning some aspects of care and their degree of satisfaction 1.
User satisfaction can be analyzed either as a dependent variable, assumed as an outcome of a process of care backed by a given structure, or as an independent variable, and can be useful for predicting user behaviors 1, like utilization of care, continuity, and adherence 2,3,4.
Esperidião & Trad 5 systematized four principal theoretical approaches for understanding user satisfaction, based on: attitude, which views satisfaction as the individual's evaluation of certain aspects of the service received; discrepancy, which predicts the level of satisfaction according to the difference between expectation and perception of the experience; fulfillment, or the difference between what is desired and what is obtained; and equity, in which individuals base their evaluation of services in terms of individual inputs and outputs and in a comparison of what is obtained by others, which introduces elements of social comparison.
According to Sitzia & Wood 6, many models give value to patients' characteristics, including their expectations, age, schooling, gender, and ethnic group, and these influence the answers in satisfaction surveys, demonstrating that such features should be described and controlled in the analysis of outcomes in order to support the validity of any conclusions. Other models are based on the belief that certain aspects of care are discernible by users and that they affect the satisfaction with care received with a certain degree of independence 6. Most classifications of these aspects are based on the eight-dimension scheme (interpersonal manner, technical quality, accessibility/convenience, finances, efficacy/outcomes, continuity, physical environment, availability) proposed by Ware et al. 7.
Despite the existence of such theories, the lack of a robust theoretical construction underlying the concept of satisfaction and guiding its measurement is the main limitation of studies in this field 5,6,8. Most of them are undertaken within the context of health service quality evaluation, in which satisfaction is a concept of increasing importance 6.
In the field of satisfaction evaluation, few studies have shown evidence of the validity and reliability of the instruments used 9. These concepts are involved in determining the extent to which an instrument actually measures what it is supposed to 9,10. Thus, they are related to the instrument's quality 11. According to Sitzia 9, some authors appear to be unaware of the importance of such properties in an evaluation, which can cast doubt on the credibility of their findings.
In 2005, a study entitled the National Evaluation of Medicine Dispensing for People Living with HIV/AIDS (Andime-PVH) was conducted in Brazil 12. One of its overall objectives was to evaluate the quality of dispensing services for antiretrovirals (ARVs) and other medicines used to treat opportunistic infections, focusing on the structure, processes, user satisfaction (in relation to outcomes) and adherence to the prescribed treatment. The component on evaluation of satisfaction with dispensing required specific theoretical and operational development, due to the lack of a validated instrument and methodology in Brazil.
This study evaluates the reliability (internal consistency) and the construct validity of a patient satisfaction instrument developed in the Andime-PVH study. Its development involved a review of the literature, discussions with experts and persons living with HIV, and identification, adaptation, testing and selection of the most suitable dimensions to describe and measure user satisfaction within the context of pharmaceutical services 12.
The model that guided the instrument construction was based on the attitude theory and defines user satisfaction as a positive multiple context-dependent reaction to services provided 5,7,10,12,13. Attitudes are subjective experiences that include an evaluative dimension towards an object, fact or person 14. They can be understood as an organization of beliefs and cognitions, generally with some affection involved, for or against a social object 15, or, more simply, as the evaluative judgments that integrate and summarize these cognitive/affective reactions 16,17. The attitude theory was considered to be the most suitable for the study, because it focuses on relevant aspects of the evaluated object, allowing the evaluation of specific health service features 10. What is more, according to Ware et al. 7, although satisfaction with health services is influenced by a mixture of characteristics that are particular to the user of the service and others, it is the quality of service itself that is mainly responsible for determining satisfaction 7.
Method
Study population and data source
The data source for this article came from the Andime-PVH study, which was a cross-sectional and descriptive study. The satisfaction measurement instrument consisted of a structured questionnaire applied by means of face-to-face interviews with 1,412 people living with HIV under ARV therapy, receiving care at 29 dispensing units distributed across ten Brazilian states. Data collection was carried out in August, 2005 12.
Satisfaction scale
The questionnaire contains 37 questions/items, four pertaining to general satisfaction with the services provided (general satisfaction scale) and 33 covering six satisfaction dimensions: (a) convenience - refers to the distance/ease of geographic access, service organization, time spent by the user, choice for care provider (eight items); (b) presence of medicines - medicines' availability in the pharmacy in the amount prescribed (two items); (c) technical quality of dispensing - providers' competence and compliance to good dispensing practices (three items); (d) technical quality of medicines - aspects of packaging and labeling, side-effects, symptoms improvement (seven items); (e) ambience - aspects relating to basic amenities such as cleanliness, signage, seats, drinking water (five items); and (f) interpersonal aspects - dispensers' manners towards users' rights, like dignity, autonomy and confidentiality (eight items). Each item consists of a statement expressing a positive or negative opinion or a judgment on the above-mentioned satisfaction dimensions. The choices for answers were: 1 (completely disagree); 2 (partially disagree); 3 (partially agree); and 4 (completely agree). Answers to items expressing unfavorable opinions were recoded - 4 (completely disagree); 3 (partially disagree); 2 (partially agree); and 1 (completely agree) - in order to give a single meaning to the answers, with higher values corresponding to favorable opinions.
The equation for adjusting the score in each scale was:
Possible score range = highest possible score - lowest possible score
Thus, a score of zero corresponds to the worst possible opinion (or evaluation) for that scale, while 10 is the best possible opinion.
An average satisfaction score was calculated using the mean dimension scores, not including the general satisfaction scale. We considered the average satisfaction score an indirect measure of user satisfaction, while the general satisfaction scale represents a direct measure.
Data analysis
The data analysis procedures described below were based on the steps proposed by Griep et al. 18 for construct validation and estimation of internal consistency in a social support scale.
Construct validity
The main objectives of validating an instrument are to determine whether its scope corresponds with the scope of the variable of interest, comprehending the most relevant aspects of the concept studied, and to demonstrate that the scores obtained assume values that are consistent with the understanding of how the variable of interest varies in the real world 19,20.
The focus of the current article is to analyze aspects of construct validity, which refers to the extent to which the results obtained by a measurement are in agreement with the underlying theoretical construction 9. According to Pasquali 21, construct validity is the most important validity property in psychometric instruments as it aims to assess if a test is a legitimate, adequate representation of the construct of interest.
a) Factor analysis
The study began by testing the adequacy of the variables (questionnaire items) for the factor analysis, using Bartlett's sphericity test and Kaiser Meyer-Olkin's adequacy test 22,23.
Factor analysis was performed in two stages, first using the principal components method and secondly making a comparison of solutions obtained from two different methods: principal axis factoring and maximum likelihood method 23. The first stage explored relations between proposed variables and served as an initial refinement of the variables matrix, identifying which items could be excluded, in order to simplify the matrix interpretation. The criteria for remaining in the matrix were: to present a module factor load > 0.40 in any factor after rotation (promax with Kaiser Meyer Olkin normalization) and to suggest a reasonable theoretical meaning for the factor. In the second stage, after the exclusion of items, a new factor analysis was performed, thus serving to guide the theoretical meaning of each factor and serving to confirm which items should be aggregated to construct scales.
The decision of how many factors should be retained was made considering the amount of total sample variance explained, the relative sizes of the eigenvalues, the observation of scree plots and, most importantly, the semantic meaning suggested by the items with high factor loads in each factor 23.
b) Item-total correlation
Based on the dimensional structure suggested by factor analysis, the correlation (Pearson's r) between each item and each one of the dimensions was estimated. The scores for each dimension were calculated according to previously described procedures. Additionally, to avoid an artificial increase in the coefficient's value, each item analyzed was excluded from the calculation of its dimension's score (corrected item-total correlation) 20. To assess the homogeneity of the scales, the mean of the item-total correlation coefficients for each dimension was calculated. The degree of overlapping among dimensions was estimated by the inter-scale correlation.
c) Test of hypotheses
A review of the literature indicates that variables like gender, age, health status, and schooling are generally associated with the satisfaction reported by health service users - men, the elderly, individuals with less schooling, and those with better health status are generally more satisfied than others 6,24,25,26.
In addition to these relations, we tested the association between satisfaction scores and economic class, the desire to receive Aids medicines at a different pharmacy, the general satisfaction scale, and another measurement of satisfaction in the questionnaire. Economic class, used as a proxy for socioeconomic status, was stratified into categories from A (those with the highest mean income) to E (with the lowest). The general satisfaction scale and another general satisfaction question (a direct question with four response categories: not at all satisfied; hardly satisfied; satisfied; very satisfied) were used as comparison criteria.
These associations were estimated by chi-square tests (Pearson's χ2 and linear by linear) and odds ratios (OR) with 95% confidence intervals (95%CI), using high satisfaction (above the median) as the outcome variable for each dimension.
Reliability: internal consistency
An instrument's reliability is related to its capacity to capture real variations in the characteristics measured in the population and different from systematic or random error, and represents the amount of error inherent to any measurement process 20. Two features account for an instrument's reliability: stability - linked to the reproducibility of the results obtained - and internal consistency - based on a mean correlation between the instrument's items 9,11,19.
In this study, the instrument's stability will not be measured, since the instrument was not submitted to test-retest. The internal consistency of scales pertaining to the dimensions suggested by the prior analyses was estimated by means of the statistic α (Cronbach's alpha) 19,20,21.
Ethical aspects
This study was approved by the Institutional Review Board/Research Ethics Committee. The database did not contain any information identifying the subjects, who participated voluntarily in the Andime-PVH study.
Results
Study population profile
The study population consisted of a majority of male (62%) and/or single (54.8%) respondents, who self-identified as white (44.9%) or mixed-race (37.9%), worked outside of the home (38.9%) or were retired or on pensions (28.2%), and belonged to the lowest mean income classes (71.3%). As for education, 25.8% had between four and seven years of schooling and 29.3% from eight to ten years. Mean age, time since first HIV+ test, and treatment time were 39.4, 5.8, and 4.7 years, respectively. Mean number of household members was 3.6. The majority of the interviewees assessed their own health status as good (53.7%) or very good (41.4%), and only 4.9% as bad or very bad.
Construct validity
Factor analysis
The hypothesis that variables (questionnaire items) were orthogonal, that is, correlated only to themselves, was rejected by the Bartlett sphericity test (p = 0.00), and the Kaiser Meyer-Olkin adequacy test was 0.855, indicating that the data matrix was adequate for the factor analysis.
Following the procedures in the principal component analysis, nine factors were obtained. These factors jointly explain 49.8% of the data variance (Table 1).
The results presented here concern factors with eigenvalues greater than 1. This criterion is the most popular and commonly used one, yet it is not without controversy 27. However, the solution obtained with the adoption of this criterion agreed with the other criteria used in this study.
The majority of items with a higher factor load in the first factor belonged to the interpersonal aspects dimension. In the second factor, two out of four items with higher factor loads belonged to the ambience dimension. In the third factor, we only found items pertaining to the dimension concerned with the technical quality of medicines. The fourth factor included items from different dimensions, but they all appear to point to undesirable aspects of the services, like "the place where I get my medicines could be cleaner", "I don't like the quality of the medicines", and the impossibility of choosing a different pharmacy (indicated by the high negative factor load, -0.61, for the item "if I want, I cant choose the pharmacy where I get my medicines"). In the fifth factor, there were high loads for the items belonging to the dimension presence of the medicines. The sixth included two items from the convenience dimension and one item from the ambience dimension pertaining to the signage for users to locate the pharmacy inside the health clinic. In the seventh factor, two items from the dimension about the technical quality of medicines referring to beliefs and perceptions about the medicines and the resolution of symptoms had high factor loads. In the eighth factor only one item, "At the pharmacy, anybody can find out about my treatment", from the interpersonal aspects dimension, which sought to grasp the service's confidentiality, showed a high factor load. In the last factor that was kept in the model, two items showed a high factor load, one related to convenience and the other to ambience (Table 1).
In the factor analysis, based on the simplified items matrix, the solutions obtained from the two methods employed (principal axis factoring and maximum likelihood) were consistent. In both, six factors were kept which jointly explained 49.9% of the data variance (Table 2). The goodness-of-fit test exhibited a significant result (p = 0.000). The principal axis factoring analysis resulted in a solution that was slightly easier to interpret, which is presented in Table 2.
The first factor that was maintained included items concerning the quality of the interaction between dispensing staff from the pharmacy and users, like dispenser friendliness, respect for users, and users' opinions concerning the treatment and care and the time involved in receiving their prescriptions. Users' notions of comfort (having chairs available while waiting and availability of drinking water at the dispensing facility) and confidentiality (other individuals not seeing their medicines) were present in the items with high factor loads in the second factor, suggesting that the answers to these items referred to amenities provided by the pharmacy unit. All the items in factor three referred to characteristics of the medicines, like their size, ease (or difficulty) in swallowing, adverse-effects (feeling sick when taking the medicines), and perceived relief of symptoms due to the medicines. In the fourth factor items that evoked negative experiences in relation to the services were aggregated. The fifth factor combined the same items as in the previous analysis, related to availability of medicines. The notions of ease and time spent to reach the pharmacy are present in the items included in the last factor, whose meaning thus refers to geographic accessibility. Table 2 shows the names chosen to describe the dimensions representing the items with high loads in the factors derived from the second analysis.
Correlation between items and satisfaction scales
Table 3 shows the correlation coefficients between the items and the satisfaction scales.
The items "dispenser properly explains the reasons for taking the medicines" and "I feel comfortable saying what I think is important" showed correlations with the dimension dispenser-user relationship, similar to those shown with the dimension of which they were part of, infrastructure aspects.
Items in the fourth dimension (dissatisfaction issues) showed higher correlations with various other dimensions than those shown in relation to their own dimension.
The means of the corrected item-total correlation coefficients for the dimensions dispenser-user relationship, infrastructure aspects, characteristics of the medicines, dissatisfaction issues, availability of medicines, and geographic accessibility were 0.45, 0.44, 0.40, 0.12, 0.54, and 0.33, respectively.
The correlation coefficient (r) between the dimensions proposed here varied from 0.12, between the dimensions dissatisfaction issues and infrastructure aspects, to 0.58, between dispenser-user relationship and infrastructure aspects, with a mean of 0.29. Regarding dimensions proposed in the Andime-PVH study, this coefficient varied from 0.18, between the dimensions for technical quality of the medicines and convenience, to 0.55, between technical quality of dispensing and interpersonal aspects, with a mean of 0.31. The correlation between overall scores for the two studies was 0.91. All the correlations were statistically significant (p < 0.01).
Tests of hypotheses
All the variables tested showed a significant association with at least two of the satisfaction dimensions and with the average satisfaction score (Table 4).
There was an association between interviewees' gender and the dimensions for characteristics of medicines, availability of medicines, geographic accessibility and the average satisfaction score, with men showing 2.0 (95%CI: 1.6-2.5), 1.2 (95%CI: 1.0-1.5), 2.0 (95%CI: 1.6-2.5), and 1.7 (95%CI: 1.4-2.1) times greater odds of showing high satisfaction with these scales, respectively.
There was an association between the scales of the dimensions for characteristics of the medicines, dissatisfaction issues, availability of the medicines, geographic accessibility, and the average satisfaction score and the following variables: age, health status, and economic class.
Individuals older than the median (39 years) had higher odds, 1.4 (95%CI: 1.1-1.7) to 1.8 (95%CI: 1.5-2.2), of showing high satisfaction. Among those reporting health statuses as good or very good at the time of the interview, the odds of high satisfaction were significantly greater (Table 4). The odds of high satisfaction with medicines were three times greater among those with better health status (95%CI: 1.8-5.0).
The proportion of satisfied users tended to decrease in lower income groups when compared to those with higher mean income. The largest difference was related to geographic accessibility: while 67.4% of economic class A users were satisfied, only 20.8% of economic class E users showed high satisfaction with this dimension. Class A individuals showed about 8 times greater odds of high satisfaction as compared to class E individuals (and the confidence interval showed that the odds could reach about 18!).
The associations between schooling and satisfaction dimensions showed varying trends. Regarding infrastructure aspects, individuals with more schooling were less likely to express high satisfaction. Meanwhile, concerning dissatisfaction issues, geographic accessibility and the average satisfaction score, individuals with more schooling showed a higher proportion of satisfied users. The difference between the proportions of satisfied users in relation to geographic accessibility is worthy of note: 33% of those with less than four years of schooling as compared to 66.2% of those with more than 15 years of schooling, with the latter showing four times the odds of expressing high satisfaction (95%CI: 2.6-6.2).
Comparing the questionnaire's approaches to measure satisfaction, satisfied or very satisfied individuals (according to a direct question) or with high satisfaction (according to the general satisfaction scale) showed 1.7 (95%CI: 1.4-2.2) to 6.5 (95%CI: 5.1-8.3) greater odds of high satisfaction for all dimensions. The odds increased to 18.4 (95%CI: 9.3-36.4) for high satisfaction in the average satisfaction score for those who responded that they were satisfied or very satisfied with the pharmacy services received.
Reliability: internal consistency
The mean scores were 8.8 (standard deviation - SD = 1.5) for the dimension dispenser-user relationship, 6.4 (SD = 2.7) for infrastructure aspects, 7.2 (SD = 2.5) for characteristics of medicines, 5.7 (SD = 1.9) for dissatisfaction issues, 7.7 (SD = 3.0) for availability of medicines, 6.1 (SD = 3.1) for geographic accessibility, and 7.0 (SD = 1.5) for the average satisfaction score. The scales' internal consistency, estimated by Cronbach's alpha, varied from 0.22 to 0.70 (Table 3). In only one dimension, dissatisfaction issues, the alpha coefficient would increase greatly by excluding one of the items - ability to choose the pharmacy -, while in all other dimensions the exclusion of any item would decrease the alpha (Table 3).
Discussion
Although the results of the factor analysis point to a reorganization of the items in relation to the dimensions predicted in the tested model, four out of the six new dimensions suggested by this analysis presented meanings similar to those previously proposed in the Andime-PVH study 12. Interpersonal aspects, ambience, technical quality of the medicines, and presence of the medicines in the Andime-PVH study have quite similar if not equal meanings to the dimensions dispenser-user relationship, infrastructure aspects, characteristics of the medicines, and availability of the medicines as proposed here.
A fifth dimension proposed here, geographic accessibility, is related to the meaning of the convenience dimension. However, this dimension was intended to capture a broader and more heterogeneous content than that indicated by the factor analysis. The items pertaining to the other contents expected in the convenience dimension, like waiting time and pharmacy service hours, were not aggregated in a single factor and none of them showed a high factor load, and they were even excluded from the matrix. This does not mean that these contents are irrelevant, but that it is necessary to review them and perhaps accompany them with other items with similar theoretical meanings that express, in the items matrix, a different aspect of users' evaluation of the pharmaceutical services.
Items pertaining to user privacy in the services, explanation of treatment by the dispenser and the user feeling comfortable to say what is important appear in the dimension labeled infrastructure aspects. One might question the pertinence of these items in the dimension. However, according to Luiza et al. 12, in the health clinics where the survey was held, the limited privacy afforded by the environment where the medicines were dispensed may have affected the ability of the pharmacy staff to provide proper counsel to users. In pharmacies where the environment afforded privacy in user care, more time was available for dispensing and a higher percentage of users received treatment counseling 12.
These findings explain why interviewees' opinions of the pharmacy's environment, privacy, and counsel were correlated, and supported the decision to keep items that appeared to refer to different domains in the same dimension in order to understand better the target population's satisfaction. However, such correlations may not be valid in other contexts, an issue that should be taken into consideration when using the model in other surveys.
In the dimension dissatisfaction issues, items whose contents refer to the other dimensions were aggregated by the factor analysis, probably because they displayed lower scores. This dimension does not refer to a specific aspect of the services provided, but rather an aggregation of the items that are more sensitive for capturing negative experiences in the other dimensions. Thus, one should consider whether to keep these items in the questionnaire and in which dimensions they should be included, because the inclusion of an item with a lower correlation coefficient in any of the dimensions can cause a reduction in the internal consistency. However, if the hypothesis that these items are sensitive to weak points in the services is true, their inclusion increases the content validity of the questionnaire.
The estimated scores for the proposed dimensions in the previous study varied from 6.7 (convenience) to 7.7 (presence of medicines and technical quality of dispensing), with an average satisfaction score of 7.3 12. The scores of the new dimensions proposed here varied from 5.7 to 8.8, with 7.0 for the new overall scale. Similarity between the overall scores and particularly the correlation between them showed that there were no significant losses in the variance captured by the set of items after excluding some of them. The greater difference and lower correlation between the scores of the new dimensions indicate that they overlap less. This means that the organization of items proposed here is better at discriminating between the various facets (dimensions) of the concept measured.
The two main questionnaires used to construct the instrument evaluated here were the Patient Satisfaction Questionnaire (PSQ) developed by Ware et al. 7, more specifically the PSQ-III 28,29, and the PSQ adapted for the evaluation of pharmacy services by MacKeigan & Larson 3.
The PSQ-III covered seven dimensions: (a) interpersonal aspects, (b) communication, (c) time spent with doctor, (d) access/availability/convenience, (e) financial aspects, (f) technical quality and (g) general satisfaction 29 and the instrument adapted by MacKeigan & Larson 3 included eight: (i) explanation, (ii) consideration, (iii) accessibility, (iv) financial aspects, (v) technical competence, (vi) drug efficacy, (vii) quality of the drug product, and (viii) product availability. Dimensions that were considered as distinct in these studies were used to compose a single dimension in the instrument, as in the case of dimensions (a), (b), (c), (i), and (ii), whose contents were aggregated in the dispenser-user relationship dimension. In the instrument, the dimension geographic accessibility reflects what was recognized in our population in relation to dimensions (d) and (iii). The dimension financial aspects [(e) and (iv)] was not part of the instrument, since users of antiretroviral medicines in Brazil receive their medicines free of cost. A dimension equivalent to dimensions (f) and (v) was planned, but was not maintained by the factor analysis. Dimensions (vi) and (vii) comprised a single dimension in the study. Dimensions (g) and (viii) have equivalent versions in our study.
The PSQ and MacKeigan & Larson's instruments used both a five points Likert scale as response options 3,28,29. The response options adopted by Luiza et al. 12, although keeping the bipolarity from 'completely disagree' to 'completely agree', did not include a neutral point, which is the main constraint of four points scales 30. The absence of the neutral point may have reduced the scales' reliability 20, so one should consider the use of a five point scale if this model is to be used in other studies.
The tests of hypotheses showed that the measurement of user satisfaction proposed here behaves similarly to satisfaction measurement performed in other studies. Consistent with the literature 6,24,26, older individuals were more satisfied with the services received. Individuals with better self-reported health status showed higher satisfaction, in agreement with the hypothesis proposed by Weiss 25. Furthermore, in agreement with Sitzia & Wood 6 and Atkinson & Haran 24, our data showed that lower-income individuals were less satisfied with the services.
The comparison of different measurements of the target concept showed consistency in the classifications of individuals as satisfied versus dissatisfied, based on these measurements. This reinforces the hypothesis that the same concept (user satisfaction) is being measured by all of them.
One could claim that the scales' alpha coefficients are not high. However, when one compares the new scales to those proposed in the previous study, there is an increase in the alpha for all the scales with the new proposed arrangement of items, except for the scale dissatisfaction issues, which does not correspond to any of the originally proposed dimensions and showed the lowest internal consistency.
The more the items in a unidimensional scale are inter-correlated, the greater their reliability 19. As explained by Streiner & Norman 20, assuming that every answer involves some measurement error, one can reduce the importance of this error by aggregating the answers from a series of questions. A simple way of increasing reliability of the scales proposed here would be to increase the number of items, although this would increase the size of the questionnaire and thus the time needed to apply it. Although more difficult, one way of increasing reliability while maintaining the questionnaire's brevity would be to increase the similarity of the content in the existing items, approaching the meaning of each individual item to that elucidated by the dimensions emerging from the factor analysis.
A limiting factor in the interpretations in this study is the sample's representativeness regarding users of the AIDS medicine dispensing units. The sampling process was submitted to prior choice of the states selected as cases, based on AIDS prevalence criteria in Brazil, as described by Luiza et al. 12. According to the sampling plan, the level of estimation would be each of the 10 target states and the set of the ten states. However, patient selection was not random; due to deficiencies in updating patient registration and ethical issues (participation was voluntary). Losses due to refusal represented an average of 41% of those included in the study, varying from 8% in the State of Rondônia to 124% in Santa Catarina. Thus, the results may not be representative of all people living with HIV under antiretroviral treatment in Brazil, but are valid for describing the study sample.
Notwithstanding, according to the information published in the official Epidemiological Report, the sociodemographic profile of people living with HIV notified to the National STD/AIDS Program is quite similar to that of this study sample 31. The majority of the AIDS cases notified are males, white or mixed-race, with complete primary education (eight years of schooling).
The various associations hypothesized in the literature and confirmed in this study, the maintenance of the multidimensional structure and the convergence between the alternative measurements of satisfaction demonstrate an adequate degree of agreement between the proposed theoretical construct and the results obtained with the instrument. Some changes in the organization of the items in scales were proposed and some items were excluded, resulting in a refinement of the original instrument.
Further research is needed to increase the scales' internal consistency, since this is the only information available thus far on the instrument's reliability. We emphasize that to understand this property better, stability studies are needed, since an instrument's internal consistency can be considered less important when the target concept includes a broad content of meanings. Low internal consistency indicates heterogeneity of the aspects being measured, which may be desirable. However, when one wishes to reduce the data by constructing scales, a minimum level of internal consistency should be obtained, although this value depends on the intended utilization of the data.
Contributors
T. B. Azeredo designed the study, conducted the data analysis, and wrote the draft and final version of the article. M. A. Oliveira, V. L. Luiza and A. Esher helped develop various concepts and interpret the findings.
M. R. Campos contributed in the data analysis and interpretation of findings. All authors reviewed the draft and contributed to the final version of the article.
References
1. Donabedian A. An introduction to quality assurance in health care. New York: Oxford University Press; 2003.
2. Pascoe GYC. Patient satisfaction in primary health care: a literature review and analysis. Eval Program Plann 1983; 6:185-210.
3. MacKeigan LD, Larson LN. Development and validation of an instrument to measure patient satisfaction with pharmacy services. Med Care 1989; 27:522-36.
4. Bernhart MH, Wiadnyana IGP, Wihardjo H, Pohan I. Patient satisfaction in developing countries. Soc Sci Med 1999; 48:989-96.
5. Esperidião MA, Trad LAB. Avaliação da satisfação de usuários: considerações teórico-conceituais. Cad Saúde Pública 2006; 22:1267-76.
6. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Soc Sci Med 1997; 45:1829-43.
7. Ware JEJ, Snyder MK, Wright WR, Davies AR. Defining and measuring patient satisfaction with medical care. Eval Program Plann 1983; 6:247-63.
8. Turris SA. Unpacking the concept of patient satisfaction: a feminist analysis. J Adv Nurs 2005; 50:293-8.
9. Sitzia J. How valid and reliable are patient satisfaction data? An analysis of 195 studies. Int J Qual Health Care 1999; 11:319-28.
10. Schommer JC, Kucukarslan SN. Measuring patient satisfaction with pharmaceutical services. Am J Health Syst Pharm 1997; 54:2721-32.
11. Contandriopoulos AP. Saber preparar uma pesquisa: definição, estrutura, financiamento. 2a Ed. São Paulo: Editora Hucitec/Rio de Janeiro: ABRASCO; 1997.
12. Luiza VL, Esher A, Santos EM, Avelar FG, Emmerick ICM, Oliveira MA, et al. Avaliação nacional da dispensação de medicamentos para as PVHA: relatório final de pesquisa. Rio de Janeiro: Núcleo de Assistência Farmacêutica, Escola Nacional de Saúde Pública, Fundação Oswaldo Cruz; 2006.
13. Esher AF, Azeredo TB, Luiza VL, Oliveira MA, Santos EM. Satisfação do usuário e responsividade do programa nacional de dispensação de medicamentos para o tratamento da AIDS no Brasil: discutindo importantes dimensões [CD-ROM]. Ciênc Saúde Coletiva 2005; 10 Suppl 1.
14. Barbará A, Sachetti VAR, Crepaldi MA. Contribuições das representações sociais ao estudo da AIDS. Interação Psicol 2005; 9:331-9.
15. Rodrigues A, Assmar EML, Jablonski B. Psicologia social. Petrópolis: Editora Vozes; 1999.
16. Crano WD, Prislin R. Attitudes and persuasion. Annu Rev Psychol 2006; 57:345-74.
17. Ajzen I. Nature and operation of attitudes. Annu Rev Psychol 2001; 52:27-58.
18. Griep RH, Chor D, Faerstein E, Werneck GL, Lopes CS. Validade de constructo da escala de apoio social do Medical Outcomes Study adaptada para o português no Estudo Pró-Saúde. Cad Saúde Pública 2005; 21:703-14.
19. DeVellis RF. Classical test theory. Med Care 2006; 44(11 Suppl 3):S50-9.
20. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 3rd Ed. Oxford: Oxford University Press; 2003.
21. Pasquali L. Psicometria: teoria dos testes na psicologia e na educação. 2a Ed. Petrópolis: Editora Vozes; 2004.
22. Stewart DW. The application and misapplication of factor analysis in marketing research. J Mark Res 1981; 18:51-62.
23. Johnson RA, Wichern DW. Applied multivariate statistical analysis. 6th Ed. Upper Saddle River: Prentice Hall; 2007.
24. Atkinson S, Haran D. Individual and district scale determinants of users' satisfaction with primary health care in developing countries. Soc Sci Med 2005; 60:501-13.
25. Weiss GL. Patient satisfaction with primary medical care: evaluation of socialdemographic and predispositional factor. Med Care 1988; 26:383-92.
26. Jackson JL, Chamberlin J, Kroenke K. Predictors of patient satisfaction. Soc Sci Med 2001; 52:609-20.
27. Zwiick WR, Velicer WF. Comparison of five rules for determining the number of components to retain. Psychol Bull 1986; 99:432-42.
28. Long form patient satisfaction questionnaire (PSQ-III). http://www.rand.org/health/surveys_tools/psq/psq3_survey.pdf (accessed on 15/Jan/2005).
29. Hays RD, Davies AR, Ware JE. Scoring the medical outcomes study patient satisfaction questionaire: PSQ-III. http://www.rand.org/health/surveys_tools/psq/psq3_scoring.pdf (accessed on 15/Jan/2005).
30. Alföldi F. La fabrication des critères: savoir évaluer en action sociale et médico-sociale. Paris: Dunod; 2006. p. 33-71.
31. Programa Nacional de DST e AIDS. Boletim Epidemiológico AIDS 2004; Ano I, nº. 1.
Correspondence:
T. B. Azeredo
Núcleo de Assistência Farmacêutica
Escola Nacional de Saúde Pública Sergio Arouca
Fundação Oswaldo Cruz.
Rua Leopoldo Bulhões 1480
Rio de Janeiro, RJ 21041-210, Brasil
thiago.azeredo@ensp.fiocruz.br
Submitted on 26/Oct/2007
Final version resubmitted on 22/Jul/2008
Approved on 11/Dec/2008