Predictive validity of a questionnaire to identify older adults at risk for hospitalization


Validez predictiva de instrumento para identificación del anciano en riesgo de hospitalización



Michel Machado DutraI; Emilio Hideyuki MoriguchiII; Melissa Agostini LampertIII; Carlos Eduardo Poli-de-FigueiredoI

IPrograma de Pós-Graduação em Medicina e Ciências da Saúde. Faculdade de Medicina. Pontifícia Universidade Católica do Rio Grande do Sul. Porto Alegre, RS, Brasil
IIPrograma de Pós-Graduação em Saúde Coletiva. Universidade do Vale do Rio dos Sinos. São Leopoldo, RS, Brasil
IIIDepartamento de Clínica Médica. Faculdade de Medicina. Universidade Federal de Santa Maria. Santa Maria, RS, Brasil





OBJECTIVE: To test the predictive validity of the Probability of Repeated Hospital Admissions questionnaire among older adults.
METHODS: A population-based cohort study with a 6-month follow-up was implemented with 515 non-institutionalized older people (>60 years) cared for by the Family Health Strategy in the city of Progresso, Southern Brazil, in 2005. Participants answered eight objective questions that were entered in a logistic regression model to estimate the risk of future hospital admission, by risk strata. Survival analysis and the receiver operating characteristics curve were utilized to assess instrument validity.
RESULTS: Among participants, 56.1% were women and 10.1% were hospitalized. The high-risk group had a 6.5 times greater frequency of hospitalization in comparison to the low-risk category.
CONCLUSIONS: The instrument is effective in assessing the risk of hospitalization among older adults attended by the Family Health Program of the National Unified Health Care System.

Descriptors: Aged. Risk Factors. Hospitalization. Family Health Program. Validation Studies.


OBJETIVO: Evaluar la validez de predicción del Instrumento de Predicción de Riesgo de Admisión Hospitalaria Repetida de la hospitalización de ancianos.
MÉTODOS: Estudio de cohorte de base poblacional con seguimiento de seis meses con 515 ancianos (>60 años) no institucionalizados, atendidos por la Estrategia Salud de la Familia en la ciudad de Progresso, Sur de Brasil, en 2005. Los ancianos respondieron a ocho preguntas objetivas, que fueron reunidas en modelo de regresión logística para estimar su riesgo de admisión hospitalaria futura, por estratos de riesgo. Análisis de sobrevida y la curva Receiver Operating Characteristics fueron empleadas para comparar la validez del instrumento.
RESULTADOS: De los entrevistados, 56,1% eran mujeres y 10,1% fueron hospitalizados. El grupo de riesgo alto tuvo frecuencia de internación hospitalaria 6,5 veces superior con relación al grupo de riesgo bajo.
CONCLUSIONES: El instrumento es efectivo al predecir el riesgo de hospitalización de los ancianos atendidos por la Estrategia Salud de la Familia del Sistema Único de Salud.

DESCRIPTORES: Anciano. Factores de Riesgo. Hospitalización. Programa de Salud Familiar. Estudios de Validación.




Spending by the Sistema Único de Saúde (SUS -Brazilian National Unified Health System) will increase considerably over the next decades with an increased older population and greater frequency of chronic degenerative diseases. In 2004, the hospitalization rate in the SUS was 41.8 admittances per 1,000 residents among the 0 to 14 year age group, 61.2 for the 15 to 59 year age group and 143.3 for the 60 or older group.3,5,8

In 2004, 25% of SUS expenses were attributed to hospitalization of older adults, which were 8.4% of national population in this period. Expenditures on hospitalizations were R$ 20.6 (US$ 7.36ª) per person/year for the younger group, R$ 33.2 (US$ 11.86) for the 15 to 59 year age group and R$ 107.3 (US$ 38.32) for the group equal to or older than 60 years. In the period from 2001 to 2004, hospitalization expenses increased 7% per individual in the 0 to 14 group, 9% in the 15 to 59 year group and 13% in the 60 or older group.15,b,c

Hospitalization of older adults generates a series of events that frequently culminate in diminished functional capacity and quality of life, complications unrelated to the reason for admission.4,7,18

Therefore, for many older adults hospitalization does not result in improved health and on the contrary, is often correlated with increased mortality and morbidity rates, worse prognosis and predisposition for frailty.7,18

In SUS, health care for older adults is defined from an understanding of population heterogeneity in terms of health service utilization. According to this concept, care for older adults should be hierarchized and health actions for healthy older people should be prioritized, together with programs suited for those already sick. The Family Health Strategy is responsible for managing care of older people in SUS, performing risk stratification and identifying fragile adults.12,16,17,b

To enable these actions, the development of an objective method to identify older adults at risk of hospitalization is fundamental. This procedure should have measurement validity, be easy to administer and produce easy to understand results so that Community Health Agents and Family Health Teams can utilize results to establish care priorities and preventive interventions among the population.6,10

In a cohort study, Boult et al1 (1993) identified eight risk factors for increased use of hospital services by older adults (age greater than 75 years, male sex, willingness of caretaker, self-perceived poor health, cardiovascular disease, diabetes mellitus, hospitalization in previous 12 months and more than six medical visits in previous 12 months), which are also valid for the identification of fragile adults. Based on these results, Boult et al1 developed a Probability of Repeated Hospital Admissions (Predição de Risco de Admissão Hospitalar - PRA) questionnaire.1,3,12,13 In accordance with the presence or not of risk factors, the logistic regression model applied to PRA generates a value from 0 to 1. Higher values correspond to increased risk of hospital admittance.

Two studies evaluated the validity of PRA in Brazil. The first study, performed by Veras et al19 (2000), administered PRA at the reception of a geriatric ambulatory clinic, and the second, by Negri et al12 (2004), utilized PRA in the reception of a Family Health Basic Unit. In both studies, the PRA showed validity as an instrument to identify and classify older adults into risk strata for hospitalization. Nonetheless, these studies were cross-sectional and therefore did not allow correlation of PRA risk strata for hospitalization to the outcome hospitalization. In addition, samples were restricted to older adults that sought care in specific health services.

PRA validity for prediction of hospitalization among older adults in the United States can not be generalized to the Brazilian older population, due to characteristics of SUS and other country-specific factors, such as: availability of hospital sites, access to medical assistance and to basic health care and determinants of health service utilization. 20,21

This study's objective was to evaluate PRA predictive validity for hospital admittance of older adults.



A population-based prospective cohort study was performed in the municipality of Progresso, Southern Brazil, in 2005. Progresso has 6,256 residents, including 694 (11.1%) age 60 years or older. The human development index was 0.738.d The municipality relies on a low complexity hospital accredited by SUS.

The Municipal Health Secretary of Progresso, in partnership with the Medical and Health Sciences Post-Graduate Program of Pontifícia Universidade Católica do Rio Grande do Sul, developed a publicly available database on health status of older adults registered in the Basic Care Information System of the Family Health Strategy. The questionnaire, used to create the municipal database, has 20 questions, eight pertaining to the PRA and the others evaluating functioning and health.b

Information on the PRA questionnaire, logistic equation and regression coefficients are described by Pacala et al13 (1995).

Collection of primary data was performed in February and March of 2005, with non-institutionalized older people of both sexes, in rural and urban areas. Community Health Agents were trained to administer the research tool and conducted household visits.

In February of 2005, 551 people age 60 years or older were registered in the Basic Care Information System in Progresso municipality. Among these older people, 36 (6.5%) were excluded from analysis, 18 because the questionnaire was answered by or with assistance from the caretaker and 14 had a terminal or advanced-stage illness. The sample contained 515 people, which corresponded to 74.2% of the elderly population in Progresso.

Study participants were classified into strata according to PRA value and cutoff points suggested by Veras19 (2003) and Lourenço et al9 (2005): low risk (PRA<0.30), medium risk (PRA=0.30-0.39), medium-high risk (PRA=0.40-0.49) and high risk (PRA>0.500).

The primary outcome studied was occurrence of hospital admittance within six months (182 days) after questionnaire administration.

Since the PRA predicts hospitalization resulting from non-communicable, chronic degenerative diseases,2,3,13,14 the analysis excluded 14 (13.7%) hospitalizations that were unrelated health problems, medical issues of minor clinical importance and admittances for elective surgical procedures. These exclusions corresponded to admittances for: unspecified dizziness (1), fall due to environmental factor (1), contact with spider (2), anxiety (2), medical complication of skin graft (1), watery diarrhea (3), surgical correction of utero-vaginal prolapse (1), unspecified abdominal pain (1), prosthetic hip surgery (1) and mandibular disease (1).

Descriptive statistics and a 95% confidence interval were calculated to estimate average frequency of hospitalizations according to risk strata.

The association between the frequency distribution of nominal variables from the questionnaire and the outcome were analyzed by contingency tables. Chi-square test was used to analyze the association between qualitative variables.

Survival analysis was performed to determine the probability of older adults presenting the outcome hospitalization within six months, according to PRA risk strata. Outcome probability distribution was estimated by the Kaplan-Meier method with censoring.

Survival functions were estimated separately for each group, and a log rank test was used to compare survival curves of different risk groups.

Receiver Operating Characteristics analysis was used to ascertain PRA validity for prediction of hospitalization within six months, by calculating sensitivity and specificity of different cutoff points.

Data analysis was done with SPSS 11.5.

During the study period, the Municipal Health Secretary of Progresso and the Family Health teams did not have access to study data.

This research project was approved by the Research Ethics Committee of São Lucas Hospital, Pontifical Catholic University of Rio Grande do Sul (Approval number: 05/02501 of 07/08/2005).



Of the total older adults studied (515), 289 were female, with average age 70.0 years (SD=7.16), and among the 226 males, average age was 69.9 years (SD=6.77).

Among participants, 31.6% (n=163) considered their health excellent or very good and 21.9% (n=113) average or poor. In addition, 77.3% (n=398) were less than 75 years (Table 1).



Of the total participants, 52 (10%) had 88 hospital admittances, and 17 (3.3%) had hospital readmissions and were responsible for 53 (60.2%) hospitalizations. The hospitalization rate for men was 7.1% (n=16) and among women was 12.5% (n=36) (chi-square test, X2 = 0.03).

In the high risk group, the frequency of hospital admittance was 6.5 times greater than low risk groups (Table 2).

The outcome prevalence changes for PRA scores between 0.3 and 0.5. Using a value of 0.5 it was possible to identify 7% of the study population that had 33.3% hospital admittance prevalence (Table 3).



The ROC curve showed a predictive accuracy of 0.674 (95%CI: 0.588;0.760) (Figure 1). With a cutoff point of 0.4, the sensitivity and specificity for prediction of hospitalization were 42.3% and 84.7% respectively, with 18.1% of older adults classified in this stratum.



Survival curves (proportion of patients free of hospitalization) for the risk strata showed no statistical difference between the medium risk and medium-high risk strata (log rank test: p=0.618). The same phenomenon occurs when comparing medium-high risk to high risk (log rank test: p=0.076). However, statistically significant differences were found between the other strata during study period (log rank test: p<0.01) (Figure 2).



The grouping of strata in low risk (PRA<0.30), medium risk (PRA=0.30-0.49) and high risk (PRA>0.500) generates survival curves with statistically significant differences for outcomes (log rank test p<0.001).

The three main causes of admittances among women were circulatory system diseases (36.1%), respiratory system diseases (19.4%) and endocrine, nutritional and metabolic diseases (16.7%). Among men, the three main causes were respiratory system diseases (53.3%), nervous system diseases (13.3%) and endocrine, nutritional and metabolic diseases (13.3%).



PRA allowed identification of 7% of older adults as high risk for hospitalization within six months, a similar result as found in United States (7%) and Europe (4% to 7%).1,11,21 Nonetheless, some study limitations should be mentioned.

The logistic regression model for prediction of hospitalization is based on data from the United States elderly population that participated in the original study.3 Therefore, the present study has the same limitations, including the definition of elderly (age 70 years or older) and omission from the logistic equation of factors that could influence hospitalization, such as lifestyle, dementia and psychological conditions.1,21

The follow-up period was six months, while Boult et al1 (1993) extended the follow-up for four years. Nonetheless, other studies evaluated PRA for shorter follow-up periods, such as Wagner et al21 (one year), Pacala et al13 (1995) and Vojta et al20 (2001) (six months) and obtained a predictive performance similar to the original study.1

Adjustment for confounding variables, such as exclusion of older adults that required caretaker assistance to answer the questionnaire, had cognitive deficits or were gravely ill, can underestimate selection bias, prevalence of hospitalization and incapacity in the study population.6

An estimated 25.8% (179) of older residents in Progresso did not participate in the study. It was not possible to determine any potential differences between the participant and non-participant groups. The possibility that the non-participant group had worse health conditions should be considered.

The group identified as high risk experienced 6.5 times more hospitalizations than the low risk group. The four cutoff points for stratification of low, medium, medium-high and high risk were not statistically significant for identifying risk strata for hospitalization in six months. Nonetheless, the following strata were statistically significant: low risk (PRA<0.30), medium risk (PRA=0.30-0.49) and high risk (PRA>0.500).1,11,19

The observed predictive accuracy of the area under the ROC curve was 0.674, similar to results by Boult et al1 (area=0.610), Mosley et al11 (area=0.638), Wagner et al21 (area=0.64) and Vojta et al20 (area=0.68).

In our study, the PRA showed improved ability to identify elderly at risk of hospitalization within six months, when a cutoff value of 0.4 was used to define the high risk population.

A group of 17 older adults (3.3%) was responsible for 60.2% of hospitalizations. These data reinforce the fact that the group of older adults is not homogeneous in regards to use of health services. The Medicare program in the United States found that 62% of hospital expenses with older adults are due to excessive demand generated by a relatively small subgroup (5%). In addition, these older people receive care only in the most advances stages of disease, and despite the excessive utilization of hospital resources, spending does not correspond to improved health status for this minority.1,9,14,21

The use of PRA would allow Family Health Teams to intervene with preventive actions and avoid hospitalization of older adults. In this context, the Family Health Strategy can meaningfully impact demand for basic health care among older adults.2,8,17,19,b

PRA would permit the Family Health team to define appropriate actions and establish an intervention strategy with older adults to reduce hospitalizations, considering the amount of high risk individuals. Care can be hierarchized in accordance with risk of hospitalization, and prioritization in care for older adults can be established according to cutoff value selected. This decision will be based on the desired sensitivity and specificity of PRA. The choice of a high cutoff value (PRA>0.4) would tend to increase the number of vulnerable individuals detected, with a lower proportion of misidentified cases.2,9,12,19

This study identified older adults at risk of hospitalization through household visits by Community Health Workers and concluded that PRA validity was similar to initial studies. Therefore, inclusion of PRA in the routine of Family Health Teams and in SUS information systems should be considered.



1. Boult C, Downd B, McCaffrey D, Boult L, Hernandez R, Krulewitch H. Screening elders for risk of hospital admission. J Am Geriatr Soc. 1993;41(8):811-7.         

2. Boult C, Boult L, Murphy C, Ebbitt B, Luptak M, Kane RL. A controlled trial of outpatient geriatric evaluation and management. J Am Geriatr Soc. 1994;42(5):465-70.         

3. Boult L, Boult C, Pirie P, Pacala JT. Test-retest reliability of a questionnaire that identifies elders at risk for hospital admission. J Am Geriatr Soc. 1994;42(7):707-11.         

4. Campbell SE, Seymour DG, Primrose WR. A systematic literature review of factors affecting outcome in older medical patients admitted to hospital. Age Ageing. 2004;33(2):110-5. DOI:10.1093/ageing/afh036        

5. Chaimowicz F. A saúde dos idosos brasileiros às vésperas do século XXI: problemas, projeções e alternativas. Rev Saude Publica. 1997;31(2):184-200. DOI:10.1590/S0034-89101997000200014        

6. Costa MFL, Barreto SM. Tipos de estudos epidemiológicos: conceitos básicos e aplicações na área do envelhecimento. Epidemiol Serv Saude. 2003;12(4):189-201.         

7. Creditor MC. Hazards of hospitalization of the elderly. Ann Intern Med. 1993;118(3):219-23.         

8. Lima-Costa MF, Veras R. Saúde pública e envelhecimento. Cad Saude Publica. 2003;19(3):700-1. DOI:10.1590/S0102-311X2003000300001        

9. Lourenço RA, Martins CSF, Sanches MAS, Veras RP. Assistência ambulatorial geriátrica: hierarquização da demanda. Rev Saude Publica. 2005;39(2):311-8. DOI:10.1590/S0034-89102005000200025        

10. MacCusker J, Bellavance F, Cardin S, Belzile E, Verdon J. Prediction of hospital utilization among elderly patients during the 6 months after an emergency department visit. Ann Emerg Med. 2000;36(5):438-45.         

11. Mosley DG, Peterson E, Martin DC. Do hierarchical condition category model scores predict hospitalization risk in newly enrolled medicare advantage participants as well as probability of repeated admission scores? J Am Geriatr Soc. 2009;57(12):2306-10.         

12. Negri LSA, Ruy GF, Collodetti JB, Pinto LF, Sorans DR. Aplicação de um instrumento para detecção precoce e previsibilidade de agravos na população idosa. Cienc Saude Coletiva. 2004;9(4):1033-46. DOI:10.1590/S1413-81232004000400024        

13. Pacala JT, Boult C, Boult L. Predictive validity of a questionnaire that identifies older persons at risk for hospital admission. J Am Geriatr Soc. 1995;43(4):374-7.         

14. Pacala JT, Boult C, Reed RL, Aliberti E. Predictive validity of the PRA instrument among older recipients of managed care. J Am Geriatr Soc. 1997;45(5):614-7.         

15. Peixoto SV, Giatti L, Afradique ME, Lima-Costa MF. Custo das Internações Hospitalares entre idosos brasileiros no âmbito do Sistema Único de Saúde. Epidemiol Serv Saude. 2004;13(4):239-46.         

16. Ribeiro EM, Pires D, Blank VLG. A teorização sobre processo de trabalho em saúde como instrumental para análise do trabalho no Programa Saúde da Família. Cad Saude Publica. 2004;20(2):438-46. DOI:10.1590/S0102-311X2004000200011        

17. Silvestre JA, Costa Neto MM. Abordagem do idoso em programas de saúde da família. Cad Saude Publica. 2003;19(3):839-47. DOI:10.1590/S0102-311X2003000300016        

18. Siqueira AB, Cordeiro RC, Perracini MR, Ramos LR. Impacto funcional da internação hospitalar de pacientes idosos. Rev Saude Publica. 2004;38(5):687-94. DOI:10.1590/S0034-89102004000500011        

19. Veras RP. Em busca de uma assistência adequada à saúde do idoso: revisão da literatura e aplicação de um instrumento de detecção precoce e de previsibilidade de agravos. Cad Saude Publica. 2003;19(3):705-15. DOI:10.1590/S0102-311X2003000300003        

20. Vojta CL, Vojta DD, TenHave TR, Amaya M, Lavizzo-Mourey R, Asch DA. Risk screening in a medicare/medicaid population administrative data versus self report. J Gen Intern Med. 2001;16(8):525-30. DOI:10.1046/j.1525-1497.2001.016008525.x        

21. Wagner JT, Bachmann LM, Boult C, Harari D, von Renteln-Kruse W, Egger M, et al. Predicting the risk of hospital admission in older persons - validation os a brief self- administered questionnaire in three European countries. J Am Geriatr Soc. 2006;54(8):1271-6. DOI:10.1111/j.1532-5415.2006.00829.x        



Emilio Hideyuki Moriguchi
Av. Carlos Gomes, 1000, Conj. 401
90480-001 Porto Alegre, RS, Brasil
Email: moriguch@terra.com.br

Received: 2/17/2010
Approved: 6/6/2010
Research funded by Conselho Nacional de Desenvolvimento Científico e Tecnológico (Nº Procedure: 401221/2005-9).



Article based on the master's dissertation by Dutra MM, presented to the Programa de Pós-Graduação em Medicina e Ciências da Saúde da Faculdade de Medicina da Pontifícia Universidade Católica do Rio Grande do Sul in 2005.
The authors declare that there are no conflict of interests.
a US$ 1.00 was worth R$ 2.80 in 2004.
b Ministério da Saúde. Departamento de Informação e Informática do SUS - DATASUS. Informações de Saúde: Nota Técnicas. [cited 10 Nov 2009] Available from: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sih/cnv/miuf.def
c Dutra MM. Identificação do idoso sob risco de hospitalização no Programa Saúde da Família [master's dissertation]. Porto Alegre: Pontifícia Universidade Católica do Rio Grande do Sul; 2005.
d Instituto Brasileiro de Geografia e Estatística. Resultados da amostra do Censo Demográfico 2000 - Malha municipal digital do Brasil: situação em 2001. Rio de Janeiro: IBGE, 2004 [cited 2010 May 12]. Available from: http://www.ibge.gov.br/cidadesat/topwindow.htm

Faculdade de Saúde Pública da Universidade de São Paulo São Paulo - SP - Brazil
E-mail: revsp@org.usp.br