SECTION II
RESEARCH AND METHODOLOGIES
Development and validation of predictive MoSaiCo (Modello Statistico Combinato) on emergency admissions: can it also identify patients at high risk of frailty?
Sviluppo e validazione di MoSaiCo, Modello Statistico Combinato, sul ricovero non programmato: può identificare pazienti ad alto rischio?
Pasquale Falasca; Arianna Berardo; Francesca Di Tommaso
Servizio Valutazione Strategica, Azienda USL di Ravenna, Ravenna, Italy
SUMMARY
The prospective historical cohort study develops and validates a method of identifying patients at high risk of emergency admission to hospital in the population of the Province of Ravenna (no. = 296 641). The main outcome measure is: emergency hospital admission analyzed using multivariate logistic regression (MoSaiCo - Modello Statistico Combinato). To validate the findings, the coefficients for 30 most powerful variables found on half of the population (derivation data set) were then applied to the rest of the population (validation data set). The key predicting factors included some demographic variables, social variables, clinical variables and use of health/social services. Discriminatory power and validation both reached good results. Risk score increases when variables indicating the individual vulnerability raise. The predictive frailty risk resulting from MoSaiCo allows to stratify the population, to organize care services, to provide a practical planning tool in the field of case management and management of frail patients.
Key words: frailty, chronic care model, predictive model, score index.
RIASSUNTO
Lo studio, di coorte prospettico storico, sviluppa e valida un metodo per identificare i pazienti ad alto rischio di ricovero urgente della popolazione della Provincia di Ravenna (n. = 296 641). La principale variabile di esito: il ricovero in emergenza è analizzato con un modello multivariato di regressione logistica (MoSaiCo - Modello Statistico Combinato). Per validare i risultati, i coefficienti di 30 variabili individuate su metà della popolazione (gruppo di derivazione) sono state applicate al resto della popolazione (gruppo di validazione). I principali fattori predittivi includono variabili demografiche, sociali, cliniche e di uso dei servizi sanitari e sociali. Il potere discriminante e la validazione hanno ottenuto buoni risultati. L'aumento del risk score corrisponde all'aumento delle variabili che indicano una situazione di vulnerabilità dell'individuo. Il rischio di predizione ottenuto permettono di stratificare la popolazione, pianificare gli interventi clinico-assistenziali, guidare la riorganizzazione dei servizi per gli interventi di case management e per la gestione del paziente fragile.
Parole chiave: fragilità, modello malattie croniche, modello predittivo, indice di rischio.
INTRODUCTION
Background
European health policies recently stressed the importance of the strategic role of long term condition management to prevent health deterioration. Supporting healthy ageing means both promoting health throughout the lifespan, aiming to prevent health problems and disabilities from an early age, and tackling inequities in health linked to social, economic and environmental factors [1]. To fulfil this aim, it is important to identify chronic subjects and consequently adopt health care interventions such as self/disease/case management and implement innovation of assistive technology (Chronic Care Model) [2].
In the past years there has been significant progress in improving care of patients with chronic illness by providing guidance on evidence-based pathways of care [3], setting targets for reduced hospital re-admission [4], funding more efforts to encourage self-care [5], identifying more accurately those at highest risk of admission [6], and encouraging effective case and disease management [7, 8].
To achieve these goals different methods have been proposed to identify patients at high risk of admission. Such methods are mainly based on three techniques [9]. The first is "threshold modelling" or "criterion-based modelling" which identifies any patient who meets a specific risk criterion (eg. age threshold to identify patients who have to undergo screening). The second is "clinical knowledge", which is based on the ability of the clinician to identify patients at high risk of future admissions [10]. The third is "predictive modelling", which tries to establish a relation between a set of variables in order to predict future outcomes using a regression model [11]. Whatever technique may be used, evidence shows that the prediction ability of the model depends on the number and the quality of patient characteristics: social, demographic and clinical variables, use of health and social services, functionality and perceived health status [9].
Many predictive algorithms, based on administrative data, have been produced in the United States to identify high risk of admission among elderly subjects [12, 13]. The risk stratification model based on admissions data, has been developed by the King's Fund, New York University, and Health Dialog, but currently this is only based on predicting readmission in those who have already experienced an admission [14]. A further step has been the development of a case finding algorithm to identify high risk patients accurately so as to enable preventive and targeted interventions [15]. The Combined Predictive Model [16] is heading precisely in this direction. It is based on a comprehensive dataset of patient information obtained by healthcare databases, including inpatient, outpatient, Accident & Emergency (A&E) and data from secondary care sources (such as electronic clinical records of primary care). Recently, new algorithms have been introduced that increase variables not only on previous hospital admissions, but on the use of social services, its related costs [17] and the influence of the deprivation index [18].
Finally, apart from being applied to the elderly population (> 65 years), these algorithms have been also applied to young subjects with an experience of emergency admission (> 40 years [19]) or to subjects of every age [20].
Although the previously mentioned models enabled recognition of the risk degree in different age groups, little attention has been paid to the role played by a range of social variables. In our opinion and on the basis of recent empirical [21] evidence, to include and analyse the role of such variables could increase the accuracy of the model's prediction.
Furthermore, in Italy there are still few validated models devoted to the stratification of the population into classes of risk and consequently to promoting forms of case-management along with social and health policies according to the needs of the population.
Objective
The aim of this study is to develop and validate a statistical combined model named MoSaiCo (Combined Statistical Model) to predict future emergency hospital admission or mortality in all individuals aged 18 or above in the following year in the Province of Ravenna. This model could be used by clinicians and policy makers to guide and implement proactive interventions.
METHODS
Study design and data source
We conducted a prospective historical cohort study among all residents alive on 1 January 2006 and aged >18 years (310 920), residing in the Province of Ravenna (Emilia Romagna Region, Northern Italy) and who were entered into the RAA (Ravenna Population Registry). The baseline was defined (296 641, Figure 1) after the initial period of 2 years (from 1 January 2006 to 31 December 2007) and follow-up occurred over the following year (from 1 January 2008 to 31 December 2008). Those subjects with either less than 2 years'history or less than 1 year of follow-up data were excluded.
The Ravenna Population Registry is a high quality database created over almost twenty years of activity. It attributes to each patient a unique identification number enabling a record linkage with all other available databases mainly derived and validated by the dataflow of Emilia Romagna Region [22]: SDO, Hospital Discharge Record; ESE, participation in the prescription charges; PS, Accident & Emergency data; ASA, Outpatient data; AFT, Territorial Drug Prescription; ADI, home care services; SA, lone elderly aged > 75 years and elderly couples aged > 75 years; SS, social services data; SM, mental health services data.
Baseline risk factors and outcome
During the observation period 2006-2008, for each individual a set of risk factors was detected. This set includes some variables derived from the English Combined Model [16], such as demographic and clinical variables, use of health services and further social variables and use of social services.
Demographic variables reflect the age group, gender and citizenship.
Social variables include: exemption from prescription charges for invalidity and primary social care network for the elderly (people living alone aged > 75 years and elderly couples aged > 75 years).
For those who had at least one hospitalization, some clinical variables were identified such as chronic diseases based on ICD9-CM in any diagnostic field in hospital discharge record (asthma, coronary artery disease, congestive heart failure, cancer, depression, chronic obstructive pulmonary disease, hypertension, diabetes, dementia) as well as the Charlson co-morbidity index [23].
The "use of social services" variables refer to the use of home care services and the use of the Mental Health Department.
The "use of the health services" variables include: emergency admission (number of visits, tests performed, non-injury medical diagnoses, arrived by ambulance), hospital admissions (number of inpatient admissions including passive inter-regional mobility, protected discharge), polipharmacy (at least 4 drug prescriptions in the last 3 months from different ATC-groups at Level 3).
Finally, the "use of social services" variables include the use of at least one social service (home meal delivery, tele-assistance, day care centres, etc.) and were collected with an ad hoc study in which all social services provided the names of their users.
For privacy reasons [24], in the final dataset a unique anonymous identification number was introduced.
The main outcome was a single binary variable, EAM (Emergency hospital Admission and Mortality), which includes both the first emergency hospital admission obtained from the hospital discharge record (field "Type of admission": 2 = emergency, 3 = forced mental health treatment, 5 = short-stay emergency observation) and mortality in the follow-up year derived from the Ravenna Population Registry, updated monthly.
Statistical methods
Risk factor variables are summarized as percentages.
The data set was split in half at random into a derivation data set and a validation data set (each corresponds to the 50% of the population). In the derivation data set, the main binary outcome of first emergency admission or mortality in the following year was modelled using logistic regression. From this model, odds ratio (ORs) and associated 95% confidence intervals (CIs) were obtained by exponentiating the regression coefficients. Absolute risk was estimated from the linear prediction starting from the log odds of the final model as a risk score, calculated for each individual from 1 to 100.
Model building
All initial bivariate and multivariable models were developed on the derivation set and the performance of the final models was tested using the validation set.
In order to potentially include confounding factors in MoSaiCo, a total of 57 variables were considered. Of these, a total of 37 had a frequency >5% and so these were included in subsequent multivariable modelling. Having a large number of covariates, we opted for a selection method based on the combination of stepwise logistic regression with a "right" critical p-value = 0.15 [25].
Model performance
The performance of the algorithm obtained from the derivation data set was tested on the randomly selected validation data set. Firstly, overall discrimination ability was assessed for the derived function on the derivation data itself, and secondly, this model was used on the validation data set. Discrimination was assessed using the c statistic, or area under curve, which is an estimate of the probability of assigning a higher risk to those who have an emergency admission in the following year compared with those who do not. This is an important criterion when ranking people by risk and is clearly essential for risk stratification.
The Brier score [26] has been used to determine the calibration of the model. The Brier score is a measure of suitable matching where lower values indicate better accuracy.
Finally, the c statistic, or area under curve, and calibration test were also calculated for the derived algorithm applied to the test validation data set. All analyses were implemented in SAS version 9 (SAS Institute Inc, Cary, North Carolina) statistical software.
RESULTS
The creation of derivation and validation cohorts is shown in Figure 1, while Table 1 compares the characteristics of eligible patients in both cohorts. The baseline validation cohort characteristics were very similar to those in the derivation cohort, without statistically significant variations.
Table 2 describes the variables used in the model and gives the odds ratios for the factors in the final model (in the derivation cohort). Those who experienced an outcome of an emergency admission tended to be older and males (7.2% with slightly less difference for females).
The factors mainly related with the EAM variable (outcome of an emergency admission or mortality, rate 6.6%) proved to be age, both from 65 to 84 years (OR 2.42; CI 2.28-2.57) and from 85 years onwards (OR 5.69; CI 5.19-6.24), one or more emergency admissions over the last 30 days (OR 2.20; CI 1.85-2.63), three or more visits to the A&E (OR 1.77; CI 1.50-2.09).
ROC Analysis revealed reasonable predictive power of the risk score in the validation data set with a c statistic = 0.77; applying the model to the random split-half validation data set, discriminatory power was still good (c = 0.79). Calibration was good (Brier score was 0.053 for derivation data set and 0.054 for validation data set). Figure 2 shows the curves of the expected and observed cases of risk of EAM divided into deciles of risk. The close correspondence between predicted and observed emergency admission risks within each decile (reaching 95% in the last decile) suggests that the model was well calibrated. The appropriateness of the calibration is further demonstrated in Figure 3 illustrating observed versus expected EAM percentages in the several risk categories. Dots are aligned to the diagonal line that represents perfect calibration.
Table 3 indicates EAM risk categories with some descriptive variables which we believe to be indicative of frailty condition: the use of social services and being alone are, in fact, indicators of the paucity of the social network. The presence of chronic diseases and invalidity can indicate a decline in functional independence or a worsening of the health condition as well as becoming non self-sufficient. Such variables show a monotonic increase for each of the EAM risk categories. This trend reflects the possibility of also using such risk scores to predict long term frailty conditions.
DISCUSSION
The main result of this study is the calculation, for each subject, of the risk score of EAM. This was possible thanks to the MoSaiCo predictive algorithm, which, reasonably powerful and calibrated, is aimed at the stratification of the resident population to provide suggestions that could help planning health care interventions. From this point of view, this algorithm can be a precious tool to carry out a re-engineering of health and social services and improve different activities of case managers. Besides, this predictive model can help doctors to make decisions by providing more objective estimates of probability as a supplement to other relevant clinical information [27].
MoSaiCo is oriented on the assessment, coordination, monitoring and delivery of services to meet patients' needs [28], in the present case preventive and responsive care for patients aged over 65 years at high risk of emergency hospital admission [29]. Qualitative evidence suggests that access to case management added a frequency of contacts, regular monitoring, psychosocial support, and a range of proactive medicine initiatives on behalf of social care that had significant impact on rates of emergency admission [30, 31].
In fact, decisions on admission to hospital are usually made with a holistic view of the patient's current state of health, existing co-morbidities, available social support, and the patient's concerns and expectations. Future guidance of health care services should incorporate perspectives from social services, primary care, patients, and carers [32] and also use the tool of MoSaiCo, that suggests the conditions on which institutions take decisions, the user of the service, the variety and quantity of services that should be supplied in order to provide an efficient service.
As reported in the recommendations by Hutt et al. [33], thanks to this tool Primary Care Groups (PCGs) will have a clear idea of the needs of the population at whom case management is targeted. Case management should be developed in close collaboration with social care providers to ensure that an appropriate range of health and social care services is available to prevent unplanned hospital admission. In addition to identifying people who will most benefit from case management, PCGs need to ensure that services are in place for people with less severe illnesses who nevertheless have significant health and social care needs. To conclude, all case-management initiatives should be evaluated in terms of their impact on the use of health services, including primary care and patient satisfaction. The whole of these initiatives can be realized through the use of MoSaiCo.
Comparing previous studies that calculate the emergency admission risk on the basis of administrative databases (Table 4) one can observe a certain tendency towards increased performance (AUC = area under curve) along with the introduction of different types of variables, namely social and use of social services ones (independent factors type).
The innovation of this study derives from the hypothesis that MoSaiCo can calculate a predictive risk not only of hospital re-admission, but also of frailty condition. In writings on the subject, in fact, frailty is defined as a decrease in the capacity to carry out the main social and practical activities of daily life [34]. It is a multidimensional concept that considers the complex interplay of physical, psychological, social, and environmental factors such as: medical and biological factors (chronic diseases), psychological factors (depression, coping skill) and social factors (relationships, interaction with the environment, social adaptability) [35]. As frailty can appear in different degrees of gravity, and even lead to adverse health-related outcomes such as an increased risk of morbidity, of emergency hospitalization and long term assistance, prevention and, where possible, treatment of frailty should be high on the medical [21] and social agenda.
The main tools used in the geriatric and medical field to carry out prognoses and to predict the degree of frailty is face-to-face interviews that contain several information possibly measurable from current data flows [36, 37]. Since MoSaiCo contains many information of this type, it could provide the possibility to predict the level of frailty in health areas, using the existing electronic databases of the Italian National Health System. It would allow a systematic screening applied to the whole of the population and consequently expose the services less to the inverse care law [38, 39].
Strengths and weaknesses of the study
Among the strengths of this study, we can highlight the breadth of the population and the use of high quality administrative databases. The latter though, may contain errors due to misclassification (e.g. diagnosis of chronic diseases).
Another limit of the study is the absence of external validation in order to verify that the model performs as expected in new but similar patients.
Further improvements could be applied to the model by adding the variables linked to socio-economic variables (the deprivation index is under observation in the Emilia Romagna Region), the propensity score [40, 41] and other data provided by general practitioners.
The latter could provide the lists of their frail patients and include in the MoSaiCo model the information on deprivation index, social capital and, at the same time, give the patients a chance to be monitored and receive tailored preventive measures.
The hypothesis of using MoSaiCo to predict the frailty level offers a new perspective which requires further validation studies and a new assessment scenario on the impact of Health and Social care on long-term frail patients management. Bearing this in mind, as did Lyon et al. [42], we started a survey on a representative sample of the elderly population in order to detect the social and psychological characteristics that cannot be detected in the administrative flows. The goal of this survey is to create a tool (Frailty Risk Chart) which allows social and health operators the calculation of some (standardized and easy-to-use) indicators in order to timely examine the psychosocial conditions of individuals and calculate an individual score of frailty to implement preventive measures.
Conflict of interest statement
There are no potential conflicts of interest or any financial or personal relationships with other people or organizations that could inappropriately bias conduct and findings of this study.
References
1. Commission of the European Communities. White Paper. Together for health. A strategic approach for the EU 2008-2013. Brussels: Commission of the European Communities; 2007. Available from: http://ec.europa.eu/health/strategy/policy/index_en.htm.
2. Ham C. Chronic care in the English National Health Service. Progress and challenges. Health Affairs 2009;Jan/Feb;28:1.
3. United Kingdom. Department of Health. Supporting people with long term conditions: an NHS and social care model to support local innovation and integration. London: Department of Health; 2005. Available from: www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4100252.
4. Lyratzopoulos G, Havely D, Gemmell I, Good GA. Factors influencing emergency readmission risk in a UK district general hospital: a prospective study. BMC Emerg Med 2005;5:1.
5. Foster G, Taylor SJC, Eldridge S, Ramsay J, Griffiths CJ. Self-management education programmes by lay leaders for people with chronic conditions. Cochrane Database of Systematic Reviews 2009;Issue 3.
6. Roland M, Dusheiko M, Gravelle H, Parker S, Follow up of people aged 65 and over with a history of emergency admissions: analysis of routine admission data. BMJ 2005;330:289-92.
7. Hutt R, Rosen R, McCauley J. Case-managing long-term conditions. What impact does it have in the treatment of older people? London: King's Fund; 2004. Available from: www.kingsfund.org.uk/document.rm?id=90.
8. United Kingdom. Department of Health. Our health, our care, our say. London: Department of Health; 2006. Available from: www.dh.gov.uk/en/Healthcare/Ourhealthourcareoursay/index.htm.
9. Curry N, Billings J, Darin B, Dixon J, Williams M, Wennberg D. Predictive risk project. Literature review. London: King's Fund; 2005. Available from: www.networks.nhs.uk/62.
10. Black DA. Case management for elderly people in the community. BMJ 2007;334(7583):3-4.
11. Cousins MS, Shickle, Bander JA. An introduction to predictive modeling for disease management risk stratification. Disease Management 2002;5(3):157-67.
12. Marcantonio ER, McKean S, Goldfinger M, Kleefield S, Yurkofsky M, Brennan TA. Factors associated with unplanned hospital readmission among patients 65 years of age and older in a Medicare managed care plan. Am J Med 1999;107(1):13-7.
13. Reuben DB, Keeler E, Seeman TE, Sewall A, Hirsch SH, Guralink JM, Development of a method to identify seniors at high risk for high hospital utilization. Med Care 2002;40(9):782-93.
14. Billings J, Mijanovich T, Dixon J, Curry N, Wennberg D, Darin B, Steinort K. Case finding algorithms for patients at risk of re-hospitalisation Part 1 and Part 2. London: King's Fund; 2006. Available from: www.kingsfund.org.uk/document.rm?id=6209.
15. Billings J, Dixon J, Mijanovich T, Wennberg D. Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients. BMJ 2006;333(7563):327.
16. Health Dialog, King's Fund, New York University. Combined predictive model. Final Report & technical documentation. London: King's Fund; 2006. Available from: www.kingsfund.org.uk/document.rm?id=6745.
17. Billings J, Mijanovich T. Improving the management of care for high-cost medicaid patients. Health Aff (CD Millwood) 2007;26(6):1643-54.
18. Delivering for Health Information Programme. SPARRA: Scottish patients at risk of readmission and admission. Edinburgh, Scotland: NHS Scotland, Information Services Division; 2006. Available from: www.isdscotland.org/isd/files/SPARRA_Report.pdf.
19. Peter T, Donnan, David W, Dorward T, Mutch B, Morris AD. Development and validation of a model for predicting emergency admissions over the next year (PEONY). Arch Intern Med 2008;168(13):1416-22.
20. Long Term Conditions Programme. Scottish patients at risk of readmission and admission. A report on development work to extend the algorithm's applicability to patients of all ages. Edinburgh, Scotland: NHS Scotland, Information Services Division; 2008. Available from: www.isdscotland.org/isd/servlet/FileBuffer?namedFile=2008_06_16_SPARRA_All_Ages_Report.pdf&pContentDispositionType=inline.
21. Jan De Lepeleire, Steve Iliffe, Eva Mann, Jean Marie Degryse. Frailty: an emerging concept for general practice. Br J Gen Practice May 2009.
22. Saluter.it. Emilia Romagna: Agenzia sociale e sanitaria regionale. Available from: https://siseps.regione.emilia-romagna.it/flussi/html/index.html.
23. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45(6):613-9.
24. Italia. Decreto legislativo 30 giugno 2003, n. 196. Codice in materia di protezione dei dati personali. Gazzetta Ufficiale n. 174 del 29 luglio 2003 (Suppl. ord. n. 123).
25. Shtatland, ES, Kleinman K, Cain EM. (2003). Stepwise methods in using SAS PROC LOGISTIC and SAS ENTERPRISE MINER for prediction. SUGI '28 Proceeding, Paper 258-28, Cary, NC: SAS Institute, Inc.
26. rier GW. Verification of forecasts expressed in terms of probability. Monthly Weat Rev 1950;78(1):1-3.
27. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009;338:b606.
28. American Nurses Association. Nursing case management. Kansas City, KS: American Nurses Association; 1988.
29. Sargent P, Boaden R, Roland M. How many patients can community matrons successfully case manage? J Nurs Manag 2008;16:38-46.
30. United Health Europe. Assessment of the Evercare Programme in England 2003-2004. Executive Summary. February 2005. Available from: www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4114121.
31. Sheaff R, Boaden R, Sargent P, Pickard S, Gravelle H, Parker S, Roland R. Impacts of case management for frail elderly people: a qualitative study. J Health Serv Res Policy 2009;14:88-95.
32. Purdy S, Griffin T. Reducing hospital admissions Guidance should be evidence based and take a holistic view of patient care. BMJ 2008;336:4-5.
33. Hutt R, Rosen R, McCauley J. Case-managing long-term conditions. What impact does it have in the treatment of older people? London: King's Fund; 2004.
34. Markle-Reid M, Browne G. Conceptualizations of frailty in relation to older adults. J Advanc Nursing 2003;44(1):58-68.
35. Lally F, Crome P. Understanding frailty. Postgrad Med J 2007;83:16-20.
36. Rockwood K, Andrew M, Mitnitski A. A comparison of two approaches to Measuring frailty in elderly people. J Gerontol 2007;62A:7.
37. Abellan van Kan G, Rolland Y, Bergman H, Morley JE, Kritchevsky SB, Vellas B. On behalf of the geriatric advisory panel. The I.A.N.A. Task Force on Frailty Assessment of older people. In clinical practice J Nutr Health Aging 2008;12(1).
38. Hart JT. The inverse care law. Lancet 1971;1:405-12.
39. Watt G. The inverse care law today. Lancet 2002;360:252-54.
40. Rubin D. Propensity score methods. Am J Ophtalmol 2010;January:7-9.
41. Rosenbaum P, Rubin D. The cental role of the propensity score in observational studies for casual effect. Biometrics 1983;79(1):41-55.
42. Lyon D, Lancaster GA, Taylor S, Dowrick C, Chellaswamy H. Predicting the likelihood of emergency admission to hospital of older people: development and validation of the Emergency Admission Risk Likelihood Index (EARLI). Family Practice 2007;24:158-167.
Address for correspondence:
Pasquale Falasca
Servizio Valutazione Strategica, Azienda USL di Ravenna
Via De Gasperi 8
48121 Ravenna, Italy
E-mail: p.falasca@ausl.ra.it,
Received on 26 October 2010.
Accepted on 24 March 2011.