SYSTEMATIC REVIEWS
Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review
Facteurs soutenant ou entravant l'implémentation de systèmes d'e-santé: une analyse systématique explicative
Factores que promueven o dificultan la implementación de los sistemas de telemedicina: revisión sistemática explicativa
Frances S MairI,*; Carl MayII; Catherine O'DonnellI; Tracy FinchIII; Frank SullivanIV; Elizabeth MurrayV
IInstitute of Health and WellBeing, University of Glasgow, 1 Horselethill Road, Glasgow, G12 9LX, Scotland
IIFaculty of Health Sciences, University of Southampton, Southampton, England
IIIInstitute of Health and Society, Newcastle University, Newcastle-upon-Tyne, England
IVPopulation Health Sciences, Medical Research Institute, University of Dundee, Dundee, Scotland
VResearch Department of Primary Care and Population Health, University College London, London, England
ABSTRACT
OBJECTIVE: To systematically review the literature on the implementation of e-health to identify: (1) barriers and facilitators to e-health implementation, and (2) outstanding gaps in research on the subject.
METHODS: MEDLINE, EMBASE, CINAHL, PSYCINFO and the Cochrane Library were searched for reviews published between 1 January 1995 and 17 March 2009. Studies had to be systematic reviews, narrative reviews, qualitative metasyntheses or meta-ethnographies of e-health implementation. Abstracts and papers were double screened and data were extracted on country of origin; e-health domain; publication date; aims and methods; databases searched; inclusion and exclusion criteria and number of papers included. Data were analysed qualitatively using normalization process theory as an explanatory coding framework.
FINDINGS: Inclusion criteria were met by 37 papers; 20 had been published between 1995 and 2007 and 17 between 2008 and 2009. Methodological quality was poor: 19 papers did not specify the inclusion and exclusion criteria and 13 did not indicate the precise number of articles screened. The use of normalization process theory as a conceptual framework revealed that relatively little attention was paid to: (1) work directed at making sense of e-health systems, specifying their purposes and benefits, establishing their value to users and planning their implementation; (2) factors promoting or inhibiting engagement and participation; (3) effects on roles and responsibilities; (4) risk management, and (5) ways in which implementation processes might be reconfigured by user-produced knowledge.
CONCLUSION: The published literature focused on organizational issues, neglecting the wider social framework that must be considered when introducing new technologies.
RÉSUMÉ
OBJECTIF: Analyser de façon systématique la documentation sur l'implémentation de l'e-santé afin d'identifier: (1) les éléments entravant et soutenant l'implémentation de l'e-santé, et (2) les lacunes majeures dans la recherche sur le sujet.
MÉTHODES: Des recherches ont été réalisées dans MEDLINE, EMBASE, CINAHL, PSYCINFO et la bibliothèque Cochrane afin de trouver les revues publiées entre le 1er janvier 1995 et le 17 mars 2009. Les études devaient consister en des revues systématiques, des revues narratives, des métasynthèses ou des méta-ethnographies qualitatives d'implémentation d'e-santé. Une double analyse fouillée a été effectuée parmi les résumés et les journaux, et les données ont été extraites par pays d'origine, domaine d'e-santé, date de publication, objectifs et méthodes, bases de données analysées, critères d'inclusion et d'exclusion et nombre de journaux inclus. Les données ont été analysées qualitativement au moyen d'une théorie du processus de normalisation comme cadre de codification explicatif.
RÉSULTATS: Les critères d'inclusion ont été respectés par 37 journaux, 20 avaient été publiés entre 1995 et 2007, et 17 entre 2008 et 2009. La qualité méthodologique était faible: 19 journaux ne spécifiaient pas les critères d'inclusion et d'exclusion, et 13 n'indiquaient pas le nombre exact d'articles analysés. L'utilisation de la théorie du processus de normalisation comme cadre conceptuel a permis de révéler le peu d'attention portée à: (1) le travail visant à justifier les systèmes d'e-santé, en spécifiant leurs objectifs et leurs avantages, en établissant leur valeur pour les utilisateurs et en planifiant leur implémentation; (2) les facteurs soutenant ou entravant l'engagement et la participation; (3) les effets sur les rôles et les responsabilités; (4) la gestion des risques, et (5) les moyens par lesquels les processus d'implémentation pourraient être reconfigurés par les connaissances des utilisateurs.
CONCLUSION: La documentation publiée se concentrait sur les problèmes organisationnels, négligeant le cadre social plus large qui doit être pris en compte en cas d'introduction de nouvelles technologies.
RESUMEN
OBJETIVO: Realizar una revisión sistemática de la literatura existente sobre la implementación de la telemedicina para identificar: (1) los obstáculos y estímulos para la implementación de la telemedicina, y (2) las lagunas pendientes en la investigación sobre el tema.
MÉTODOS: Se realizó una búsqueda en MEDLINE, EMBASE, CINAHL, PSYCINFO y la Cochrane Library de revisiones publicadas entre el 1 de enero de 1995 y el 17 de marzo de 2009. Los estudios debían ser revisiones sistemáticas, revisiones narrativas, metasíntesis cualitativas o metaetnografías sobre la implementación de la telemedicina. Los resúmenes y documentos se investigaron doblemente y se extrajeron los datos del país de origen, el área de la telemedicina, la fecha de publicación, los objetivos y métodos, las bases de datos buscadas, los criterios de inclusión y exclusión y el número de documentos. Se analizaron los datos de forma cualitativa mediante un proceso de normalización como marco de codificación explicativo.
RESULTADOS: De los documentos analizados, 37 cumplían con los criterios de inclusión, 20 habían sido publicados entre 1995 y 2007 y, 17, entre 2008 y 2009. La calidad metodológica fue escasa: 19 documentos no especificaban los criterios de inclusión y exclusión y 13 no indicaban el número concreto de artículos examinados. El uso de un proceso de normalización como marco conceptual reveló la poca atención que se había prestado a: (1) el trabajo dirigido a dotar de sentido a los sistemas de telemedicina, especificando sus objetivos y beneficios, estableciendo su valor para los usuarios y planificando su implementación; (2) los factores que promueven o dificultan el compromiso y la participación; (3) los efectos sobre las funciones y responsabilidades; (4) la gestión del riesgo, y (5) los modos en los que los procedimientos de implementación podrían reconfigurarse con los conocimientos generados por los usuarios.
CONCLUSIÓN: La literatura publicada se centró en cuestiones organizativas, descuidando el vasto marco social que debe tenerse en consideración a la hora de introducir nuevas tecnologías.
Introduction
Health-care providers have increasingly sought to utilize e-health systems that employ information and communications technologies to widen access, improve quality and increase service efficiency. Enthusiasm for technological innovation around e-health among policy-makers and health officials has, however, not always been matched by uptake and utilization in practice.1, 2 Professional resistance to new technologies is cited as a major barrier to progress, although evidence for such assertions is weak.3 Implementing and embedding new technologies of any kind involves complex processes of change at the micro level for professionals and patients and at the meso level for health-care organizations themselves. The European Union has recently argued that implementing e-health strategies "has almost everywhere proven to be much more complex and time-consuming than initially anticipated".4
Over the past decade the number of primary studies evaluating the practical implementation and integration of e-health systems has steadily grown. Sometimes these studies describe important successes, but more often they are accounts of complex processes with ambiguous outcomes. As the research community has sought to make sense of these studies, systematic reviews attempting to identify and describe "barriers" and "facilitators" to implementation have proliferated. Although the reviews have furthered knowledge by identifying factors thought to influence implementation processes and their outcomes, the underlying mechanisms at work have not been well characterized or explained. The literature is fragmented across multiple subspecialty areas, so those charged with designing and implementing e-health systems may find it difficult to locate an appropriate body of evidence and to determine the relevance of that evidence to their specific circumstances.
In this meta-review we have sought to address these problems in two ways. First, we have performed a systematic review of reviews of e-health implementation studies, focusing on implementation processes rather than outcomes, to critically appraise such reviews, evaluate their methods, synthesize their results and highlight their key messages. Our meta-review has enabled us to explore and evaluate a large and fragmented body of research in a coherent and economical way. Second, we have interpreted our results in the light of an explanatory framework -Normalization Process Theory (NPT) 5, 6 -that specifies mechanisms of importance in implementation processes. This approach has facilitated the explanation of those factors shown to influence the implementation of e-health systems in practice and allowed us to identify important gaps in the literature and to make rational recommendations for further primary research.
The objective of this review was to synthesize and summarize the findings of identified reviews and inform current and future e-health implementation programmes. The review set out to answer two key questions: (1) What does the published literature tell us about barriers and facilitators to e-health implementation? (2) What, if any, are the main research gaps?
Methods
Inclusion and exclusion criteria
Box 1 lists the inclusion and exclusion criteria. We used a previously developed method of categorization to classify e-health interventions into four domains: 7 management systems, communication systems, computerised decision support systems and information resources.
Finding relevant studies
We searched the following electronic bibliographic databases: MEDLINE; EMBASE; CINAHL; PSYCINFO; the Cochrane Library, including the Cochrane Database of Systematic reviews and the Cochrane Central Register of Controlled Trials; DARE; the National Health Service (NHS) Economic Evaluation Database, and the Health Technology Assessment Database. Box 2 describes our search terms.
We limited the MEDLINE database search to studies published in any language from 1990 to 2009. None of the non-English-language citations or of the papers published before 1995 were relevant. Hence, the MEDLINE search was re-run for only English-language reviews published between 1 January 1995 and 31 July 2009. These limits were used for searching all other databases. Each database's thesaurus terms were used to perform the search.
Data abstraction and analysis
Citations were downloaded into Reference Manager 11 (ISI ResearchSoft, Carlsbad, United States of America), and screened by two reviewers. If either reviewer could not exclude the paper based on the abstract or citation, the full paper was obtained. All papers obtained were double screened. In case of disagreement about inclusion or exclusion of a given paper, all reviewers read the paper and reached agreement through discussion.
Data were extracted in two stages. First we used a standardized data extraction instrument to categorize papers on the basis of country of origin; e-health domain; publisher; date of publication; review aims and methods; databases searched within the review; inclusion and exclusion criteria of review; number of papers identified and number included in the review.
Second, as the literature under study focused on implementation processes rather than outcomes, we analysed the extracted data qualitatively using NPT, which has four constructs (coherence, cognitive participation, collective action and reflexive monitoring) as a coding framework.5, 6 This theory provides a conceptual framework to explain the processes by which new health technologies and other complex interventions are routinely operationalized in everyday work (embedded) and sustained in practice (integrated).8, 9 Use of this framework to aid data analysis in systematic reviews of qualitative data has recently been described.10 For every paper two reviewers judged whether material relevant to the four constructs of NPT was present or absent, using the coding frame shown in Table 1. As this was a qualitative content analysis,11 we did not try to quantify the weight put on any one NPT construct in a given review. Each statement in a paper relating to findings regarding barriers or facilitators to e-health implementation was treated as an "attributive statement"; two reviewers coded these statements to the relevant construct of the NPT. If an "attributive statement" could not be coded to the NPT framework, this was stated to ensure that issues outside the scope of the theory would still be captured. Dual coding enabled differences in coding and interpretation to be identified and discussed. Disagreement, which was minimal, was resolved through discussion. If any areas of disagreement remained, a final reviewer served as arbiter.
The methods used for study identification and data collection in this study were in keeping with the recent PRISMA statement.12 However, as this was a review of process rather than outcome studies, some aspects of the PRISMA statement were not applicable.
Results
From 8206 unique citations screened, we excluded 7973 on the basis of the title or abstract and retrieved 233 full-text articles. Of these, 37 met the inclusion criteria ( Fig. 1 ). Of note, 20 of these reviews were published between 1995 and 2007 and 17 were published in the following two years.
Of the 37 included reviews, 18 originated in the United States of America, 10 in Canada, 3 in the United Kingdom of Great Britain and Northern Ireland, 2 in the Netherlands and 1 each in Australia, Germany, Malaysia and Norway.13 - 49 Reviews generally covered one or more e-health domains: eight reported on management systems; 10 on communication systems; 6 on decision support; 1 mainly on information systems, and 12 on combinations of these. Full details of included papers are available from the authors.
When judged against the PRISMA checklist,12 many of the included reviews were methodologically poor. For example, 5 of the 37 reviews did not clearly describe the databases searched.22, 25, 32, 36, 49 Information on search strategies was often rudimentary. Of the 37 included reviews, 7 searched only one or two databases or sources, such as the proceedings of a particular conference.14, 15, 28, 38, 42, 44, 48 Information about study selection criteria was also inadequate: 19 of the 37 reviews did not specify the criteria for inclusion or exclusion.15, 16, 18, 21 - 23, 25, 26, 29, 31 - 36, 41 - 44 For 13 of the 37 papers reviewed it was impossible to ascertain precisely how many studies had been included.17, 18, 22, 23, 25, 26, 2 8, 29, 32, 35, 38, 41, 42 .
As the reviews under investigation dealt with organizational and other processes rather than with numeric outcome measures, the PRISMA checklists for summary measures and result synthesis were not applicable. NPT was used to aid analysis and conceptualization of the qualitative data regarding barriers and facilitators of implementation.
Content analysis of the 37 reviews identified 801 attributive statements about implementation processes that could be interpreted using NPT as an explanatory framework.
Coherence
Coherence refers to the "sense-making" work undertaken when a new e-health service is implemented (e.g. to determine whether users see it as differing from existing practice, have a shared view of its purpose, understand how it will affect them personally and grasp its potential benefits). A surprising and important result of this review was the discovery that work directed at making sense of e-health systems received very little coverage. Content analysis of the reviews showed it to be the focus of less than 12% (95/801) of the attributive statements. Coherence work was seen to be concerned with preparatory activities -often policy building or dissemination of information -undertaken either locally or nationally.
Since "sense-making" work is an important aspect of implementation, this review reveals an important gap in the literature. We do not know whether the gap reflects the exclusion of coherence work from process-oriented studies (which is possible if evaluations commence in the delivery phase of a project) or the systematic absence of sense-making work itself.
Cognitive participation
Cognitive participation focuses upon the work undertaken to engage with potential users and get them to "buy into" the new e-health system. Although this work of relating and engaging with users is central to the successful implementation of any new technology, work aimed at actively involving health professionals in e-health services rarely figured in the reviews we examined. The same was true of work leading to the initiation and legitimation of health technologies or geared towards sustaining them in practice. Less than 11% (88/801) of attributive statements fell within this category, and those that did took the form of general recommendations rather than specific design and delivery considerations. For example, Hilty 23 proposes ways to encourage health professional participation, including "incentives for each of the parties involved".
Other issues that fell within the cognitive participation category included a range of actions to legitimize participation in the implementation process and promote it as a worthwhile activity, such as the recruitment of local "champions". Such champions were seen as having the ability to promote utilization of new e-health services by more reticent colleagues. However, this approach could be a double-edged sword. Health professionals who enthusiastically support e-health can get colleagues to enrol in and commit to e-health programmes. However, those who project a negative attitude can jeopardize the staff commitment needed to make an e-health system work and thus impede implementation.
Since participation and engagement are vital for the success of new technologies, the lack of coverage in the reviews of the factors that promote or inhibit user engagement and participation is clearly a major weakness in the literature.
Collective action
Work involved in implementing or enacting e-health systems was the topic of 65% (518/801) of the attributive statements identified. The emphasis in this domain was on the work performed by individuals, groups of professionals or organizations in operationalizing a new technology in practice. We found that the research emphasis changed over time. Up to 2007 the emphasis lay on organizational issues, but after that year it shifted towards socio-technical issues (e.g. how e-health systems affected the everyday work of individuals).
Addressing organizational issues
Most reviews focused on the ways in which the e-health innovation affected organizational structures and goals. This was especially true up to 2007, when 35% (142/411) of the attributive statements identified focused on this point, as opposed to only 20% (77/390) after 2007. The literature highlighted the need for adequate resources, particularly financial. Administrative support, policy support, standards and interoperability also fell within this research category.
This area's emphasis is on the contextual integration of e-health systems, particularly on the extent to which they are managed and resourced. The focus on management is not surprising, but the emphasis on a "top down" approach draws attention away from other equally important aspects of collective action.
Effects on health care tasks
The interactional workability of e-health systems accounted for less than 18% of the attributive statements identified during content analysis. Many of these focused on the "ease of use" of the new systems for clinicians, with the underlying assumption that clinicians would be deterred from or resistant to using systems that added complexity or required additional effort or time.
Ease of use for patients or other service users (or even health professionals besides clinicians, such as nurses) did not figure prominently in the reviews we investigated. However, the effect of e-health systems on physician-patient interaction did receive some attention, as exemplified by the following quote:
"... an effective clinical decision support system must minimise the effort required by clinicians to receive and act on system recommendations".30
Thus, implementation may be retarded or destabilized by the competing priorities of powerful participants.
Confidence and accountability
The relational integration of e-health systems (confidence, security and accountability) accounted for 15% (116/801) of the attributive statements identified. Such concerns could act as either facilitators or barriers. Users may see in e-health technologies a way to reduce errors, which would encourage uptake; alternatively, security and safety concerns could undermine confidence in e-health systems and hinder their widespread utilization.
Roles, responsibilities and training
Roles, responsibilities and training or support issues accounted for only about 10% (77/801) of attributive statements. Most emphasis was placed on the need to adequately train staff members for engagement in implementation, although division of labour was also a concern, as were effects on workload. However, these issues were often discussed superficially, without examining the types of training or ongoing support that would be required.
Reflexive monitoring
While the majority of attributive statements identified in the content analysis dealt with managerial interventions and controls, much less information was provided on the ways in which managers and other users appraise whether an e-health intervention is worthwhile or not. Only 13% (104/801) of the attributive statements fell within this category. Most of these dealt with evaluation and monitoring and how they are used to influence utilization and future e-health implementations. However, evaluation was also promoted as necessary to ensure that safety concerns were addressed.
Evaluation could, of course, either allay concerns or confirm the need for amendments to the e-health service being implemented. There was little evidence of local appraisals or of the ways in which implementation processes might be reconfigured by user-produced knowledge.
Only 6% of issues fell outside our coding framework, either because they were strictly technical and attitudinal or because they were so generic and vague, without accompanying contextual data, that it was not possible to determine whether the concept really lay outside the model or was simply too general to be coded.
Discussion
A thorough and systematic search of reviews published over the preceding 15 years identified 37 reviews on the implementation of e-health technologies in health-care settings. Most were from North America. Methodological deficiencies were common and the findings should be interpreted with caution. The number of publications has risen rapidly since 2008, which suggests that there is growing awareness of the need to understand and address issues related to implementation if e-health services are to become a core component of routine service delivery.
This review breaks new ground. It not only collates and summarizes data but also analyses it and interprets it within a theoretical framework. Our approach has allowed us to explore the factors that facilitate and hinder implementation, identify gaps in the literature and highlight directions for future research. In particular, this work highlights a continued focus on organizational issues, which, despite their importance, are only one among a range of factors that need to be considered when implementing e-health systems.7 - 9, 50
Although our meta-review was rigorous and carefully executed and employed a robust conceptual framework, it has limitations. Since not all primary research has been captured by previous reviews, our meta-review does not include findings from all studies in this field. Furthermore, review data is two steps removed from primary data, and the quality of the primary research may not be properly assessed in reviews of substandard quality. Finally, since the reviews we identified were of poor quality on average and their search strategies were not always comprehensive, their findings may be biased.
Conclusion
Our review has revealed a growing emphasis on problems related to e-health systems' workability but relatively little attention to: (1) e-health's effects on roles and responsibilities; (2) risk management; (3) ways to engage with professionals; and (4) ensuring that the potential benefits of new technologies are made transparent through ongoing evaluation and feedback. These areas deserve more empirical investigation, as do ways to identify and anticipate how e-health services will impact everyday clinical practice. This involves examining how new e-health services will affect clinical interactions and activities and the allocation and performance of clinical work. Also in need of investigation are the effects of different methods of engaging with professionals before and during the implementation of e-health services. 
References
1. May CR, Finch TL, Cornford J, Exley C, Gately C, Kirk S et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res 2011;11:131. doi: 10.1186/1472-6963-11-131 PMID:21619596
2. Jha AK, DesRoches C, Campbell EG, Donelan K, Rao SR, Ferris TG et al. Use of electronic health records in US hospitals. N Engl J Med 2009;360:1628-38. doi: 10.1056/NEJMsa0900592 PMID:19321858
3. Balfour DC, Evans S, Januska J, Lee HY, Lewis SJ, Nolan SR et al. Health information technology - results from a roundtable discussion. J Manag Care Pharm 2009;15:10-7. PMID:19125556
4. Watson R. European Union leads way on e-health but obstacles remain. BMJ 2010;341:c5195. doi: 10.1136/bmj.c5195 PMID:20858645
5. May C, Finch T. Implementation, embedding, and integration: an outline of normalization process theory. Sociology 2009;43:535-54. doi: 10.1177/0038038509103208
6. May CR, Mair FS, Finch T, Macfarlane A, Dowrick C, Treweek S et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci 2009;4:29. doi: 10.1186/1748-5908-4-29 PMID:19460163
7. Mair FS, May C, Murray E, Finch T, Anderson G, O'Donnell C, et al. Understanding the implementation and integration of e-Health services. Report for the NHS Service and Delivery R & D Organisation (NCCSDO). London: SDO; 2009.
8. Elwyn G, Legare F, Edwards A, van der Weijden T, May C. Arduous implementation: does the normalisation process model explain why it is so difficult to embed decision support technologies for patients in routine clinical practice. Implement Sci 2008;3:57. doi: 10.1186/1748-5908-3-57 PMID:19117509
9. Murray E, Burns J, May C, Finch T, O'Donnell C, Wallace P et al. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci 2011;6:6. doi: 10.1186/1748-5908-6-6 PMID:21244714
10. Normalization Process Theory [Internet]. May C, Murray E, Finch T, Mair F, Treweek S, Ballini L et al. Normalization process theory on-line users' manual and toolkit. 2010. Available from: http://www.normalizationprocess.org [accessed 27 March 2012]
11. Ritchie J, Lewis J. Qualitative research practice: a guide for social science students and researchers. London: Sage Publications; 2003.
12. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. BMJ 2009;339:b2700. doi: 10.1136/bmj.b2700 PMID:19622552
13. Adaji A, Schattner P, Jones K. The use of information technology to enhance diabetes management in primary care: a literature review. Inform Prim Care 2008;16:229-37. PMID:19094410
14. Botsis T, Hartvigsen G. Current status and future perspectives in telecare for elderly people suffering from chronic diseases. J Telemed Telecare 2008;14:195-203. doi: 10.1258/jtt.2008.070905 PMID:18534954
15. Broens TH, Huis in't Veld RM, Vollenbroek-Hutten MM, Hermens HJ, van Halteren AT, Nieuwenhuis LJ. Determinants of successful telemedicine Implementations: a literature study. J Telemed Telecare 2007;13:303-9. doi: 10.1258/135763307781644951 PMID:17785027
16. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical Care. Ann Intern Med 2006;144:742-52. PMID:16702590
17. Copen J, Richards P. Systematic review on PDA clinical application implementation and lessons learned. J Inf Technol Healthc 2008;6:114-28.
18. Demaerschalk BM, Miley ML, Kiernan TEJ, Bobrow BJ, Corday DA, Wellik KE et al. Stroke telemedicine. Mayo Clin Proc 2009;84:53-64. doi: 10.4065/84.1.53 PMID:19121244
19. Fitzpatrick LA, Melnikas AJ, Weathers M, Kachnowski SW. Understanding communication capacity: communication patterns and ICT usage in clinical settings. J Healthc Inf Manag 2008;22:34-41. PMID:19267030
20. Gagnon MP, Legare F, Labrecque M, Fremont P, Pluye P, Gagnon J et al. Interventions for promoting information and communication technologies adoption in healthcare professionals. Cochrane Database Syst Rev 2009;1:CD006093. PMID:19160265
21. Gruber D, Cummings GG, LeBlanc L, Smith DL. Factors influencing outcomes of clinical information systems implementation: a systematic review. CIN 2009;27:151-63. PMID:19411944
22. Hebert MA. Moving research into practice: a decision framework for integrating home telehealth into chronic illness care. Int J Med Inform 2006;75:786-94. doi: 10.1016/j.ijmedinf.2006.05.041 PMID:16872892
23. Hilty DM. Telepsychiatry: an overview for psychiatrists. CNS Drugs 2002;16:527-48. doi: 10.2165/00023210-200216080-00003 PMID:12096934
24. Jarvis-Selinger S, Chan E, Payne R, Plohman K, Ho K. Clinical telehealth across the disciplines: lessons learned. Telemed J E Health 2008;14:720-5. doi: 10.1089/tmj.2007.0108 PMID:18817503
25. Jennett PA, Scott RE, Affleck Hall L, Hailey D, Ohinmaa A, Anderson C et al. Policy implications associated with the socioeconomic and health system impact of telehealth: a case study from Canada. Telemed J E Health 2004;10:77-83. doi: 10.1089/153056204773644616 PMID:15104919
26. Jennett PA. Preparing for success: readiness models for rural telehealth. J Postgrad Med 2005;51:279-85. PMID:16388170
27. Jimison H, Gorman P, Woods S, Nygren P, Walker M, Norris S et al. Barriers and drivers of health information technology use for the elderly, chronically ill and underserved. Evid Rep Technol Assess (Full Rep) 2008;175:1-1422. PMID:19408968
28. Johnson KB. Barriers that impede the adoption of pediatric information technology. Arch Pediatr Adolesc Med 2001;155:1374-9. PMID:11732959
29. Jones R. The role of health Kiosk in 2009: literature and informant review. Int J Environ Res Public Health 2009;6:1818-55. doi: 10.3390/ijerph6061818 PMID:19578463
30. Kawamoto K. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330:765-8. doi: 10.1136/bmj.38398.500764.8F PMID:15767266
31. Kukafka R. Grounding a new information technology implementation framework in behavioral science: a systematic analysis of the literature on IT use. J Biomed Inform 2003;36:218-27. doi: 10.1016/j.jbi.2003.09.002 PMID:14615230
32. Leatt P. IT solutions for patient safety - best practices for successful implementation in healthcare. Healthc Q 2006;9:94-104. PMID:16548440
33. Lu YC, Xiao Y, Sears A, Jacko JA. A review and a framework of handheld computer adoption in healthcare. Int J Med Inform 2005;74:409-22. doi: 10.1016/j.ijmedinf.2005.03.001 PMID:15893264
34. Ludwick DA, Doucette J. Adopting electronic medical records in primary care: lessons learned from health information systems implementation experience in seven countries. Int J Med Inform 2009;78:22-31. doi: 10.1016/j.ijmedinf.2008.06.005 PMID:18644745
35. Mack EH, Wheeler DS, Embi PJ. Clinical decision support systems in the pediatric intensive care unit. Pediatr Crit Care Med 2009;10:23-8. doi: 10.1097/PCC.0b013e3181936b23 PMID:19057443
36. Mashima PA, Doarn CR. Overview of telehealth activities in speech-language pathology. Telemed J E Health 2008;14:1101-17. doi: 10.1089/tmj.2008.0080 PMID:19119834
37. Mollon B, Chong J Jr, Holbrook AM, Sung M, Thabane L, Foster G. Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials. BMC Med Inform Decis Mak 2009;9:11. doi: 10.1186/1472-6947-9-11 PMID:19210782
38. Ohinmaa A. What lessons can be learned from telemedicine programmes in other countries? J Telemed Telecare 2006;12:S40-4. doi: 10.1258/135763306778393135 PMID:16989673
39. Oroviogoicoechea C, Elliott B, Watson R. Review: evaluating information systems in nursing. J Clin Nurs 2008;17:567-75. doi: 10.1111/j.1365-2702.2007.01985.x PMID:18093116
40. Orwat C, Graefe A, Faulwasser T. Towards pervasive computing in health care - a literature review. BMC Med Inform Decis Mak 2008;8:26. doi: 10.1186/1472-6947-8-26 PMID:18565221
41. Papshev D. Electronic prescribing in ambulatory practice: promises, pitfalls, and potential solutions. Am J Manag Care 2001;7:725-36. PMID:11464430
42. Peleg M. Decision support, knowledge representation and management in medicine. Yearb Med Inform 2006;45:72-80. PMID:17051298
43. Shekelle PG, Morton SC, Keeler EB. Costs and benefits of health information technology. Evid Rep Technol Assess (Full Rep) 2006;132:1-171. PMID:17627328
44. Studer M. The effect of organizational factors on the effectiveness of EMR system implementation: What have we learned? Healthc Q 2005;8:92-8. PMID:16323520
45. van Rosse F, Maat B, Rademaker CM, van Vught AJ, Egberts AC, Bollen CW. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009;123:1184-90. doi: 10.1542/peds.2008-1494 PMID:19336379
46. Vreeman DJ, Taggard SL, Rhine MD, Worrell TW. Evidence for electronic health record systems in physical therapy. Including commentary by Zimny NJ with author response. Phys Ther 2006;86:434-46. PMID:16506879
47. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009;39:705-12. doi: 10.1017/S0033291708004224 PMID:18812006
48. Yarbrough AK. Technology acceptance among physicians: a new take on TAM. Med Care Res Rev 2007;64:650-72. doi: 10.1177/1077558707305942 PMID:17717378
49. Yusof MM, Stergioulas L, Zugic J. Health information systems adoption: findings from a systematic review. Stud Health Technol Inform 2007;129:262-6. PMID:17911719
50. May C, Gask L, Atkinson T, Ellis N, Mair F, Esmail A. Resisting and promoting new technologies in clinical practice: the case of telepsychiatry. Soc Sci Med 2001;52:1889-901. doi: 10.1016/S0277-9536(00)00305-1 PMID:11352414
Submitted: 19 November 2011
Revised version received: 17 March 2012
Accepted: 20 March 2012
Funding: We thank the Service and Delivery Organisation (SDO) for funding the study via project grant 08/1602/135. This article presents independent research commissioned by the National Institute for Health Research (NIHR) SDO programme. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The NIHR SDO programme is funded by the Department of Health, United Kingdom. No funding bodies played any role in the design, writing or decision to publish this manuscript.
Competing interests: CM and TF led on developing normalization process theory, and all authors have made important contributions to its development.
* Correspondence to: Frances S Mair (e-mail: frances.mair@glasgow.ac.uk).