ORIGINAL ARTICLES AND REVIEWS
Nurses' reporting of suspect adverse drug reactions: a mixed-methods study
Alessia De AngelisI; Angela GiustiII; Sofia ColaceciI; Ercole VelloneI; Rosaria AlvaroI
IDipartimento di Biomedicina e Prevenzione, Università degli Studi di Roma Tor Vergata, Rome, Italy
IICentro Nazionale di Epidemiologia, Sorveglianza e Promozione della Salute, Istituto Superiore di Sanità, Rome, Italy
ABSTRACT
OBJECTIVE: To assess nurses' knowledge, attitudes and practices (KAP) towards spontaneous adverse drug reactions (ADRs) reporting.
METHODS: The mixed-method study was conducted following a quanti-qualitative sequential approach: a survey (using a KAP questionnaire) followed by a focus group was performed.
RESULTS: In the quantitative findings, responders (570 hospital nurses) declared that they were unaware of the pharmacovigilance system (58.1%, n = 331); where to find the reporting form (63.5%, n = 362); how fill it in (71.6%, n = 408); to whom and how to send it (65.8%, n = 375). Only 11.1% (n = 63) reported ADRs. The qualitative phase supported the quantitative findings and provided new information about other factors that condition ADR reporting: misinterpretation of the meaning of "reporting", unawareness of nurses' autonomy in ADR reporting and fear of consequences after ADR reporting.
CONCLUSION: Nurses are not fully aware of their role in ADR reporting. We recommend educational interventions and management changes.
Key words: nurses, pharmacovigilance, adverse drug reaction reporting systems, knowledge, practices
INTRODUCTION
Subsequent to the disaster that occurred in the nineteen-sixties with thalidomide, drug regulation systems were developed in many countries to monitor adverse drug reactions (ADR) [1]. Improving a good system for drug safety is an important indicator of social progress and a duty of every government [2].
In this contest, pharmacovigilance was founded as "the science and activity relating to the detection, assessment, understanding and prevention of adverse effects or any other possible drug-related problem" [1]. After the medication has been approved, during the post-marketing phase, spontaneous ADR reporting is a cornerstone for evaluating and monitoring the drugs benefit-risk profile [3]. Following the analysis of the reports and the causality assessment, the regulatory authorities of pharmacovigilance may decide to change the labelling or remove the drug from the market [4]. ADR is defined by the World Health Organization (WHO) as "a response to a drug which is noxious and unintended, and which occurs at doses normally used in man for the prophylaxis, diagnosis, or therapy of disease, or for the modification of physiological function" [1].
Since July 2012 the new European Union pharmacovigilance legislation [5, 6] introduced important changes aimed to a better prevention, detection and assessment of ADRs. Also the definition of ADR has been updated, including that any reaction should be reported also in cases of medication errors, overdose, off-label use [5]. In the European Union, these changes were due to new available data regarding the impact of ADRs on public health, as they are the fifth leading cause of death in hospitals. Specifically, ADRs are 5% of all hospital admittances, are responsible for about 197 000 deaths per year, with a social cost assessed at 79 billion euro [7]. Also, ADRs have a cost in the United States of some 30 billion dollar each year [8] and are the cause of 10% (68/678) of hospitalisations in US Veterans' Affairs Medical Centres [9].
A major issue to detect new and potential ADRs is under-reporting among healthcare providers. A systematic review has shown that only 6% of all ADRs are reported [10]. Under-reporting among physicians could be explained by some attitudes, defined as "complacency", "fear", "indifference", "ambition", "ignorance", "diffidence", and "lethargy" [11]. Partially in agreement, a recent systematic review has identified other attitude among healthcare providers such as "insecurity", described as the opinion that it is nearly impossible to determine causality of ADRs [12]. A qualitative study has evaluated four potential barriers to the spontaneous reporting among hospital physicians: obstacles related to diagnosis and suspicion of ADRs, to clinical activities (i.e. lack of time, difficulties in filling out records), to organizing a hospital pharmacovigilance system and to potential conflicts (i.e. confidentiality with patients' data, legal liability) [13]. However, the available scientific literature on the under-reporting phenomena mainly derives from investigations on physicians [11-13].
The contribution of all healthcare providers is a key to good post market surveillance [5]. In particular, nurses are in a unique position to monitor the patient's responses to drugs and, consequently, if necessary, report potential ADRs [14-16]. For the development of clinical nursing expertise, more on the job experience could be required [17]. In particular, a recently study has shown that newly graduated nurses had a lack of pharmacological knowledge and skills to detect adverse drug effects [18]. Improving one's own competence in ADRs reporting is a nurse's duty. The Nursing and Midwifery Council includes adverse drug monitoring and reporting among the safety standards in the nursing management of medications [19]. Two studies [20, 21] showed that nurses can contribute quantitatively and qualitatively in ADRs reporting and consequently, increase the safety related to the use of drugs. In fact, no difference in quality and quantity emerges when comparing nurses' and physicians' ADR reports [22]. However the nurse's contribution on ADR reporting is still marginal in some Countries. Among spontaneous reports of suspected ADRs received by the Central Portugal Regional Pharmacovigilance Unit from January 2001 to October 2011, it is estimated that only 0.55% of nurses employed reported [23]. In Sweden, where some nurses have a specialized education on prescribing a limited selection of drugs, the number of ADR reported by nurses increased from 2-3% in 1995 to 12% in 2004 of total reporting [24].
An analysis of pharmacovigilance Italian databases showed that 74657 ADRs were received from different categories of reporters, with only 1951 reports (approximately 2.6%) from nurses in the years 2004-2010 [25]. Yet, Italian nurses are expected to report an ADR through the pharmacovigilance Italian system since 2003 [26]. The reasons of under-reporting among Italian nurses have not been sufficiently explored and only one Italian study has described low skill of the hospital nurse staff in detecting ADRs. According to the authors, this result is probably due to lack of knowledge on pharmacology and ADR [27].
Therefore, the aims of the present study was to describe nurses' knowledge, attitudes and practices towards spontaneous ADRs reporting.
MATERIAL AND METHODS
Study design
This was a mixed-methods study and was conducted in line with the six-step guide for the WHO survey [28]. A multiphase design was used. This approach is used when researchers "examines a problem or topic through an iteration of connected quantitative and qualitative studies that are aligned sequentially, with each new approach building on what was learned previously to address a central program objective" [29]. Thus, the study is built on 2 sequential quanti-qualitative phases: a descriptive study that is followed by an explanatory focus group (Figure 1).
Phase 1: quantitative study
Instrument
A questionnaire for knowledge, attitudes and practices of ADRs (KAP questionnaire) was developed by the research team and was based on an existing instrument aimed at General Practitioners [30]. It consists of three sections and a section with socio demographic and professional data.
The knowledge section of the KAP questionnaire is composed of 11 multi-choice questions and measures the theoretical knowledge about the pharmacovigilance in general and the reporting of suspected ADRs. An example of item is: "the WHO definition of ADR", "the aims of pharmacovigilance", "the limitations of pre-market studies on drugs". Each item is composed of a statement or question with three possible response options, of which only one is correct ("0- wrong response"; "1- correct response"). Correct responses are summed up and converted into an 11-point score.
The attitudes section of the KAP questionnaire contains 6 items aimed to measure attitudes towards under-reporting of suspect ADRs. The items are scored on a 5-point Likert scale ranging from "1-Strongly disagree" to "5-Strongly agree". An example of item is: "the ADR reporting is a bureaucratic process", "I do not have time to report ADR during my work", "the report of ADR is an exclusive duty of the physician". Lower scores indicated more positive attitude toward ADRs reporting.
The practices section of the KAP questionnaire is composed of 7 questions and measures procedural knowledge and reporting practice. An example of item is: "I know where to find the ADR reporting form", "I know how to fill and submit it", "I reported an ADR in the last year". Each item is dichotomous with two possible responses: "0-No" or "1-Yes". Higher score indicated a higher level of potential (procedural knowledge) or actual (number of reporting) practice in the reporting of suspected ADRs.
The socio-demographic section of the KAP questionnaire is used to collect variables such as gender, age, years of work experience, role, department affiliation. An additional question investigates the professional interest for pharmacovigilance training.
Before administration, the KAP questionnaire underwent validity and reliability testing. Initially, a group of pharmacoepidemiologists established its face-validity. Then the KAP underwent content validity with a group of nurses, further establishing its face-validity in nursing contests. Later, the KAP was administered to 22 nurses to estimate test-retest reliability. Test-retest reliability estimated with the interclass correlation coefficient (ICC) with a 2 week interval, was supportive with the following indices: ICC = 0.95, IC 95% 0.90-0.98 for the knowledge section; ICC = 0.99, IC 95% 0.98-0.99 for the attitudes section; ICC = 0.92, IC 95% 0.82-0.92 for practice section.
Post-administration, the knowledge section of the KAP questionnaire underwent content validity using item analysis [31]. Then, the reliability of attitudes section of the KAP questionnaire items was estimated with Cronbach' alpha, while for the reliability of practices items Kuder-Richardson-20' was used [32].
The item analysis indexes were: difficulty index (DI), discrimination power (DP) and distracters effectiveness (DE), and have been calculated using the software SITA (System for ITem Analysis) for the analysis of multiple choice tests and learning assessment [31]. As for the knowledge items, the DI showed a range between 3087% (mean = 62%) indicating a good balance between facility and difficulty on each item. The DP showed a value between 0.3-0.7 indicating a good consistency of each item with respects to the total item result. The DE showed a value ≠ 0 indicating good effectiveness of each distracter in the multiple choice tests.
Cronbach's alpha was 0.70 for the attitudes section, while Kuder-Richardson-20' coefficient was 0.82 for practices section.
Procedures of data collection
The KAP was anonymously self-administered to all nurses employed in five public hospitals in Rome and its province. The only exclusion criterion adopted in the study was the prolonged (> 2 weeks) leave of absence from work. The respondents participated on a voluntary basis during their work and gave the completed questionnaire back after about a week from enrollment.
Data analysis
All analyses were conducted using IBM software Statistical Package for Social Sciences (SPSS) version 15.0 for Windows (SPSS Inc. Chicago, Illinois, USA).
A descriptive analysis was performed using frequencies, mean and standard deviation for quantitative variables.
Phase 2: qualitative study
The qualitative part of the study was conducted with a focus group. The main objective was to explain the discrepancies between the declared and the actual practice of ADR reporting. The secondary objective was to explore the reasons of under-reporting and to identify strategies to improve ADR's reporting according to the participant's point of view.
The research hypothesis was focused on a possible misinterpretation of the meaning of "ADR reporting". ADR reporting is a structured process where a citizen or a health professional recognizes a suspect ADR, find and fill in the reporting form and send it to the Pharmacovigilance System. Data emerging from the quantitative study suggest that nurses might declare to "have reported suspect ADRs" when they simply "report" it to the doctor, to the nurse manager or in the patient's or other clinical records.
Instrument
Data was collected with a focus group performed according to the needs emerging from the descriptive study.
Procedures of data collection
A theoretical sample of nurses was involved in a focus group. We selected the hospital where nurses declared to have reported ADRs but no records were found in the pharmacovigilance system. Nurses have been recruited in the hospital units where ADRs reporting is expected to be higher. The audio-recorded focus group has been facilitated by and experienced researcher using pre-defined semi-structured questions. The objectives and methodology of data collection and data treatment have been explained by the facilitator at the beginning of the meeting and all participants signed a consent form.
Data analysis
Data has been analyzed with NVivo 9.0. The audio recording have been listened, re-listened and coded deductively, according to the research objectives, and inductively, seeking new or unexpected contents.
RESULTS
Quantitative study
Sample characteristic
The sample was composed of 570 hospital nurses and head nurses (response rate 73.8%). Participant ranged in age from 22-60 years (mean = 37.68; SD = 8.72). Most nurses were women and working in emergency departments. Their professional experience ranged from less than one year to 40 years (mean = 13.07; SD = 9.23). 87.4% (n = 498) would be interested in participating in training program on Pharmacovigilance and ADRs reporting. The socio-demographic characteristic are summarized in Table 1.
Nurses' knowledge, attitudes and practices
Knowledge score was calculated on a scale of 0-11 (mean = 6.56, SD = 2.25, min = 0, max = 11). Attitude score was calculated on a scale of 6-30 (mean = 12.74; SD=3.91; min = 6, max = 26). As shown in Table 2, only 20% (n = 113) of respondents considers that "ADR reporting is a bureaucratic process", 17.7% (n = 100) thinks that they "do not have time to report ADR" and 3.6% (n = 20) that "the data collected through the reporting system have little utility". Only 9.7% (n = 55) thinks that "the reporting could be used for legal proceedings against the reporter", 5.1% (n = 29) that "my report of ADR does not make any difference", 14.7% (n = 83) that "the report of ADR is an exclusive duty of the physician".
Practice score was calculated on a scale of 0-7 (mean = 1.76; SD = 2.05; min = 0, max = 7). Responders declared that they were unaware of: the system of pharmacovigilance (58.1%, n = 331); where to find the reporting form (63.5%, n = 362); how fill it in (71.6%, n = 408); to whom and how to send the reporting form (65.8%, n = 375); who is in charge of pharmacovigilance in their hospital (75.6%, n = 431). Finally, only 7.2% (n = 41) reported a suspect ADR in the last year and 11.1% (n = 63) reported at least 3 suspect ADRs during their entire professional practice.
Qualitative study
An explanatory focus group has been organized in a hospital where nurses declared to have reported ADRs but no reports were found on the pharmacovigilance database.
Fourteen nurses participated in the focus group, coming from emergency care, radiology, surgery, nephrology, cardiology, neonatal intensive care. One nurse was working at the Hospital Management. The mean age was 42 years (range 31-57), 10/14 were female nurses, 9/14 had a post-bachelor degree in nursing management. The mean of professional experience was 17.7 years (range 7-31). According to the answer to the data collection form, 12/14 declared to know the pharmacovigilance system and 5/12 to have reported at least 1 ADR during their nursing career.
The main reasons for under-reporting in the participant's perception are lack of knowledge of the pharmacovigilance system, unawareness of the nurse's role within the pharmacovigilane system ("there is a lack of culture of ADRs reporting") and fear of consequences, both as legal consequences and conflicts with doctors. The question focusing the meaning of "ADR reporting" revealed a spread misinterpretation. The nurses considered they were reporting suspect ADR's while, in fact, they were merely informing the physician or the head nurses, or registering the suspected ADR in the patient's clinical records. The participants expressed the need to improve the accessibility to the ADR reporting form through the intranet system and to provide specific communication or training interventions to improve their competencies in ADR reporting.
DISCUSSION
The quantitative and qualitative results explain the various aspects that relates to the spontaneous ADRs reporting among nurses. The results highlight the need to promote both knowledge and attitudes but along with organizational changes. Similarly, another study has indicated that the factors that condition the ADR reporting for the medical profession was both the intrinsic aspects related with the doctors' knowledge and attitude, and the extrinsic aspects that include all those factors associated to the interaction with their work environment [33]. According to our findings, under-reporting of ADRs among nurses could be mainly due to lack of knowledge and to heath care settings that do not support them in this activity.
The need to participate in specific training courses is strongly perceived among nurses to develop their knowledge and competencies in ADRs reporting, as emerging both in quantitative and qualitative findings. Previous studies have shown the importance of training to improve attitudes [20] and increase the number of reports of suspected ADRs among nurses [16, 20, 27]. Recently, a critical and systematic review on the strategies to promote ADRs reporting indicated that 87% was educational interventions such as presentations, report reviews, Problem-Based Learning and clinical cases [34]. This means that training courses or other educational interventions should address, apart from the classical model of the learning to do, to be and to learn, the dimensions of the "know why", as a lever for motivation and positive attitudes, changing routine practices into meaningful experiences. Even the systematic review conducted by Lopez-Gonzalez et al. [12] indicates that the attitudes and knowledge influence the under-reporting among healthcare professionals more than their personal characteristics.
The procedural aspects of ADRs reporting system (who is the person in charge of pharmacovigilance in the hospital, where to find the reporting form, how fill it in, to whom and how to send the reporting form) were mainly unknown by the responders. In accordance with our results, a survey conducted in UK indicates that less than 50% of health care providers are aware of the national reporting system [35]. In other studies, some of the emerging barriers to nurses ADR reporting are the unavailability of the reporting form [36], the ignorance of how to fill it in [36, 37] and the unawareness of the pharmavigilance system functioning [38]. Changing and improving these aspects in health care settings might be very effective to increase ADRs reporting. In particular, a study indicated that giving a feedback after reporting could encourage the heath care provider in this activity [37].
Other specific reasons of under-reporting among nurses emerges in qualitative findings. In fact, there is a misinterpretation of the meaning of "reporting" and an unawareness of nurses' autonomy in ADRs reporting. In line with this, previous research has shown that suspect ADRs was communicated only verbally in 75% of cases by the nurses who said that they had reported (42.7%) [36]. In other studies, nurses do not consider the reporting of ADRs as an activity they are expected to perform autonomously: nurses tend to inform the physician of suspect ADRs and leave him the decision of reporting it [20, 39-40]. Furthermore, fear of consequences after ADR reporting emerges from the qualitative findings. Yet, previous studies have shown how this factor was associated with the belief of having legal liability following ADR reporting [20, 3941]. Misinterpretation of the meaning of "reporting", unawareness of nurses' autonomy, fear of consequences could be due to ignorance regarding the activity of ADR reporting and possible training should focus on these aspects.
The study has some limitations due to the descriptive and qualitative character, the heterogeneity of the sample and of health care settings where the study was conducted. Furthermore, the data were collected in one Italian region which not represent all nurses in Italy. In line with this, wide variations in the contents of the courses regarding the pharmacovigilance could exist across universities and regions. Although these limits influence the generalizability of results, this study provides valuable insights to improve the understanding of ADRs reporting among nurses.
CONCLUSIONS
In order to enhance the Italian pharmacovigilance system, an "ADRs reporting culture" should be promoted with the contribution of all healthcare providers including nurses. At present, nurses are not fully aware of their active role in ADR reporting. Different actions might be proposed to integrate ADR reporting into nursing practice. The under-reporting phenomenon among hospital nurses is associated with knowledge, attitudes and organizational factors. These results are valuable for the pharmacovigilance system as all these factors can be modified setting up appropriate educational interventions and management changes.
This is one of few studies which have explored nurses' reporting of ADRs in Italy. Further research is needed to investigate the barriers and the obstacles associated with nurses' under-reporting. It is necessary to assess the mutual influence of knowledge, attitudes and practice and the burden of organizational aspects on ADR reporting.
From a public health perspective and according to the new European Union Pharmacovigilance legislation, improving strategies for spontaneous ADRs reporting is one of the keys to reducing the risks and increasing the benefits of the use of drugs.
Acknowledgments
The authors are grateful to Pharmacovigilance Committee of the Lazio Region for their support in conducting the study.
REFERENCES
1. World Health Organization. The importance of pharmacovigilance: Safety monitoring of medicinal products. Geneva: WHO; 2002. Available from: http://apps.who.int/medicinedocs/pdf/s4893e/s4893e.pdf.
2. World Health Organization. Safety of medicines in public health programmes: Pharmacovigilance an essential tool. Geneva: WHO; 2006. Available from: www.who.int/medicines/areas/quality_safety/safety_efficacy/Pharmacovigilance_B.pdf?ua=1.
3. Lexchin J. Is there still a role for spontaneous reporting of adverse drug reactions? CMAJ 2006;174(2):191-2.
4. Harmark L, van Grootheest AC. Pharmacovigilance: methods, recent developments and future perspectives. Eur J Clin Pharmacol 2008;64(8):743-52. DOI: 10.1007/s00228-008-0475-9.
5. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP). HMA-EMA; 2014. Available from: www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129135.pdf.
6. Agenzia Italiana del farmaco. La nuova normativa di Farmacovigilanza. AIFA; Available from: www.agenziafarmaco.gov.it/sites/default/files/la_nuova_normativa_di_farmacovigilanza_0.pdf.
7. European Commission (EU). Pharmacovigilance Impact Assessment 2008. Commission Staff Working Document. 2670 Volume I. 2008. Available from: http://ec.europa.eu/health/files/pharmacos/pharmpack_12_2008/pharmacovigilance-ia-vol1_en.pdf.
8. Sultana J, Cutroneo P, Trifirò G. Clinical and economic burden of adverse drug reactions. J Pharmacol Pharmacother 2013;4(Suppl 1):S73-7. DOI: 10.4103/0976-500X.120957.
9. Marcum ZA, Pugh MJV, Amuan ME, Aspinall SL, Handler SM, Ruby CM, et al. Prevalence of Potentially Preventable Unplanned Hospitalizations Caused by Therapeutic Failures and Adverse Drug Withdrawal Events Among Older Veterans. J Gerontol A Biol Sci Med Sci 2012;67(8):867-74. DOI: 10.1093/gerona/gls001.
10. Hazell L, Shakir SAW. Under-reporting of adverse drug reactions - A systematic review. Drug Saf 2006;29(5):385-96.
11. Inman WHW. Attitudes to adverse drug-reaction reporting. Br J Clin Pharmacol 1996;41(5):434-5.
12. Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Determinants of Under-Reporting of Adverse Drug Reactions A Systematic Review. Drug Saf 2009;32(1):19-31. DOI: 10.2165/00002018-200932010-00002.
13. Vallano A, Cereza G, Pedros C, Agusti A, Danes I, Aguilera C, et al. Obstacles and solutions for spontaneous reporting of adverse drug reactions in the hospital. Br J Clin Pharmacol 2005;60(6):653-8.
14. Arnold GJ. Clinical recognition of adverse drug reactions: Obstacles and opportunities for the nursing profession. J Nurs Care Qual 1998;13(2):45-55.
15. Griffith R. Nurses must report adverse drug reactions. Br J Nurs 2013;22(8):484-5.
16. Valente S, Murray L, Fisher D. Nurses improve medication safety with medication allergy and adverse drug reports. J Nurs Care Qual 2007;22(4):322-7.
17. Bobay K, Gentile DL, Hagle ME. The relationship of nurses' professional characteristics to levels of clinical nursing expertise. Appl Nurs Res 2009;22(1):48-53. DOI: 10.1016/j.apnr.2007.03.005.
18. Lim AG, Honey M. New Zealand newly graduated nurses medication management: results of a survey. Nurse Educ Pract 2014;14(6):660-5. DOI: 10.1016/j. nepr.2014.08.005.
19. Nursing and Midwifery Council. Standards for medicines management. London: NMC; 2008. Available from: www.nmc-uk.org/Documents/NMC-Publications/NMC-Standards-for-medicines-management.pdf.
20. Hanafi S, Torkamandi H, Hayatshahi A, Gholami K, Shahmirzadi NA, Javadi MR. An educational intervention to improve nurses' knowledge, attitude, and practice toward reporting of adverse drug reactions. Iran J Nurs Midwifery Res 2014;19(1):101-6.
21. Backstrom M, Ekman E, Mjorndal T. Adverse drug reaction reporting by nurses in Sweden. Eur J Clin Pharmacol 2007;63(6):613-8.
22. Morrison-Griffiths S, Walley TJ, Park BK, Breckenridge AM, Pirmohamed M. Reporting of adverse drug reactions by nurses. Lancet 2003;361(9366):1347-8.
23. Mendes D, Alves C Batel, Marques F. Nurses' spontaneous reporting of adverse drug reactions: expert review of routine reports. J Nurs Manag 2014;22(3):322-30. DOI: 10.1111/jonm.12003.
24. Ulfvarson J, Mejyr S, Bergman U. Nurses are increasingly involved in pharmacovigilance in Sweden. Pharmacoepidemiol Drug Saf 2007;16(5):532-7.
25. Conforti A, Opri S, D'Incau P, Sottosanti L, Moretti U, Ferrazin F, et al. Adverse drug reaction reporting by nurses: analysis of Italian pharmacovigilance database. Pharmacoepidemiol Drug Saf 2012;21(6):597-602. DOI: 10.1002/pds.3225.
26. Italia. Decreto Legislativo 8 aprile 2003, n. 95. Attuazione della direttiva 2000/38/CE relativa alle specialità medicinali. Gazzetta Ufficiale - Serie Generale n. 101, 3 maggio 2003.
27. Opri S, D'Incau P, Meglioranzi M, Pignata C, Brasola S, Conforti A, et al. Venus study: a research and education project on nurses and drug surveillance. Assist Inferm Ric 2011;30(1):6-15.
28. World Health Organization. Advocacy, communication and social mobilization for TB control: a guide to developing knowledge, attitude and practice surveys. Geneva: WHO; 2008. Available from: http://whqlibdoc.who.int/publications/2008/9789241596176_eng.pdf.
29. Creswell JW, Plano Clark VL. Designing and conducting mixed methods research (2 ed). Thousand Oaks, CA: Sage; 2011.
30. Caffari B, Giusti A, Nati G, Raschetti R. I percorsi della formazione per la farmacovigilanza nella Regione Lazio: costruzione e valutazione. In: XII Conferenza Nazionale di Sanità Pubblica.Roma: Ottobre 12-15, 2011. Atti. Roma: Società Italiana di Igiene (SItI); 2011. p. 683.
31. Massari M, Giusti A, Maggini M, Barbariol P, Raschetti R. Valutazione della formazione in salute pubblica: il software SITA (Sistema per l'ITem Analysis). Roma: Istituto Superiore di Sanità; 2009. (Rapporti ISTISAN 09/28). Available from: www.iss.it/binary/publ/cont/09_28_web.pdf.
32. Barbaranelli C, D'Olimpo F. Analisi dei dati con SPSS.I. Le analisi di base. Milano: LED; 2007.
33. Herdeiro MT, Polonia J, Gestal-Otero JJ, Figueiras A. Factors that influence spontaneous reporting of adverse drug reactions: a model centralized in the medical professional. J Eval Clinical Pract 2004;10(4):483-9.
34. Gonzalez-Gonzalez C, Lopez-Gonzalez E. Herdeiro MT, Figueiras A. Strategies to improve adverse drug reaction reporting: a critical and systematic review. Drug Saf 2013;36:317-28. DOI: 10.1007/s40264-013-0058-2.
35. Pulford A, Malcolm W. Knowledge and attitudes to reporting adverse drug reactions. Br J Nurs 2010;19(14):899-904.
36. Fadare JO, Enwere OO, Afolabi AO, Chedi BAZ, Musa A. Knowledge, Attitude and Practice of Adverse Drug Reaction Reporting among Healthcare Workers in a Tertiary Centre in Northern Nigeria. Trop J Pharm Res 2011;10:235-42. DOI: 10.4314/tjpr.v10i3.4. Available from: www.tjpr.org/vol10_no3/2011_10_3_1_Fadare.pdf.
37. Ekman E, Petersson G, Tagerud S, Backstrom M. Awareness among nurses about reporting of adverse drug reactions in Sweden. Drug Healthc Patient Saf 2012;4:61-6. DOI: 10.2147/DHPS.S31103.
38. Li Q, Zhang SM, Chen HT, et al. Awareness and attitudes of healthcare professionals in Wuhan, China to the reporting of adverse drug reactions. Chin Med J 2004;117(6):856-61.
39. Hanafi S, Torkamandi H, Hayatshahi A, Gholami K, Javadi M. Knowledge, attitudes and practice of nurse regarding adverse drug reaction reporting. Iran J Nurs Midwifery Res 2012;17(1):21-5.
40. Hajebi G, Mortazavi SA, Salamzadeh J, Zian A. A Survey of Knowledge, Attitude and Practice of Nurses towards Pharamacovigilance in Taleqani Hospital. Iran J Pharm Res 2010;9(2):199-206.
41. Hamedivafa F, Peiravian F. A survey of knowledge, attitude and practice of nurses towards pharmacovigilance in teaching hospital, Qazvin-Iran. In: 13th Iranian Pharmaceutical Sciences Congress. Isfahan (Iran): September 3-6, 2012. Research in Pharmaceutical Sciences 2012;7(5):S863. Available from: http://rps.mui.ac.ir/index.php/jrps/article/viewFile/906/1430.
Address for correspondence:
Angela Giusti
Centro Nazionale di Epidemiologia, Sorveglianza e Promozione della Salute, Istituto Superiore di Sanità
Via Giano della Bella 34
00162 Rome, Italy
E-mail: angela.giusti@iss.it
Received on 18 February 2015.
Accepted on 21 May 2015.
Conflict of interest statement: No conflict of interest to declare.