Abstracts

Introduction

District health systems are considered as an initial autonomous part of the national health system that combines primary health care with hospital care to provide integrated and complete services for the community ¹1. World Health Organization. The challenge of implementation: district health systems for primary health care. Geneva: World Health Organization; 1988.^,22. Segall M. District health systems in a neoliberal world: a review of five key policy areas. Int J Health Plann Manage 2003; 18 Suppl 1:S5-26.. Integrated district health systems have a substantial role in delivery of comprehensive health services ²2. Segall M. District health systems in a neoliberal world: a review of five key policy areas. Int J Health Plann Manage 2003; 18 Suppl 1:S5-26.. However, it has been anticipated in the Alma-Ata Declaration that most developing countries would fail to achieve the level of a comprehensive health system would need to reform their management or governance practices ²2. Segall M. District health systems in a neoliberal world: a review of five key policy areas. Int J Health Plann Manage 2003; 18 Suppl 1:S5-26..

Iran’s health sector Iran contains a variety of different players such as: Ministry of Health (MOH), Minister of Labor, Cooperative and Welfare, Minister of Defense, Ministry of Oil and many other public departments and private healthcare providers and medical insurers. So, the MOH is responsible for the population’s health in the countryside and planning, monitoring, and supervision of public and private health sectors in Iran. MOH’s executive responsibilities are delegated to university of medical sciences and health services at the provincial level. Medical universities are responsible for public health, health care provision in public facilities, and medical education. A nation-wide network developed to providing and monitoring healthcare acts at the district level (Figure 1) ³3. World Health Organization. Country cooperation strategy for WHO and the Islamic Republic of Iran. Cairo: Regional Office for Eastern Mediterranean; 2006..

The significant role of management and managers to construct effective health systems underlines the need for a comprehensive framework for district health management. This framework could include; planning, human resources development, and supervisory systems that improves health system management performance ⁴4. Filerman GL. Closing the management competence gap. Hum Resour Health 2003; 1:7.. Literature shows the importance of these frameworks. For example, La Rue et al. ⁵5. La Rue KS, Alegre JC, Murei L, Bragar J, Thatte N, Kibunga P, et al. Strengthening management and leadership practices to increase health-service delivery in Kenya: an evidence-based approach. Human Resour Health 2012; 10:25. demonstrate that Leadership Development Programs increase health service coverage as well as the number of visits. Moreover, Dorgan et al. ⁶6. Dorgan S, Layton D, Bloom N, Homkes R, Sadun R, Van Reenen J. Management in healthcare: why good practice really matters. London: McKinsey and Company/London School of Economics; 2010. found a positive association between management capabilities and health outcomes.

Assessing the performance of a health system to monitor and evaluate various aspects of the system and collecting data, review, refine and use this information are the most important factors in improving system efficiency and getting objective health system ⁷7. Banta TW, Palomba CA. Assessment essentials: planning, implementing, and improving assessment in higher education. New Jersey: John Wiley & Sons; 2014.. A district’s health management performance framework is a conceptually structured tool to measure performance of policy makers, payers, providers, patients and population and analyze them to provide helpful information for decision-making ⁸8. Tashobya CK, Silveira VC, Ssengooba F, Nabyonga-Orem J, Macq J, Criel B. Health systems performance assessment in low-income countries: learning from international experiences. Global Health 2014; 10:5.. Egger et al. ⁹9. Egger D, Travis P, Dovlo D, Hawken L. Strengthening management in low income countries. Making Health Systems Work Series. Working Paper 1. http://www.who.int/management/mhswork/en/index.html (accessed on 24/Jan/2017).
http://www.who.int/management/mhswork/en... indicated that the lack of access to national and international guidelines and unclear responsibilities are the two most difficult barriers against achieving health system effectiveness. Therefore, different countries have adopted various approaches for management monitoring and evaluation programs using predetermined indicators. For instance, Gold Star Award in Uganda monitors the management performance of health facilities using 35 managerial standards in 47 districts ¹⁰10. Lochoro P. Measuring patient satisfaction in UCMB health institutions. Health Policy and Development 2004; 2:243-8.. District Health Barometer in South Africa uses predefined indicators and data to compare district health system performance at local and national levels ¹¹11. Day C, Barron P, Massyn N, Padarath A, English R. The district health barometer. http://www.hst.org.za/publications/District%20Health%20Barometers/DHB0708.pdf (accessed on 09/Jul/2017).
http://www.hst.org.za/publications/Distr... .

In Iran, district health system performance related to public healthcare is currently monitored through provincial health centers ¹²12. Mehrdad R. Health system in Iran. International Medical Community 2009; 52:69-73. and hospitals are evaluated by the national hospital accreditation system ¹³13. Bahadori M, Ravangard R, Alimohammadzadeh K. The accreditation of hospitals in iran. Iran J Public Health 2015; 44:295-6.. Although the hospital accreditation system is in progress to improve its standards and evaluation process, the current public healthcare monitoring program mostly focuses on inputs and health network structure instead of process and outcome indicators ¹⁴14. Zalani GS, Bayat M, Shokri A, Mirbahaeddin SE, Rasi V, Alirezaei S, et al. Affecting factors on the performance of community health workers in Iran's rural areas: a review article. Iran J Public Health 2016; 45:1399-410.. However, it seems that there is no specific and comprehensive package of indicators or framework to assess district health system management performance operationally in accordance with management framework ¹⁵15. Tabrizi JS, Gholipour K, Farahbakhsh M, Jahanbin H, Karamuz M. Developing management capacity building package to district health manager in northwest of Iran: a sequential mixed method study. J Pak Med Assoc 2016; 66:1385-91.. So it seems that it is necessary to use a comprehensive set of indicators to assess district health system management performance and identify the strengths and weaknesses by health system authorities when planning and implementing corrective actions. Thus, this study tried to design a framework based on several indicators to assess district health management performance in Iran.

Method

Design and setting

In this study, qualitative and quantitative methods were used for data collection. This study was conducted between September 2015 and April 2016 in Tabriz. Faculty members in health service management, unit chiefs in the northwest provinces of Iran Health Center and district health managers were the target population. The purposive sampling method was used to select expert managers. Forty-five participants took part in the first phase of study to develop indicators. In the second phase, 21 faculty members were surveyed to evaluate the indicators’ content validity. In this phase, the participants were the district health managers and unit chiefs with experience in management evaluation.

Data collection

Semi-structured questionnaires were used to collect data from 45 participants and Table 1 shows participant characteristics. In addition, 21 questionnaires were filled out in quantitative part to validate and finalize the performance indicators of district health management. Study participants included those who were part of the health department or district health managers registered in health management training program and were trained regarding district health system management roles and capabilities. Semi-structured questionnaires were provided to study participants in last section of health management training program when they completed the training program ¹⁵15. Tabrizi JS, Gholipour K, Farahbakhsh M, Jahanbin H, Karamuz M. Developing management capacity building package to district health manager in northwest of Iran: a sequential mixed method study. J Pak Med Assoc 2016; 66:1385-91..

Interviewing and survey

As mentioned above, a semi-structured questionnaire was designed based on district health management capacity building framework and 11 main fields and topics was presented to them to facilitate and guide them to develop related indicators. The questionnaire guidelines consisted of the objectives of the study on performance indicators of district health managers about managerial competencies ¹⁵15. Tabrizi JS, Gholipour K, Farahbakhsh M, Jahanbin H, Karamuz M. Developing management capacity building package to district health manager in northwest of Iran: a sequential mixed method study. J Pak Med Assoc 2016; 66:1385-91.^,1616. Chatora R, Tumusiime P. Health sector reform and district health systems. Brazzaville: WHO Regional Office for Africa; 2004.^,1717. Kloss-Quiroga B, editor. District health management tools. Berlin: InWEnt, Health Section; 2004.^,1818. Byleveld S, Haynes R, Bhana R, Dudley L, Barron P. A review of structures, competencies and training interventions to strengthen district management in the national health system of South Africa. http://www.healthlink.org.za/uploads/files/dmxnatsummary.pdf (accessed on 22/Jan/2017).
http://www.healthlink.org.za/uploads/fil... . Each participant was to present SMART (Specific, Measureable, Attainable, Relevant and Time bonded) indicators based on their experiences to assess district health management performance of. Participants were fully free to present any indicators based on their own interest commensurate with the predefined categories of district health managers’ management capabilities ¹⁵15. Tabrizi JS, Gholipour K, Farahbakhsh M, Jahanbin H, Karamuz M. Developing management capacity building package to district health manager in northwest of Iran: a sequential mixed method study. J Pak Med Assoc 2016; 66:1385-91.. The analysis of questionnaires and indicator extraction was done simultaneously with collecting the questionnaires. Completing the questionnaires continued until saturation in participants’ opinion. So, the latest questionnaires did not introduce any new indicator.

Analyzing the result of the qualitative part was used to extract performance indicators. Based on the findings of the qualitative part, the Validity Tool of District Management Performance Indicators was developed and its validity was examined and confirmed based on content validity index (CVI) and content validity ratio (CVR). Likert scale was used for each indicator. These indicators include the necessity (essential, useful, not essential, not useful), relevance (highly relevant, relevant, moderately relevant, not relevant), transparency (highly transparent, transparent, moderately transparent, not transparent), and simplicity (highly simple, simple, moderately simple, not simple).

Analysis

Phase 1: development of indicators

We conducted sequential data analysis, in which qualitative data is used to generate items for development of quantitative measures ¹⁹19. Östlund U, Kidd L, Wengström Y, Rowa-Dewar N. Combining qualitative and quantitative research within mixed method research designs: a methodological review. Int J Nurs Stud 2011; 48:369-83.^,2020. Jayasekara RS. Focus groups in nursing research: methodological perspectives. Nurs Outlook 2012; 60:411-6. and, in our case, to generate performance indicators. Content analysis and comparison of performance indicators which were specified by the participants were pursued. Then, key themes were determined and were classified in logical format based on relationships. The research team explained the survey clearly to participants, highlighted the confidentiality research and got voluntarily informed consent.

Data collection continued until no new performance indicators were identified. After initial coding, data (indicators) were summarized as a structured thematic table. This step allowed researchers to easily have access to aggregated themes that extracted the participants’ ideas regarding management performance indicators. Finally, clear consensus or non-consensus areas were highlighted, and relationships among indicators were explained in a logical framework. Discussion among the researchers created opportunity for more clarification, testing and revision of the findings.

Phase 2: indicators content validity

Indicators which were generated in the first phase added to content validity questioners. Based on relevant, transparency and simplicity to calculate CVI and also each indicator assessed based on necessity to calculate CVR. CVI and CVR were calculated using the following formulas:

Where:

N_h = Number of panelists indicating “highly relevant or relevant” and “highly transparent or transparent” and “highly simple or simple”;

N_e = Number of panelists indicating “essential or useful”;

N = Total number of panelists.

Also, the average CVI for final instrument was calculated based on Scale-Level CVI/Averaging Calculation Method (S-CVI/Ave). For this purpose the CVI of approved indicators was averaged to calculate S-CVI/Ave ²¹21. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health 2006; 29:489-97.. Microsoft Excel (Microsoft Corp., USA) was used for data analysis.

Ethical concerns

This study has been approved by the Tabriz University of Medical Sciences Research & Ethics Committee. In addition, the topic and the purpose of doing this study was explained to all participants and were entered into the study after getting oral and written consent. Finally, participants who refused to continue their participation in any step were excluded from study.

Result

Participant characteristics in qualitative part

Overall, 45 experts filled out semi-structured questionnaires to identify district health management’s performance indicators. Demographically, 86.7% of participants were male, 66.7% were district health managers, and 22.2% of them were head of unit in provincial health center. Also, 11.1% of participants had less than five years of experience in their current position while 55.6% had more than 20 years (Table 1).

Participant characteristics in quantitative part

In the quantitative part, 21 people participated. 19% of participants were faculty members, 38.1% district health managers and 42.9% of them were head of unit in provincial health centers. The majority of participants were male (72.7%). Furthermore, 71.4% of the participants were medical doctors. Lastly, only 5.8% of participant had less than 5 years of experience and 42.9% of respondents had over 20 years of experience (Table 1).

District health management performance indicators

Based on the results of the qualitative part, 81 indicators were found to be related to district health management performance. Management performance indicators were classified into 11 categories, as following: human resources and organizational creativity, management and leadership, rules and ethics, planning and evaluation, district management, health resources management and economics, community participation, quality improvement, research in health system, health information management, epidemiology and situation analysis (Table 2).

Out of 81 primary indicators, 68 of them that had been rated higher than 0.7 in CVI analysis were kept and 13 of them that had been rated less than 0.7 were excluded. On the other hand, 20 indicators were below CVR threshold, which was 0.42. Based on the cutoff point of 0.7 and 0.42 for CVI and CVR respectively (21 respondents who were rated CVR), 24 indicators were below CVI and CVR threshold and excluded from the final framework. In relation to indictors which were excluded from final list, in some cases participants believed that these indicators were out of district health management responsibilities and control such as: rate of achievements predicted budget, proportion of the population exposed to catastrophic costs, the existence of district data as electronic health records, or well presented in other accepted indicators as a comprehensive indicator.

Also, based on the result of the expert panel session, 4 indicators were merged with other indicators (n. 12 with n. 13, n. 56 with n. 54, n. 60 with n. 55 and n. 76 with n. 79) (Table 2).

Finally, the 53 most relevant indicators remained. The average CVI (S-CVI/Ave) of approved indicators was 0.84. Also, study findings indicate based on participants’ viewpoints epidemiology and situation analysis reached highest average CVI (S-CVI/Ave = 0.90) and research in health system reached least average CVI (S-CVI/Ave = 0.78) (Table 3).

Discussion

According to the results, 53 indicators were included in district health management performance framework. The average CVI (S-CVI/Ave) of the final framework was 0.84. The governing role of the district health system in overseeing service providers and monitoring their performance were the most important part of management performance framework.

Siddiqi et al. ²²22. Siddiqi S, Masud TI, Nishtar S, Peters DH, Sabri B, Bile KM, et al. Framework for assessing governance of the health system in developing countries: gateway to good governance. Health Policy 2009; 90:13-25. provided a framework to assess health systems performance in developing countries with 10 principles including: strategic vision, participation and consensus orientation, rule of law, transparency, responsiveness, equity and inclusiveness, effectiveness and efficiency, accountability, intelligence and information and ethics. Brinkerhoff et al. ²³23. Brinkerhoff DW, Fort C, Stratton S. Good governance and health: assessing progress in Rwanda. Kigale: Twubakane Decentralization and Health Program; 2009. indicated that responsiveness, leadership, customer voice, accountability, transparency, evidence-based practice, efficiency and effectiveness as governing outcome indicators in district health systems. Also, in some other countries, the World Health Organization’s (WHO) building blocks were used to assess district health management and related interventions ²⁴24. Drobac PC, Basinga P, Condo J, Farmer PE, Finnegan KE, Hamon JK, et al. Comprehensive and integrated district health systems strengthening: the Rwanda Population Health Implementation and Training (PHIT) Partnership. BMC Health Serv Res 2013; 13:S5.. Whereas, it should be kept in mind that these frameworks weren’t developed as specific tools to assess managerial performance at district levels.

Based on this analysis, indicators that were related to motivation and human resources developments as well as reward and performance management systems were classified under human resources management category. Dieleman et al. ²⁵25. Dieleman M, Gerretsen B, van der Wilt GJ. Human resource management interventions to improve health workers' performance in low and middle income countries: a realist review. Health Res Policy Syst 2009; 7:1-13. grouped continuing education, supervision, incentives payment, legal and regulatory compliance, improving quality of care and mixed targets under human resources development interventions. In another study, Liu et al. ²⁶26. Liu X, Martineau T, Chen L, Zhan S, Tang S. Does decentralisation improve human resource management in the health sector? A case study from China. Soc Sci Med 2006; 63:1836-45. categorized human resource management (HRM) indicators into three areas; appropriate staff numbers, appropriate skills and experiences, and providing appropriate productivity to organizational performance. In another study, Kennedy ²⁷27. Kennedy VC. A study of local public health system performance in Texas. J Public Health Manag Pract 2003; 9:183-7. defined HRM performance based on employees’ incentives and rewards, active participation in continuing education and comprehensive feedback systems. In this study, human resource management indicators were classified in a different category than resource management. However, in literature, other studies have classified them in the same category. Also, from the perspective of study participants, approved indicators in this category were able to represent district health system management performance in motivating and directing human resources in health network to achieve its objectives.

Resource management includes budget management, costs and financial resources, equipment and facility management. Dadgar et al. ²⁸28. Dadgar E, Janati A, Tabrizi JS, Asghari-Jafarabadi M, Barati O. Iranian expert opinion about necessary criteria for hospitals management performance assessments. Health Promot Perspect 2012; 2:223. included three indicators as equipment, facility management, and financial resources in the resource management category. Kennedy ²⁷27. Kennedy VC. A study of local public health system performance in Texas. J Public Health Manag Pract 2003; 9:183-7. identified unit cost as a single indicator for resource management. Accordingly, based on the current study’s results, a comprehensive range of indicators can be used as a useful tool to assess Resource management. It appears, due to legal and regulatory limitations, district health managers were not completely free and autonomous to change system inputs and structure to improve efficiency and effectiveness. So, some of predefined indicators in the current study were rejected by participants because district health authorities do not have the power to make changes.

Based on our findings, the indicators of district health management compiled for empowerment of managers. In this respect, some studies have confirmed that managers’ performance must be assessed based on their skills and capabilities. Dadgar et al. ²⁸28. Dadgar E, Janati A, Tabrizi JS, Asghari-Jafarabadi M, Barati O. Iranian expert opinion about necessary criteria for hospitals management performance assessments. Health Promot Perspect 2012; 2:223. found that manager participation in programs, employees’ performance assessment and rewarding systems indicates managers’ performance. Kennedy ²⁷27. Kennedy VC. A study of local public health system performance in Texas. J Public Health Manag Pract 2003; 9:183-7. demonstrated that work experience, competency and having a comprehensive approach are the most important indicators of managers’ performance.

Systematic processes for human resources management, complaints management and customer satisfaction have been considered as rules and ethics management indicators. Dadgar et al. ²⁸28. Dadgar E, Janati A, Tabrizi JS, Asghari-Jafarabadi M, Barati O. Iranian expert opinion about necessary criteria for hospitals management performance assessments. Health Promot Perspect 2012; 2:223. showed that employee and customer satisfaction and their complaints must be categorized differently. It seems that the existence of lobbying and politics in managerial decisions at district levels restrict the strict implementation of rules based on rational decisions ²⁹29. Tabrizi JS, Gholipour K, Farahbakhsh M, Hasanzadeh A. Managerial barriers and challenges in Iran public health system: East Azerbaijan health managers' perspective. J Pak Med Assoc 2017; 67:409-15.. As a result, some of the indicators, such as compliance with employing position requirement that were out of district health managers’ controls were removed from the final list.

The existence and implementation of action plans and also coordination with relevant organizations were considered as planning and evaluation indicators of. The study performed by Dadgar et al. ²⁸28. Dadgar E, Janati A, Tabrizi JS, Asghari-Jafarabadi M, Barati O. Iranian expert opinion about necessary criteria for hospitals management performance assessments. Health Promot Perspect 2012; 2:223. pointed out that the existence of strategic and operational planning and stakeholder participation in program development are planning indicators. Fang et al. ³⁰30. Fang C-H, Chang S-T, Chen G-L. Competency development among Taiwanese healthcare middle manager: a test of the AHP approach. African Journal of Business Management 2010; 4:2845-55. found that planning is an important dimension in management performance framework. Though, it should be noted that development and implementation of strategic and operational planning in district health systems from the perspective of participants can represent the existence of a monitoring and evaluation program in the health network.

According to our results, supporting the quality improvement programs and enriching the infrastructural capacities to improve the quality of services were quality management indicators. Also, research in the health system area is closely related to quality improvements. In addition, based on our findings, it is necessary that employees participate in problem solving training programs. Moreover, data collection and data analysis on the performance indicators and health services indicators, in addition to feedback and evidence based planning, were other categories in the district health system management performance framework. Dadgar et al. ²⁸ indicated that clinical governance, quality improvement, data collection and information management were four important components of their framework. ²⁸28. Dadgar E, Janati A, Tabrizi JS, Asghari-Jafarabadi M, Barati O. Iranian expert opinion about necessary criteria for hospitals management performance assessments. Health Promot Perspect 2012; 2:223. Brinkerhoff et al. ²³23. Brinkerhoff DW, Fort C, Stratton S. Good governance and health: assessing progress in Rwanda. Kigale: Twubakane Decentralization and Health Program; 2009. concluded that high quality service provisions and customer satisfaction are important indicators of district health system management performance. Based on WHO’s building blocks, quality and safety are the intermediate goals of any health system. Kennedy ²⁷27. Kennedy VC. A study of local public health system performance in Texas. J Public Health Manag Pract 2003; 9:183-7. argued that access to the latest information technology, information utilization and information sharing must be considered as management performance indicators.

By participating in health services prioritization of, the community will trust the health system, the health system’s responsiveness of will improve and people will collaborate more to achieve an efficient and cost effective system ³¹31. Kamuzora P, Maluka S, Ndawi B, Byskov J, Hurtig A-K. Promoting community participation in priority setting in district health systems: experiences from Mbarali district, Tanzania. Glob Health Action 2013; 6:22669.^,3232. Heritage Z, Dooris M. Community participation and empowerment in healthy cities. Health Promot Int 2009; 24 Suppl 1:i45-i55.. Community participation promotes healthy behaviors, service provision, resource mobilization and community empowerment ³³33. Eftekhari MB, Mirabzadeh A, Forouzan AS, Dejman M, Afzali HM, Djalalinia S, et al. A qualitative study of community-based health programs in Iran: an experience of participation in IR Iran. Int J Prev Med 2014; 5:679-86.. In this regard, advocacy and inter-sectoral collaboration are two effective strategies for health promotion and health system development ³⁴34. Pathfinder International. Straight to the point: setting advocacy priorities. http://www.pathfinder.org/publications-tools/publication-series/Straight-to-the-Point-Advocacy.html (accessed on 17/Nov/2017).
http://www.pathfinder.org/publications-t... that were an important category in management framework. Therefore, inter-sectoral collaboration and community contribution had the same score as the management effectiveness indicators and managerial capabilities and competencies in health systems ³⁵35. Adeleye OA, Ofili AN. Strengthening intersectoral collaboration for primary health care in developing countries: can the health sector play broader roles? J Environ Public Health 2010; 2010:272896.^,3636. World Health Organization. The world health report 2008: primary health care now more than ever. Geneva: World Health Organization; 2008.^,3737. Peake S, Gallagher G, Geneau R, Smith C, Herel M, Fraser H, et al. Health equity through intersectoral action: an analysis of 18 country case studies. Ottawa: Public Health Agency of Canada/Geneva: World Health Organization; 2008..

In this study, the management performance indicators were extracted using a qualitative and quantitative study of managers from the district health system, faculty members and other participants. However, identified indicators weren’t weighted and ordered. On the other hand, the real world applicability of a designed framework must be evaluated in health systems to find out its weaknesses and limitations. Also, a small sample size in the quantitative phase makes it necessity to use real field data to assess framework validity. These indicators must be practical and applicable for district health system. However, we identified the comprehensive collection of indicators and the results of this study can be useful for monitoring and improving district health system management. The proposed framework as a comprehensive assessment tool can be used by national and provincial health authorities to monitor and direct district health systems. District health authorities can also use the current framework as a self-assessment tool and performance improvement guide to manage their own programs and fully respond to their duties. For this purpose, the first step is developing approved indicators as a check list format and cut off points must be determined for specific indicators such as unit costs, catastrophic payment, health insurance coverage rate and healthcare provider per population.

Conclusion

A district health system management performance assessment framework was developed according to district health managers’ capacity building program and covers all dimensions of district health management, such as: human and capital management, leadership, governance, planning and evaluation, and efficiency and effectiveness. Considering the relatively high validity and reliability of acquired indicators and the absence of a comprehensive framework to assess the performance of district health managers, it seems that this framework can be helpful to assess the performance of district health management and create performance improvement programs at the district level. It has been recommended that provincial health system managers provide regular monitoring programs and feedback to district health managers using this framework. Also, health managers can use this framework to control and manage district health systems.

Acknowledgments

We are deeply grateful of the contribution of Eastern Azerbaijan Province Health Center employees and district health managers on data collection. This study was supported by the School of Management and Medical Informatics, Tabriz Health Services Management Research Center and Health Deputy of the Tabriz University of Medical Sciences.

References

Publication Dates

History