RESEARCH
THEME PAPERS
Cost and results of information systems for health and poverty indicators in the United Republic of Tanzania
Coûts et résultats des systèmes d'information fournissant les indicateurs de santé et de pauvreté en République-unie de Tanzanie
Costo y resultados de los sistemas de información para los indicadores de salud y pobreza en la República
Unida de Tanzanía
Vanessa RommelmannI; Philip W. SetelII,1; Yusuf HemedIII; Gustavo AngelesIV; Hamisi MponezyaV; David WhitingVI; Ties BoermaVII
IStatistics Research Division, RTI International, Research Triangle Park, NC, USA
IIDepartment of Epidemiology and MEASURE Evaluation, Carolina Population Center, University of North Carolina at Chapel Hill, 206 West Franklin St, 2nd Floor, Chapel Hill, NC 27516, USA
IIIMinistry of Health, United Republic of Tanzania, Dar es Salaam, United Republic of Tanzania
IVDepartment of Maternal and Child Health and MEASURE Evaluation, Carolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
VTanzanian Public Health Association, Dar es Salaam, United Republic of Tanzania
VIFaculty of Medicine, School of Clinical Medical Sciences, University of Newcastle upon Tyne, Newcastle upon Tyne, England
VIIWorld Health Organization, Geneva, Switzerland
ABSTRACT
OBJECTIVE: To examine the costs of complementary information generation activities in a resource-constrained setting and compare the costs and outputs of information subsystems that generate the statistics on poverty, health and survival required for monitoring, evaluation and reporting on health programmes in the United Republic of Tanzania.
METHODS: Nine systems used by four government agencies or ministries were assessed. Costs were calculated from budgets and expenditure data made available by information system managers. System coverage, quality assurance and information production were reviewed using questionnaires and interviews. Information production was characterized in terms of 38 key sociodemographic indicators required for national programme monitoring.
FINDINGS: In 200203 approximately US$ 0.53 was spent per Tanzanian citizen on the nine information subsystems that generated information on 37 of the 38 selected indicators. The census and reporting system for routine health service statistics had the largest participating populations and highest total costs. Nationally representative household surveys and demographic surveillance systems (which are not based on nationally representative samples) produced more than half the indicators and used the most rigorous quality assurance. Five systems produced fewer than 13 indicators and had comparatively high costs per participant.
CONCLUSION: Policy-makers and programme planners should be aware of the many trade-offs with respect to system costs, coverage, production, representativeness and quality control when making investment choices for monitoring and evaluation. In future, formal cost-effectiveness studies of complementary information systems would help guide investments in the monitoring, evaluation and planning needed to demonstrate the impact of poverty-reduction and health programmes.
Keywords: Information systems/organization and administration/economics; Health status indicators; Demography; Costs and cost analysis; Evaluation studies; Comparative study; United Republic of Tanzania (source: MeSH, NLM).
RÉSUMÉ
OBJECTIF: Étudier les coûts des activités de génération d'informations complémentaires dans un pays à ressources limitées et comparer les coûts et les sorties des sous-systèmes d'information fournissant les statistiques en matière de pauvreté, de santé et de survie, nécessaires à la surveillance, à l'évaluation et à la notification dans le cadre des programmes sanitaires menés en République-unie de Tanzanie.
MÉTHODES: Neuf systèmes utilisés par quatre organismes publics et par des ministères ont été évalués. Les coûts ont été calculés à partir des données budgétaires et des informations relatives aux dépenses mises à disposition par les gestionnaires des systèmes d'information. A partir de questionnaires et d'entretiens, une évaluation de la couverture assurée par les systèmes, de l'assurance de la qualité et de la production de données a été effectuée. 38 indicateurs sociodémographiques clés, nécessaires à la surveillance opérée par les programmes nationaux, ont servi à caractériser la production de données.
RÉSULTATS: Sur la période 2002-2003, le fonctionnement des 9 sous-systèmes d’information générant les données nécessaires au calcul de 37 des 38 indicateurs sélectionnés, a coûté US $ 0,53 environ par citoyen tanzanien. Le système de recensement et de notification fournissant les statistiques sanitaires bénéficiait des plus forts taux de participation de la population et correspondait au coût total le plus élevé. Les enquêtes sur les ménages représentatifs à l'échelle nationale et les systèmes de surveillance démographique (ne recourant pas à des échantillons représentatifs au plan national) ont fourni plus de la moitié des indicateurs et appliqué l'assurance de la qualité la plus rigoureuse. Cinq systèmes ont produit moins de 13 indicateurs, avec un coût par participant comparativement plus élevé.
CONCLUSION: Les décideurs politiques et les planificateurs de programmes doivent être conscients des nombreux compromis à respecter entre les coûts des systèmes, la couverture assurée par ces systèmes, la production de données, la représentativité et le contrôle de la qualité lors des choix relatifs aux investissements bénéficiant à la surveillance et à l'évaluation. A l'avenir, des analyses coût/efficacité formelles des systèmes d'informations complémentaires pourraient guider les investissements dans la surveillance, l'évaluation et la planification, nécessaires pour prouver l'impact des programmes de lutte contre la pauvreté et d'amélioration de la santé.
Mots clés: Système information/organisation et administration/économie; Indicateur état sanitaire; Démographie; Coût et analyse coûts; Etude évaluation; Etude comparative; République-Unie de Tanzanie (source: MeSH, INSERM).
RESUMEN
OBJETIVO: Examinar los costos de las actividades complementarias de generación de información en un entorno con recursos limitados y comparar los costos y los resultados de los subsistemas de información que generan los datos sobre pobreza, salud y supervivencia requeridos para el monitoreo, evaluación y comunicación de información sobre los programas de salud en la República Unida de Tanzanía.
MÉTODOS: Se evaluaron nueve sistemas utilizados por cuatro organismos públicos o ministerios. Los costos se calcularon a partir de los datos sobre presupuestos y gastos proporcionados por los administradores de los sistemas de información. La cobertura de los sistemas, la garantía de la calidad y la producción de información se analizaron mediante cuestionarios y entrevistas. La producción de información se caracterizó en función de 38 indicadores sociodemográficos fundamentales requeridos para monitorizar los programas nacionales.
RESULTADOS: En 2002-2003 se invirtieron aproximadamente US$ 0,53 por ciudadano tanzano en los nueve subsistemas que generaban información sobre 37 de los 38 indicadores seleccionados. Las mayores poblaciones participantes y los costos totales más elevados correspondieron al censo y al sistema de comunicación de la información para las estadísticas de rutina de los servicios de salud. Las encuestas de hogares representativas a nivel nacional y los sistemas de vigilancia demográfica (que no están basados en muestras representativas a nivel nacional) generaban más de la mitad de los indicadores y utilizaban los sistemas más rigurosos de garantía de la calidad. Cinco sistemas generaban menos de 13 indicadores y tenían un costo relativamente elevado por participante.
CONCLUSIÓN: Las instancias normativas y los planificadores de programas deben ser conscientes de los numerosos compromisos que hay que barajar entre el costo de los sistemas, la cobertura, la producción, la representatividad y el control de la calidad a la hora de tomar decisiones de inversión con fines de monitoreo y evaluación. En el futuro, la realización de estudios formales de la costoeficacia de los sistemas complementarios de información podría ayudar a orientar las inversiones en monitoreo, evaluación y planificación requeridas para demostrar el impacto de los programas de salud y reducción de la pobreza.
Palabras clave: Sistemas de información/organización y administración/economía; Indicadores de salud; Demografía; Costos y análisis de costo; Estudios de evaluación; Estudio comparativo; República Unida de Tanzanía (fuente: DeCS, BIREME).
Introduction
Demand for information for monitoring and evaluating the progress and impact of health programmes is rapidly escalating in many developing countries (1). With certain exceptions, such as fertility and family planning, there is widespread agreement that the impact of investments of the past 30 years in health development and on priority areas such as maternal mortality is largely unknown (2). The reason usually cited for this lack of data is the absence of reliable information systems to produce repeated measures of appropriate indicators.
Those responsible for producing indicators and reporting on health sector progress and poverty reduction are faced with important decisions about how best to invest resources for monitoring and evaluation (M&E). Health information can be collected through a variety of data-collection platforms or subsystems, including censuses, household surveys, disease surveillance, reporting of health service statistics, facility surveys and vital registration. No single data-collection platform can meet all information needs, but for some indicators the information can be collected through different platforms. How are investors to objectively evaluate their options? Based on what criteria should additional resources be put into, say, collection of routine facility-based statistics versus repeated nationally representative surveys, or a sample vital registration system with verbal autopsy?
In the Tanzanian Ministry of Health, some initial efforts have been made to streamline and integrate various health components of routine information systems (36). The most comprehensive statement of a national information strategy for the United Republic of Tanzania across sectors has been the National Poverty Monitoring Master Plan (PMMP). The PMMP considers component costs across complementary systems in many sectors and is the mechanism for M&E and reporting on the accomplishments of debt relief under the poverty-reduction strategy (7). It lays out a vision of information needs, the diverse sources of available data, and sustainable, long-term budgets. However, both the efforts of the Ministry of Health and the PMMP are largely "additive" plans that do not explore the possibility of improving efficiency by considering the comparative costs, outputs or overlaps of the existing systems.
At the international level, general guidance about best practice for investing in the multiple components of comprehensive national health and demographic information is sparse. Although there have been no formal studies of the cost-effectiveness of information for policy- and decision-making, a WHO report on the design and management of information systems did consider various cost aspects in relation to the routine health information systems that generate service statistics (8). This is as yet the only publication to raise the issue of trade-offs between accuracy and cost in the production of health and demographic data (9); there has been no systematic investigation of this topic in the wider peer-reviewed literature.
This article reports on a study of the costs and results of information systems in the United Republic of Tanzania in 2002 and 2003. No formal cost-effectiveness study was attempted, nor was it our aim to formally rate the quality of the indicators produced by the systems. Rather the purpose of the present study was to document the cost component and put it in the perspective of the information provided for a selected set of priority indicators.
The United Republic of Tanzania makes a good-case study, as it has a national PMMP and a wide range of data-collection platforms (10). For example, national surveys such as the Demographic and Health Surveys (DHS) (1114) and the National Household Budget Survey (NHBS) (15) have been factored into the overall national information strategy articulated in the PMMP.
Methods
Because of the lack of published research in this area, no established methodology could readily be adapted to this study. We therefore developed a rapid systematic methodology for assessing the costs and results of the principal health and demographic information systems in the United Republic of Tanzania.
Sample selection
Systems that were capable of producing a range of population-based indicators required for the M&E programmes of four national reform initiatives or that are specifically mentioned in the PMMP were included in the assessment. To maximize the comparability, the following types of subsystem were excluded:
surveys carried out in the United Republic of Tanzania that were capable of producing information for indicators, but were not part of an existing system at the time of the assessment (e.g. Core Welfare Indicators Questionnaire);
systems planned, but not operating at the time of the review; and
disease-specific information systems associated with vertical programmes or specific research endeavours.
Table 1 lists basic characteristics of the final sample of nine subsystems included in the study.
Cost and coverage estimates
"Top-down" and "bottom-up" approaches were used in the cost analysis. In the top-down analysis, data were collected from the highest organizational level of the system, and then down to the most disaggregated level (16). This method is appropriate for surveys and centralized systems and was used to assess costs for DHS; national census; NHBS; Ifakara Demographic Surveillance System (DSS); Rufiji DSS; and Adult Morbidity and Mortality Project (AMMP)-2 DSS.
The community-based village register system and the facility-based routine health management information system (HMIS), were assessed using a "bottom-up" approach. This technique aggregates costs as they are incurred by or budgeted for specific tasks by starting at the point of initial data collection (e.g. health facilities), and then moving upwards to the central level (16). Interviews were conducted at four village, two district and two regional offices and at the central level to obtain information on personnel time, supplies and equipment needed for all activities of the village register system. Bottom-up costing data were made available to the investigators for HMIS from a previous review carried out in 1999 (17). Although vital registration is also decentralized, costs were assessed using the top-down approach because of limited study resources.
For all costings of information systems, expenditure reports were collected from information system managers whenever possible. Budgets were used to supplement these data, or were used for costing when no other sources of information were available. Costs were classified as either "capital" (e.g. buildings and vehicles) or "recurrent" (e.g. salaries and materials). These figures were then annualized and total system costs (represented in US$ 2001) were derived from the resulting estimates. Information on the sizes of the samples or populations covered was also collected and converted into 2001 population estimates, using growth rate adjustments when necessary. The official population data and estimates used in the study were obtained from the National Bureau of Statistics.
Cost outcome measures
Cost and coverage estimates were used to calculate three annualized outcome measures. Firstly, the total system cost which gives an idea of the overall scale of investment required annually for each system. Secondly, the total system cost divided by the population covered by these systems yields a per participant cost, which was used to estimate the cost of data collection at each collection point, interview or encounter. The third type of cost estimate assumes that the production of information and indicators for national M&E programmes can be considered a public good that benefited all Tanzanians through better policy, planning and programme decisions. To derive this cost per capita, total system costs were divided by the estimated Tanzanian population.
Measuring information output
Data on system outputs were collected using questionnaires, document review, interviews with system managers and field visits to determine system coverage, production (i.e. types of indicators produced at different levels of aggregation (e.g. village, district or region), the frequency with which they were produced and the dissemination of the findings), quality control efforts and use. First, we generated a composite list of 38 key health and sociodemographic indicators required in the M&E programmes of the PMMP, the Public Health Sector Reform Programme and the Local Government Reform Programme. We also included indicators required in the production of comprehensive annual district council health plans. The "output" of a system was characterized as the number of these statistics produced for the indicators. Effort expended in assuring data quality was assessed by a simple count of standard techniques used by each system. These methods included: retest or re-interview of subsample; translation or back-translation (a method used to check that the intended meaning of a study document in a particular language has not been altered in translation) of tools; regular meetings with enumerators; correction and clarification in the field; logical checks of responses; internal checks and comparisons; and double data entry.
It was not possible to develop a common operational metric for the use or impact of the informational outputs of these systems. The use of system outputs (including indicators, research, reports and other materials) is covered elsewhere (18).
Findings
Comparative costs
Table 2 lists total annual costs, annualized per participant costs and per capita costs in US$ 2001. The findings should be interpreted with care, as cost data obtained from the different systems varied widely in terms of type and quality. Specific concerns about data quality are noted in Table 2 and discussed below. During the assessment of the village register system it became clear that coverage and level of community investment varied widely by region and district. Therefore, a rigorous estimate of the costs for the village register could not be obtained from the data collected.
Total annual system costs ranged from approximately US$ 201 000 (Rufiji DSS) to US$ 8 000 000 (national census). The differences in system costs are largely a function of size and coverage, but also reflect costs associated with various data collection and quality assurance techniques.
Excluding the village register system, we estimated that approximately US$ 0.53 was spent on these selected systems per year for every Tanzanian. The per-participant metric showed a wide range, with the two nationally representative household surveys carrying the highest costs, and the national census, HMIS, vital registration, and AMMP-2 DSS all costing less than US$ 1 per participant in 2001. Four systems had coverage or samples of 100 000 or more and per participant costs of US$ 1 or less (AMMP-2 DSS, HMIS, national census and vital registration). Finally, when looked at from the standpoint of annualized cost per Tanzanian, except for the national census, which costs approximately US$ 0.25, the annualized per capita costs for all other systems were less than US$ 0.10.
Production of indicators, quality assurance and coverage
Table 3 provides a summary of the selected indicators currently produced by the systems, and the frequency with which they are generated. Eighty-four per cent of the possible indicators have ever been measured, and repeated measures are available for all but one of these. Indicators for immunization coverage and wasting in children are used in all four of the national M&E programmes and produced by five of the nine systems. Three M&E programmes require indicators on seroprevalence of human immunodeficiency virus (HIV). These data are obtained through surveillance activities, but not by the systems included in this study at the time it was conducted. Of the indicators ever measured, 65% had been calculated by Ifakara DSS, whereas vital registration produced no indicators. The DSS systems, household surveys and national census measured the greatest number of indicators.
The inventory of data quality assurance techniques used by each system revealed that six systems used at least five different quality assurance procedures. The most commonly used techniques were re-testing and re-interviewing of subsamples, correction or clarification in the field, internal checks and comparisons, and regular meetings with enumerators. Translation or back-translation of tools, logical checks of responses, and double data entry were less commonly used.
In terms of the levels at which each system could produce indicators (Table 4), two systems (national census and DHS) were found to be capable of producing estimates for the United Republic of Tanzania that included both the mainland and Zanzibar. Three systems could produce estimates for the mainland, rural and urban areas of the mainland, and at the regional level (national census, NHBS and DHS). HMIS gave estimates based on partial reporting, and AMMP-2 DSS provided national estimates by applying sampling weights, although the estimates were modelled on the basis of a non-representative sample (19).
Apart from the national census, no system provided estimates at the district-level or below for the majority of the population of the United Republic of Tanzania. HMIS may have adequate coverage in government health facilities to provide most districts and service populations (people living in the geographical area served by the health facility) with indicators, but this cannot be determined from central-level reporting rates, which are low (18). Vital registration and the village register system cover many districts, but coverage within those districts and quality of output over time is in doubt (20). The DSS sites incorporate parts of districts and municipalities and routinely provide data to local, district and regional authorities, as well as to stakeholders and government bodies at the national level.
Relating costs and results
Table 5 groups systems by production of indicators versus costs. Cut-off values within each cost-type (e.g. total system cost) are the median values for that category. Figures in parentheses in the table cells refer to the number of indicators (from Table 3) ever generated and quality assurance methods employed by the system. Three systems (DHS, Ifakara DSS and AMMP-2) produced half or more of the indicators. Of these, AMMP-2 DSS cost less than the median in each cost category, although it produced fewer indicators than the others. Ifakara DSS cost less than the median in two of the three categories and produced the greatest number of indicators. DHS cost less than the median in one cost category and produced 23 indicators. All three systems used a large number of quality assurance methods.
Looking across all cost and results categories, there was much greater variation among the systems producing less than half of the 38 selected indicators. The vital registration system of the United Republic of Tanzania, for example, fell below the median across the cost categories, but did not produce indicators. Three systems produced more than a third of the indicators (Rufiji DSS, HMIS and national census). Of these, national census and HMIS produced national figures but employed fewer quality assurance methods. Rufiji DSS had per participant costs above the median, and HMIS and the national census had costs above the median in two of the three cost categories. NHBS, which produced seven nationally and regionally representative indicators out of the 38 selected, used the greatest number of quality assurance methods of systems in this group, and its cost was below the median per capita value.
Discussion
This study explored the costs and results of information generation, M&E, and surveillance for national efforts on poverty reduction and health reform. The rationale for this investigation stems from the fact that demand for information is increasing, yet the investment of resources in information systems has lacked an evidence base.
All systems included in this assessment make unique and complementary contributions to information generation in the United Republic of Tanzania. Within the context of the missions of the information systems and the resources available it is clear that there are many trade-offs with respect to system coverage, representativeness, frequency of data collection and reporting, and quality control.
It is also important to bear in mind the limitations of this exploratory study in terms of design, methods and data. First, a formal cost-effectiveness analysis was not possible due not only to constraints of time and resources, but also to the complexity of characterizing an appropriate "effectiveness" parameter that could be measured across systems. Future studies that compare the costs, benefits, results and "effectiveness" of diverse systems face conceptual challenges in this regard. Characterization of the results solely in terms of the production of data for a limited set of indicators, as for this study, is only a beginning; it provides a limited metric for assessing the overall benefits of any particular system.
Our findings are also affected by the uneven quality of the cost data made available. Although the cost information provided to the study team was detailed, it was not adequate (particularly for capital costs) to allow a full and direct comparison of costs across systems. Lastly, the review of indicators was limited to those used by national M&E programmes and was therefore not representative of the true breadth of outputs from the selected systems. For example, key indicators such as migration and total fertility rate were excluded from the evaluation.
The comparison of costs with results is highly sensitive to the cost category selected (i.e. total system, per participant or per capita). Clearly this complicates the interpretation of our results, but does suggest that strategic decisions about where additional funds might be invested in information systems and what that investment might mean should be made on the basis of carefully formulated priorities.
In the United Republic of Tanzania, for example, there are at least two areas that illustrate this point. The first relates to governmental reforms, and the rapid devolution of evidence-based planning and reporting requirements to the district level. This has led directly to an urgent need for reliable and frequently updated information at a lower level of disaggregation than most systems are capable of providing. The second area is the paucity of sources for the measurement of total and cause-specific mortality statistics. The best choice for investments in information systems capable of meeting these needs will depend upon how well-equipped those who must rely on system outputs are to factor costs into a formulation of needs for quality, representativeness, frequency of measurement and so forth. It is hoped that the parameters presented here can, when coupled with careful priority-setting relevant to specific needs and trade-offs by type of information system, provide some guidance about where additional resources for strategic information or M&E should be deployed when available. 
Acknowledgements
The authors are grateful to the managers and officials of the information systems who participated in the study and provided costing data, and to Health Research for Action (HERA) for allowing us to use costing data previously collected.
Funding: This article is, in part, an output of the Adult Morbidity and Mortality Project (AMMP). AMMP is a project of the Tanzanian Ministry of Health, funded by the UK Department for International Development (DFID) and implemented in partnership with the University of Newcastle upon Tyne, England. The article is also an output of MEASURE Evaluation, Phase 2, which is a United States Agency for International Development (USAID) Cooperative Agreement (# GPO-A-00-03-00003-00) implemented by the Carolina Population Center, University of North Carolina at Chapel Hill, USA.
Ethical review: This study was approved by the Institutional Review Board of the University of North Carolina at Chapel Hill, and by the Tanzanian Ministry of Health.
Competing interests: At the time of the study, PWS, YH and DRW were all employed by the Adult Morbidity and Mortality Project, Phase 2, which operated one of the information systems discussed in this paper.
References
1. Evans T, Stansfield S. Health information in the new millennium: a gathering storm? Bulletin of the World Health Organization 2003;81:856.
2. Hill K, AbouZahr C, Wardlow T. Estimates of maternal mortality for 1995. Bulletin of the World Health Organization 2001;79:182-93.
3. Ngatunga SE. A brief on the role of the HMIS in the implementation of health sector reforms in Tanzania. In: Workshop on lessons learned in health sector reform. Bagamoyo: United Republic of Tanzania, Ministry of Health, Policy and Planning Division; 1998.
4. Brown W, Nsubuga P, Eseko N. Assessment of infectious disease surveillance systems in Tanzania. Dar es Salaam: Ministry of Health, United Republic of Tanzania/World Health Organization/US Centers for Disease Control and Prevention and the Environmental Health Project; 1999. EHP Activity Report, No. 62.
5. Health Information and Research Section. Stakeholders consultative meeting on the development of a minimum package of health information. Moshi. Dar es Salaam: Department of Policy and Planning, Ministry of Health, United Republic of Tanzania; 2001.
6. Nsubuga P, Eseko N, Tadesse W, Ndayimirije N, Stella C, McNabb S. Structure and performance of infectious disease surveillance and response, United Republic of Tanzania, 1998. Bulletin of the World Health Organization 2002;80:196-203.
7. United Republic of Tanzania. Poverty monitoring master plan. Dar es Salaam: Government Printers; 2001.
8. Lippeveld T, Sauerborn R, Bodart C, editors. Design and implementation of health information systems. Geneva: WHO; 2000.
9. Bodart C, Sapirie S. Defining essential information needs and indicators. World Health Forum 1998;19:303-9.
10. Japan International Cooperation Agency. Assessment of data collection system(s) and use in planning at regional, council and sub-council levels. Dar es Salaam: Japan International Cooperation Agency; 2002.
11. Bureau of Statistics [Tanzania] and Macro International. Tanzania Demographic and Health Survey 1991/1992. Dar es Salaam and Calverton, MD: Bureau of Statistics and Macro International; 1993.
12. Bureau of Statistics [Tanzania] and Macro International. Tanzania Demographic and Health Survey 1996. Dar es Salaam and Calverton, MD: Bureau of Statistics and Macro International; 1997.
13. Weinstein KI, Ngallaba S, Cross AR, Mburu F. Tanzania knowledge, attitudes and practices survey 1994. Dar es Salaam: Bureau of Statistics, Planning Commission, United Republic of Tanzania; 1995.
14. Bureau of Statistics [Tanzania] and Macro International. Tanzania Reproductive and Child Health Survey 1999. Dar es Salaam and Calverton, MD: National Bureau of Statistics and Macro International; 2000.
15. National Bureau of Statistics, Tanzania. Household Budget Survey 2000/01. Dar es Salaam: National Bureau of Statistics; 2002.
16. Angeles G, Stewart J, Gaete R, Mancini D, Trujillo A, Fowler C. Health care decentralization in Paraguay: Evaluation of impact on cost, efficiency, basic, quality, and equity. Baseline report. Chapel Hill: Carolina Population Center, University of North Carolina at Chapel Hill; 1999. MEASURE Evaluation Technical Report Series No. 4.
17. Health Research for Action. Review of health management of information system (HMIS/MTUHA) and Adult Morbidity and Mortality Project (AMMP). Reet: Health Research for Action; 2000.
18. Rommelmann V, Setel P, Hemed Y, Mponezya H, Angeles G, Boerma T. Costs and results of information systems in Tanzania 2003. No. 1. Description of indicator coverage and systems with preliminary comparative costings. Chapel Hill, USA and Dar es Salaam: MEASURE Evaluation, Carolina Population Center, University of North Carolina at Chapel Hill, and The Adult Morbidity and Mortality Project, Tanzanian Ministry of Health; 2003.
19. Ministry of Health. How representative are pooled sentinel area data for national estimates? A comparison of demographic and housing indicators from a national sentinel population with alternative data sources. In: Ministry of Health United Republic of Tanzania, editor. The policy implications of Tanzania's mortality burden: a ten-year community-based perspective. AMMP-2 final report. Volume 1. Dar es Salaam: Adult Morbidity and Mortality Project, Tanzania Ministry of Health, University of Newcastle upon Tyne, England, Department for International Development; 2004, pp.122-48.
20. Kobb D. Piloting a national monitoring and evaluation system. version two. Dar es Salaam: Urban Authorities Partnership Programme and Local Government Reform Programme; 2000.
(Submitted: 20 November 2004 Final revised version received: 16 May 2005 Accepted: 19 May 2005)
1 Correspondence should be addressed to this author (email: psetel@unc.edu).