In the specific field of evaluating quality of health care services, an integral part of evaluating health programs and policies, one also observes, in many countries, relevant changes in quality evaluation relating to the changes identified by Hartz vis-à-vis program and policy evaluation. Such changes result from the realization that current health care production requires quite complex systems and processes, and thus that the results of patient care are largely explainable by problems in these systems and processes and not merely by the performance of a specific health care provider (physician or nurse). Since improved performance by health care organizations depends fundamentally on actions developed by the organization itself, regulatory and quality evaluation agencies should act to motivate organizations to improve their performance, leaving sanctions for those cases in which their is evident risk to the public. An example of this new approach was led in the early 1990s by the Joint Commission for Accreditation of Health Care Organizations (JCAHCO), a traditional American institution in this field. The JCAHCO redefined its evaluation focus, concentrating more on seeking health care services' conformity to standards targeting processes and activities with a direct or indirect effect on patient care, improving their communications with health services and emphasizing their educational role, in addition to transmitting evaluation results to the public.

Despite agreeing with the concept of evaluation adopted by the author, I would like to introduce an alternative to the emphasis placed on institutionalization in her article. In expanding the concept of evaluation, I wish to do so with caution in order not to underrate the importance of research for the effectiveness of the evaluation process. In the specific case of health care services, evaluating quality of care is an extremely difficult task, given certain characteristics of physician practice, such as the fact that it is based on specialized knowledge, permeated by uncertainties with regard to a major portion of the available diagnostic and therapeutic procedures, and the fact that patients present broad variability in the risks of developing adverse results. Research in health care services is a relatively new interdisciplinary field still dealing with important technical and methodological limitations, despite advance achieved in the last two decades (Brook, 1996). In addition, in this new paradigm, the way the results are summarized and publicized gains relevance, which implies constituting evaluation teams with new competencies, such as communications, teaching, and policy-making.

Thus, from my point of view, institutionalization of evaluation in the health sector should focus not only on improving interaction between decision-makers, evaluators, and health care managers and providers. It should also consider strategies and resources needed to encourage the production of knowledge and training specialized personnel. Research producing imprecise results leads to conflicting interpretations and fails to produce discernment of the facts analyzed. It can thus generate discredit over the usefulness of evaluation activities, with a negative impact on the value ascribed to them by decision-makers, health care managers and providers, and users.

Although in the French case the process of institutionalizing evaluation began late and was less prominent than in other countries cited by the author (due to the French political and academic culture and not a lack of specialists), I believe this was not so for Brazil. Our Brazilian reality is more complicated: we lack a political culture oriented towards evaluation, and I think that despite recent efforts, we still experience a chronic and severe lack of academic and technical specialists in quality evaluation, quality management, production, and analysis of health data, medical documentation, and so forth. Thus, as pointed out by the author in quoting Gérard de Pourvourville, we should search for shortcuts, learning from the experience of other countries, but shaping it to our mesures, which unfortunately still express a multiplicity of deficiencies.

BROOK, R.; McGLYN, E. & CLEARY, P., 1996. Quality of care. Part 2: Measuring quality of care. New England Journal of Medicine, 13:966-970.