PERSPECTIVES

Evaluating physician competence

Avedis Donabedian1

What comes immediately to mind, in evaluating physician competence, is a jumble of methods that jockey for position and clash in what, at times, seems to be a battlefield of contending parties. We must return to first principles, if we are to understand this surface confusion and choose wisely from the many methods offered to us. And perhaps the most fundamental question to ask is: "Competence to do what? What are the objectives, and what are the preferred means for attaining these objectives?". I shall attempt an answer under three headings: the area of responsibility, the level of responsibility, and the means.

The area of responsibility is, itself, a complex and expanding field. At its core is the management of physical illness through the application of the science and technology of medicine. It follows that the degree of mastery over this science and technology is at the heart of competence; but by no means does this define its totality. The successful management of physical illness often requires an extension in the content of medicine to include the management of cognitive, other psychological, and social factors. This is because these factors may have a role in the causation or persistence of physical illness; and, even when this is not the case, their appropriate manipulation is often necessary for success in applying the science and technology of medicine. In addition, physical illness creates in the patient a variety of psychological needs that he expects to be satisfied in the patient–physician interaction. In these ways and for these reasons, the question "Competence for what?"acquires new meaning and makes new demands on the physician and on the organization within which he works.

A shift in focus from the management of physical illness to the management of physical health adds another conceptual dimension and brings about further extensions in the criteria of technical, psychological and social management that constitute competence. And this is not all. Parallel to the progression from physical illness to physical health there are the analogous areas of responsibility for mental illness and mental health, and for social illness and social health. At its fullest extension, the area of responsibility embraces "physical, mental and social well-being", as envisaged by the definition of the World Health Organization. This may represent an extension to absurdity, and I do not necessarily advocate it. All I am saying is that we must define the content and boundaries of our legitimate responsibilities before we are able to determine competence to discharge those responsibilities.

The levels of responsibility offer a similar prospect of progressive inclusivity and complexity, with the added threat of moral ambiguity and conflict. The primary responsibility of the physician is for the individual patient. At this level, the objective is to find and implement the optimal solution to the balance of benefits and risks in medical management. Selecting the appropriate context within which this is to be done and arriving at the optimal solution is the essence of clinical judgement; and assessing clinical judgement is the essence of evaluating physician competence. Anything less is peripheral, incomplete, and possibly dangerous.

However, the physician is responsible not only for an individual but also for a case load. This must place a limit on the amount of time, attention, and possibly other resources that can be committed to each patient. Thus, clinical judgement requires identification of relative priorities among patients. In this way, the physician becomes a major instrument for resource allocation; and the optimal resource allocation solution becomes a part of the quality of care. If so, a case-by-case assessment of competence may fail to identify an important element in aggregate quality. In private practice, each physician is responsible for resolving whatever contradiction might arise between optimal strategies for individuals and for the case load, by limiting the size of the latter. In organized practice, the decision might, to varying degrees, be taken out of his hand. If so, this must be explicitly recognized and acted upon in assessing competence.

Similar considerations arise if the physician, individually, or the organization, as a collectivity, are responsible for a target population. Here, care is doubly rationed, first by differential access, and then by differential management after access is achieved.

Analysis at the societal level includes all the considerations already introduced; but societal concerns may alter the relative emphasis to be placed on them and may add elements that are altogether new. In fact, medicine may be used to pursue collective purposes that sometimes run counter to individual interests, introducing serious conflict in the patient–physician relationship. The present cry for greater responsiveness to collective needs has serious dangers. Society and humanity are not synonymous; and the primary commitment of medicine to humane objectives may require intransigent opposition to organized collectivities, even when they purport to speak on behalf of individual patients.

One consideration that cuts across all levels of responsibility is that of cost. To the extent that individuals bear the cost of care, the optimal benefit–risk solution must take cost into account. Third-party payments, by spreading and postponing the cost consequences of medical care decisions, afforded physicians, for a time, the luxury of paying little attention to direct monetary cost as an element in clinical judgement. Now the balance is restored, but with a vital difference. From now on we shall be asked, with increasing insistence, to consider the cost consequences of clinical decisions not to the individual patient, in any immediate sense, but to the collectivity, introducing yet another element of strain in the patient–physician relationship.

We see, then, that the assessment of physician performance must recognize limitations in the means available to him. We have already alluded to the limitations in financial resources and time. The limitations in the science and technology of medicine are too painfully familiar to practitioners to require elaboration. Another set of constraints derives from the social values and conventions that govern the patient–physician interaction. These are distinctive in that they are also objectives that help define quality in care and, therefore, competence in performance. They embrace considerations that include privacy, confidentiality, avoidance of exploitation, informed consent, and the appropriate degree of participation in decision-making.

It is clear that what constitutes competence is dependent on prior specification of a social–organizational context for assessment. What remains to be emphasized is that the choice of the appropriate context is not a matter than any physician, or even any profession, can determine unilaterally. At one level it requires the consent of the individual patient and his physician; at another level it requires legitimacy through social consensus. But, at any level, physicians must insist that they cannot be held responsible for functions for which they have no special training and in areas over which they have no control. The acceptance of responsibility must be conditional on commensurate authority.

Having set the stage, we can now take a brief look at some methods for assessing physician competence. These may be conveniently classified as assessments of attributes, activities and accomplishments.

The attributes that are usually offered as indirect measures of competence describe education, training and experience. While these are significant factors in competence, it is clear that they fail to explain a great deal of the variation in observed performance. There are additional personal attributes that are poorly understood which are at least equally important. In addition, the organizational settings within which physicians practise, through a variety of incentives, can powerfully influence performance. The interaction of personal and organizational attributes is highly complex, and much more needs to be known before we can begin to predict performance, with an acceptable degree of confidence, from such attributes. For now, we must place greater emphasis on the assessment of activities and accomplishments.

Under the rubric of activities, quality is defined as behaviour calculated to accomplish appropriate objectives in clinical management. Everything that a physician says or does in his work is subject to informal assessment by his patients and coworkers; and no physician who has a proper regard for the opinion of others is immune from this influence. Hence, this informal mechanism for assessing and shaping behaviour needs to be carefully nurtured. To function properly, the informal network requires maximum visibility of practice through proper recording, consultation, conferences, teaching, team care, rotation of responsibility for patients, and the like. The network should also be extended to include branches that are often isolated, though they carry much information. These include colleagues not directly involved in patient care (such as radiologists and pathologists), other health professionals such as nurses and pharmacists, and, of course, the clients themselves.

However, important as the informal network of assessment and influence may be, it cannot stand alone. An apparatus of formal assessment is necessary to assure fairness, predictability, stability and legitimacy. Nothing is more destructive to morale than a procedure for assessment based on unverified impressions, using private criteria, that is fitfully and selectively applied.

There are two categories of methods for the formal assessment of physician activities: the assessment of performance in test situations, and the assessment of performance in actual practice.

Testing for competence is a broad and complex subject in which I have no competence. It seems, however, that testing for knowledge alone is insufficient. The test situation should be so constructed as to elicit clinical judgement and problem-solving skills. Ability to elicit and interpret sensory data (for example, in auscultation) should also be included. It is likely that even with the availability of multimedia productions the assessment of actual patient care will remain the ultimate test.

Actual practice may be assessed by observation, either directly or through videotape; by records of care kept by the physician, other professionals and, even, the patient; by interviews with the physicians, or questionnaires; and by formal ways of obtaining the opinions of other knowledgeable persons in the informal network to which we have already referred. Each of these methods has uses and limitations which are well known, so I will dwell only on some points.

It is clear that good records are a fundamental necessity. They should not only list what was found and done, but also allow a reconstruction of the physician's thinking, since our object is to assess clinical judgement and not mere compliance with stereotyped sequences.

In this respect I believe that lists of explicit criteria threaten to lead us down a blind alley. It is not true that "I have greater quality than thou because my criteria lists are longer than thine". The criteria serve a useful function in sounding an alert that something may be wrong. It would be tragic to accept them as representations of quality, except in the crudest sense. To repeat, quality resides in the exercise of appropriate judgement in the pursuit of appropriate objectives. Assessments of quality can be made valid, reliable, and explicitly defensible only if one is able to specify the objectives of care, to specify alternative strategies for attaining these objectives, and to evaluate each strategy using specified criteria. Fortunately, we are now in the midst of a revival of interest in clinical decision-making. The fruits of this research will not only be tools for quality assessment but, more importantly, better ways of teaching clinical skills. Ultimately, better understanding of the relative efficiency and effectiveness of alternative strategies of care will have an impact on public policy in the health field, in the broadest sense.

The assessment of alternative strategies of care requires linking a logically connected sequence of activities to specified outcomes of care. The measurement of these outcomes is, therefore, a necessary component in the assessment of quality. Depending on the scope and level of responsibility that is accepted, the outcomes selected may include aspects of physical, psychological and social function, measured in individuals or target populations. Whatever outcomes are selected for assessment, certain conditions need to be satisfied.

• The outcomes selected should be relevant to the objectives of care.

• The outcomes must be achievable by good care; which means that the instrumentalities are available, and that these are under the control of the physician.

• The outcomes, whether good or bad, must be attributable, first, to medical care and then to the physician's contribution to that care.

• The duration of the outcomes as well as their magnitude must be taken into account.

• As a corollary, the tradeoffs between levels and durations of alternative outcomes need to be considered. For example, a shorter life at a higher level of function may have to be weighed against a longer life with greater disability.

• As another corollary, information on the relevant outcomes must be available, which is not an easy matter, especially when it requires follow-up over long periods.

• It is necessary not only to examine the consequences of taking action but, also, the consequences of not taking action, in order to obtain a complete picture.

• Finally, the outcome cannot stand alone. The means used in achieving the outcomes have also to be considered, unless it is assumed that resources are unlimited, which is far from true.

These are demanding requirements that are not easy to meet. Attention now focuses on indices that combine the impact of death and different levels of functional disability on total populations. However, the same methods could be applied to measuring survival under treatment for specific conditions, not only in terms of longevity but also of the functional competence and other aspects of the quality of the increment of life that is achieved. Given a measure such as this, coupled with a clear model of the available strategies of care, we would be well on our way to finding a rational and clearly documented answer to the problem of defining and assessing physician competence, as a component of the more inclusive constellation that defines quality of care.

Let me conclude with a few summary remarks.

• There is now no one best method for assessing physician competence. We must rely on a system of assessment that includes attributes, activities and achievements. Our quality assurance system must also include attention to all three components. In particular, health care programmes must not be restricted by structural and process standards, and we must be unalterably opposed to such suggestions so that these programmes can experiment in new and more efficient ways of achieving comparable outcomes. The search for more efficient ways of achieving given outcomes is a major research undertaking which must be conducted with rigorous controls by fully qualified and unbiased investigators. However, the fact that answers are being sought in the marketplace points out how seriously our medical centres have failed to meet a vital obligation.

• More important than the technical refinement of the system of quality assurance that is adopted is the commitment to quality which makes the system work.

• Physicians must insist that any system for quality assessment be congruent with a realistic view of what constitutes good care, their obligations to safeguard the interests of their patients, and the means at their disposal.

We are still in the infancy of the assessment of quality. Much more basic research is needed, especially in measuring health status, and in modelling the clinical decision-making process. The real breakthrough in quality assessment will come as a by-product of research in the relationship between strategies of care and outcomes, subject to clearly defined evaluative criteria that include measures of monetary and non-monetary costs and benefits, to individuals and to society. This marriage of clinical and social medicine opens up prospects of unparalleled intellectual challenge and excitement to the scholar, with a harvest of rich rewards to society at large. In this enterprise we are all invited to participate.

1 Nathan Sinai Distinguished Professor of Public Health Emeritus, University of Michigan, Ann Arbor, Michigan, USA. Correspondence should be addressed to Professor Donabedian at 1739 Ivywood Drive, Ann Arbor, MI 48103, USA. This lecture was delivered at a Conference on Assessing Physician Performance in Ambulatory Care, held in San Francisco, USA, in 1976 by the American Society of Internal Medicine