Abstract

Introduction

“Lessons learned” is an expression that currently forms one of the chapter headings of the majority of social policy evaluation papers. Such a chapter sets out the positive and negative results of the development of projects or programs, on the basis of which some generalized conclusions are reached.

This article focuses on the evaluation of 18 Integrated Centers for the Care and Prevention of Violence against the Elderly (ICCPVE), which we monitored stage by stage over a period of three years (2007 to 2009) during the introduction, implementation and results of the programs.

In evaluating social programs, it is common to refer to the desired objectives in terms of efficiency, efficacy and effectiveness. Efficiency refers to the competence with which the appropriate resources are used to achieve results with the least expenditure of money and effort. Efficacy may be defined as the achievement of the desired results under controlled conditions; and effectiveness refers to the achievement of the intended results.

We work with the concepts of monitoring and evaluation. Monitoring may be understood as a stage or component in the evaluation process. Monitoring is carried out on the efficiency of the introduction and development of a project and also on the efficacy of actions and the effectiveness of results. Monitoring is practiced using indicators that show whether a particular proposal has been put into effect as desired or whether it is necessary to change the course of its development. Generally speaking this monitoring is recorded in brief reports containing: an analysis of the situation; details of investments made; identification of problems and the search for solutions; an assurance that the planned activities are being executed correctly; and information on any problems encountered with all or part of the proposal.

A complete evaluation comprises the whole cycle of analysis of an initiative, beginning before the process starts (ex-ante), passing through the monitoring of the introduction and the implementation, and ending with an analysis of the results and effects produced (ex-post). An evaluation may also focus simply on a certain moment or stage in the initiative. We focus here only on the question of the lessons learned from the introduction and implementation of the ICCPVE. However, even though we are analyzing only part of the process, it is necessary to look at the origin, the history and the objectives of the program and the context in which it was developed. We set out the lessons that the experience of the ICCPVE holds for the application of similar public policies, preceded by a short description of the proposal, its objectives and the methodology used in its formulation.

The Project

The Integrated Centers for the Care and Prevention of Violence against the Elderly were created in 2007 by the National Secretariat for Human Rights/Brazil (NSHR), on a cooperative basis with municipal and state governments and non-governmental organizations. They were set up as a strategy for action under the National Plan for Confronting Violence and Ill-Treatment of the Elderly.

These Centers employ a multi-disciplinary team of social workers, psychologists and lawyers to look after the needs of the elderly. Their activities are primarily concerned with guidance on human rights for old people who have been the victims of violence; the psycho-social examination and diagnosis of situations; mediation in family and community conflicts, with a view to heightening social awareness of aging and violence; encouragement of harmony groups; monitoring of aggressors; referral of specific cases of ill-treatment, abandonment or neglect to specialist services such as the Public Defender’s Office, the Public Prosecutor’s Office, Health Services, Police or Social Services; admission of elderly persons to long-stay institutions; and investment in the training of staff to deal with such questions.

Between 2007 and 2009, 18 ICCPVEs were opened in Brazil; eight in the Northeast region, three in the North region, four in the Southeast region, two in the Southern region and one in the Midwest region. Most of them do not have their own infrastructure, being based in State Departments for Justice, Public Safety, Human Rights and Citizenship, Public Defender’s Offices, Social Aid and Development and Defense of Rights.

The evaluation was based on an understanding of the objectives and aims of the Centers. The parameters for the process were the Resolutions of the 1^st National Conference on the Rights of the Elderly, the Old Persons Law and the guidelines of the Action Plan for Confronting Violence against the Elderly for the 2007-2010 four-year period. These documents contain an analysis of the principal problems of violence faced by the elderly and guidelines for the organization, control, monitoring and assessment of programs to be carried out by public authorities.

Methods used in monitoring and evaluation

The evaluation study was based on the triangulation of quantitative and qualitative methods.

The monitoring and evaluation process was constructed gradually on the basis of various participating and inter-sectorial workshops, involving all the participants in the social programs. This procedure represents the most up to date practice in the evaluation of social policies, known as “fourth generation evaluation”. The assumption behind this approach is that any evaluative process is more efficient and trustworthy when it is conceived and carried out in the form of a shared learning process that is of benefit to all, including the researchers.

At the first of the workshops, the strategies for monitoring and assessment were established and were coordinated with the assistance of NSHR and local units in order to set the correct course for the process.

This stage saw the creation of exploratory and management tools for following and analyzing the introduction and implementation of services, examining the initial history of each Center; its institutional and administrative viability; acceptance of the proposal by the managers; compatibility between the situation on the ground, local policies and the Action Plan for Confronting Violence against the Elderly; problems encountered; strategies for overcoming difficulties; and infrastructure and human resources available. A form containing open and closed questions, which was discussed with the coordinators of the Centers, was sent by email for completion by managers and their teams. The data obtained was analyzed for each unit and for the Centers as a whole.

In two other workshops, one in 2007 and another in 2008, agreement was reached on the qualitative and quantitative assessment indicators, by means of the “nominal group” method. The indicators agreed in the discussions were: structural: sustainability, team stability and continuous training; procedural: coordination, referral and counter-referral, partnerships, networks and inclusion; and results-oriented: preventive activities, attendances, cases resolved and recurring cases.

The results of the monitoring were published in the form of bulletins every four months during the years 2008 to 2010, highlighting local situations and comparing the units.

The actual evaluation process, carried out by means of qualitative and quantitative tools, covered the stages of the introduction and implementation of the Centers. The “Introduction Evaluation Form” was applied to each of the existing services and those that were set up from 2008 onwards. The answers were analyzed and the results discussed at a workshop in the presence of the Center coordinators.

The evaluation of the implementation process was carried out in two stages: in 2008, we emphasized the study of the successes and difficulties in the development of the work, in 2009, the results of action and hopes for the future. During this process, we made use of three forms with open and closed questions and we visited all the Centers. The aim of these visits was to observe activities at first hand and to interview the various personnel involved in the program (managers, staff, partners and elderly people). In all, we carried out 48 individual interviews with managers and partners, 42 focus groups with staff and 14 with elderly people.

At the final stage of the evaluation, we submitted the analyses we had made of the whole monitoring and evaluation process to the critical examination of the main personnel involved, during the course of a seminar held for this purpose. A preliminary report containing a summary of the results was sent in advance to coordinators, staff, NSHR representatives and partner institutions. After confirmation by these parties, the final report with the results was completed.

Results

Although the focus of this article is on “lessons learned”, we believe it would be useful for the reader to be supplied with some brief information on the results of the evaluation based on the indicators for the productivity and importance of the Centers. After this, we discuss the lessons that the introduction and implementation process has for similar programs, emphasizing the importance of this service, the successes and difficulties, and most of all the reasons that led to both of them.

Indicators for the productivity and importance of the Centers

Table 1 shows indicators for the productivity of the Centers in 2008 and 2009.

Preventive actions, which may be described generally as awareness-raising programs, normally involve large audiences at markets, civic and religious festivities, schools, radio and television programs, newspaper articles and open seminars, all of them focusing on the aging process, the condition of the elderly and the violence that they suffer.

With regard to the number of attendances, it is important to make clear that each Center was set up at a different time, and perhaps for this reason the oldest Centers, such as those of São Luis, Manaus, Rio Branco, Teresina and Florianópolis tend to show indices of greater efficiency. However, centers created subsequently, such as those of Palmas, Recife and Campinas, also show high productivity figures. Other factors influence the number of attendances, such as the size of the city (Cornélio Procópio) or the closing of Centers during the monitoring period (Natal, Belo Horizonte).

“Cases resolved” were those in which the elderly person’s needs were fully met. The resolution rate was high in 38.9% of the Units. However, in the majority, it remained below 50%. Referrals represented the greatest difficulty for staff. Public services are not prepared to receive elderly persons and do not normally produce results when they attend them. Elderly persons who are physically, mentally or socially dependent and who need immediate answers meet with great difficulties in being attended. We found a low proportion of recurrence in cases resolved by the Centers, which is testimony to the effectiveness of their action.

Quantitative indicators of productivity were supplemented by an analysis of the views of the different personnel involved in the work of the ICCPVE. Listening to people on the programs in which they are involved is important, because what they think about the situation influences the way they behave.

Some managers and professional staff stressed that elderly people leave the Centers calmer and more aware of their rights; they feel protected, secure, supported and satisfied. Many of them said that the Center is a service that is here to stay in their municipality and underlined the gains made through them: the mobilization of public and media opinion, the coordination of a network of local protection and the involvement of the community. The representatives of partner institutions also emphasized that, in the majority of cases, the Center succeeds in meeting the demand with an effective response. They also stressed the importance of the active and committed participation of the workers, many of them volunteers, and the fact that the multi-disciplinary work has made the group cohesive and integrated.

For the partners in NGOs and other public institutions, the service provided by the Centers has become a reference point for the citizenship of the elderly and an initiative at the forefront of the topic, as regards the places where they have been established. Prosecutors, police and public defenders say that there has been a reduction in the number of complaints lodged with them, which is evidence of the success of the Centers.

The elderly people interviewed considered that they had been well attended, because the professional staff listened to them, referred them in a correct manner and resolved their problems. They commented that they felt respected, at ease and calm after being attended. However, a number of them criticized the bureaucracy and queues at the services to which they were referred, and the often inadequate physical space at the Centers, which made it difficult to ensure privacy during the attendance and resulted in delays as regards legal action. For these reasons, many elderly people suggested improvements at the Centers, such as more space so that attendance could be conducted in private; the abolition of queues for attendance at the services to which they were referred; more visits by staff to their homes in cases of complaints of violence; more rigorous penalties for aggressors; a more flexible legal system; the establishment of Centers in accessible locations; and means of transport so that dependent elderly people could be attended at home or taken to appointments at other services.

Lessons learned from the introduction of the Centers

The time of introduction is crucial because on it will depend whether the collective practice is efficient or not. Each Center has its own history as regards its foundation, but the points in common stressed as making progress easier were: (1) workshops offered by NSHR for the professional staff of the Centers to discuss the feasibility of the work, create a common language and suggest guidelines for routine activities and more complex problems. This strategy made it possible for people to identify and work with the proposals, to obtain accurate and objective information with regard to the introduction of the Center, to ensure its operational capacity and to discuss the indicators which would be used to evaluate the services; (2) the formation of a group of dedicated professional and managers, with the basic qualifications and previous personal and institutional experience in working with the elderly; (3) financial support and technical guidance from NSHR in providing the Center with the basic conditions for operating in places where the work represented a pioneering effort; (4) acceptance by an institutional entity so as to provide the Center with premises and operating conditions; (5) provision of basic infrastructure for the project: adequate physical space; equipment; telephone and internet facilities; transport for home visits and for dependent elderly people; and rooms for individual attendances; (6) location of the Center where it can be easily accessed by elderly people using public transport; (7) autonomy for the coordinator to adapt and organize the workplace and to select the team; (8) the building of active partnerships with health services, social security organizations, non-governmental institutions and the voluntary sector.

Among the factors which made the introduction of some Centers more difficult, managers and staff identified the following: (1) scarcity of financial resources needed to carry out the number of activities planned, because some units were entirely dependent on the subsidy from NSHR; (2) bureaucratic problems leading to delays in the bidding processes for acquiring materials and equipment, or the contracting of technical personnel and companies involved in the programs; (3) management problems in the entity to which the unit was linked, or changes in the management of the Centers themselves, which made for discontinuity in attendances and referrals; (4) non-existence or inadequacy of services and programs in the support network to which elderly people must be referred. This is the case with respect to day care centers and places in long-stay institutions for elderly persons with a high degree of dependency; (5) problems of coordination between staff of different specialisms (law, health, social work, security and transport) and the entities which should be cooperating; (6) difficulties in raising public awareness with regard to questions of aging and violence against the elderly.

Lessons learned from the implementation of the Centers

Despite the problems involved in setting up a multi-disciplinary project, the development of activities in the ICCPVE witnessed important achievements, which were supported by managers, professional staff, partners and the elderly themselves.

The following points proved to be highly efficient: (1) mission focus; (2) a group of professionals who were qualified, motivated, cohesive and committed to their work; (3) multi-disciplinary attendance, including legal, psychological and social support; (4) partnerships with public institutions responsible for services (such as departments of health, social security, transport, urban mobility) or with supporting institutions such as universities, rights organizations, public prosecutions offices, federal and state justice departments, the National Network for the Rights of the Elderly, and voluntary groups; (5) the provision of personalized and direct attendances for the resolution of problems, which means receiving and listening to the elderly person, making any necessary referrals, involving the family in the resolution of problems and strengthening its links through a comprehensive mediation of its conflicts; (6) accompanying the activities of the Center with courses to improve professional skills and family guidance; (7) accompanying the development of the Center with the production and distribution of informative material to raise social awareness; (8) participation in public discussions in support of the rights of the elderly; (9) where necessary, the decentralization of activities to municipalities and districts with large numbers of old people; (10) political and administrative support and financial assistance from local government.

The above ten points should be supplemented by noting the importance of having the participation of old people and their families in developing policies. This helps in ensuring that the Centers provide suitable services and leads to less litigation over conflicts in which they are the victims.

Certain problems that obstruct the work were identified. Some are institutional, others are problems of management, and yet others refer to how daily activities are carried out.

From the institutional point of view, the main failings were: (1) the discontinuity of the agreements with the NSHR, which guided the programs and financed the Centers. Because of the interruptions, some units ceased to operate, because they did not succeed in being taken over by local government. Others continued to exist, but they lost their multi-professional focus and ended by operating in a single area, contrary to the original plan; (2) problems of infrastructure continued to be evident in some of the Centers (lack of adequate physical space to ensure privacy in attendances), lack of essential equipment, such as telephones and transport for dependent old people and for home visits, and shortage of personnel, not only as regards insufficient numbers of professional staff but also as regards high staff turnover; (3) in various units, excessive bureaucratic delays by entities to which they were subject prejudiced the carrying out of programs, the hiring and payment of professional staff and the bidding processes for the purchase of materials and equipment.

From a management point of view, the main problems were: (1) in some areas, an obvious lack of coordination in the support network and a lack of services which should ensure full attention to the needs of the elderly; (2) failings in the system of referrals and counter-referrals which hindered the resolution of cases of violence; (3) non-existence or insufficiency of places in long-stay institutions for elderly persons, meaning that old people who are dependent, living in extreme poverty or do not have family carers are left without support; (4) difficulties in securing mental health treatment for old people or abusers with psychiatric problems; (5) lack of training for professional staff in dealing with old people and families with psychiatric problems or involved in criminal activities; (6) lack of opportunities for working with aggressors; (7) delays in the processes for adjudications of mental incapacity and compulsory institutionalization for old people with dementia and no proper carers; (8) non-existence of unified databanks for the protection network.

From the point of view of the teams and their activities, professional workers mentioned the following problems: (1) lack of experience in dealing with violence against old people; (2) discontinuities in the processes of contracting and training Center personnel; (3) high turnover of professional staff and trainees.

As regards the elderly and their families, professional workers highlighted the following: (1) reluctance of old people to speak about the violence that they suffer, either because the abuser is a family member with whom they will have to live when they return home, or through the fear of a possible involvement of the police or the criminal courts in the situation; (2) the mental disturbance or chemical dependency of some old people and their abusers; (3) the frequent lack of understanding and commitment by families as regards the needs of their elderly relatives.

Discussion

There is evidence of successes and difficulties in the development of the ICCPVE, which were conceived as a strategy for constructing a platform of knowledge and practice to deal with violence against old people and to guarantee their rights. It should be remembered that, although we speak of old people in a general way, in reality the most frequent visitors to the units are poor people who have some difficulty in standing up for themselves. It is this specific group that public institutions and Brazilian society have experienced the greatest difficulties in protecting. But this is not a uniquely Brazilian problem, because in documents published by the Council of Europe, which represents the thinking of the European Community, the focus is on the dependent elderly, who are considered the most vulnerable. In Europe, it is proposed to create a specific fund for the benefit of this group, because the services targeted at the group are sometimes run by public entities and sometimes by non-governmental organizations. The main challenge has been to balance the responsibilities of the family and the State in caring for the elderly, by expanding access to specialist services and by relieving the burden placed on family carers.

Although the ICCPVE seemed to show the way towards possibilities of the kind mentioned above, their role has now faded. As often happens with public services in Brazil, changes in the management of the NSHR have established other priorities, making the Centers just one more experiment. In a survey carried out in 2013, we noted that of the 18 Centers evaluated, six had closed down due to lack of financial resources and local management support. Of the remaining 12, nine were still functioning in accordance with the multi-professional model originally conceived. The others were continuing to offer their services only in certain areas. Two Centers (Rio de Janeiro, São Luis), however, had expanded their activities within the planned philosophy, the former by creating a complaints department and by starting a unit in Niterói, and the latter by setting up a mobile facility to attend to the outlying areas of São Luis.

The proposed creation of the ICCPVE was one of several strategies devised under the National Plan for Confronting Violence against the Elderly. In this sense, it forms part of a whole and as such it has fulfilled its mission. It was for the purpose of drawing general conclusions from this strategy that the NSHR commissioned the monitoring and evaluation research in order to follow the course of the experiment.

The question that arises here is whether initiatives such as the ICCPVE make sense if they are not genuinely institutionalized. International experience shows that every country has been seeking ways of dealing with the problems produced by an aging population. However, we note that the question of dependent old people and the violence to which old people in general are subject continue to be subjects of concern to governments, health professionals, researchers and, above all, families. The ICCPVE initially created as a pilot experiment for quality, efficiency and low cost, with the aim of expanding nationally, could still be seen as an example for countries that are seeking solutions to the problems that have been mentioned.

Articles which have evaluated proposals for the prevention of violence against the elderly show that an effective program must meet the needs of each elderly person and embrace the various aspects of his or her life, such as health, education, social interaction and living conditions, providing relevant services which will lessen the risks of violence. It should also include initiatives such as media campaigns and strategies for changes that can improve the living conditions of the elderly. These objective reflect the original proposals for ICCPVE, even though there have been no corresponding initiatives. In 2007 the United States, for example, established the National Center on Elder Abuse (NCEA) within the Department of Health and Human Services to ensure funds for research, the application of social, health and legal policies, the training of carers, the defense of the rights of the elderly, and attendance on families in need. This Center operates in collaboration with the States and with local initiatives. In Canada, there is a Department for the Elderly within the Public Health System, a National Care Network, a Law Center for the Elderly and a Network for the Prevention of Abuse.

Final considerations

We have learned certain lessons from studying the evaluations in this case:

1. The first lesson is that when a new strategic arrangement is made, such as a project for the development of a public policy, the institution responsible must have a clear focus and defined objectives, and must work closely with all the personnel who will be implementing the project and all those who will be affected by it. However, this is not enough, there needs to be financial resources, technical guidance and foreseeable sustainability. In the case of the ICCPVE, there was a large initial investment by the NSHR, followed by a lack of continuity.

2. We concluded that 66% of the Centers that still exist enjoy local support and are pursuing their own program. Nevertheless, the NSHR target of establishing at least one Center in every state has not been achieved. It is our view, therefore, that the guiding role of the NSHR should not have ceased until the project had been fully established.

3. One of the crucial problems as regards the continuity of the Centers lay in the fact that the project for their creation was never fully agreed between the NSHR and local authorities. During the evaluation process, it was noticed, on the contrary, that there was a great deal of internal resistance and discordant views with regard to taking over the projects and incorporating them at the local level. We noted that, in practice, plans for the care of the elderly are not a priority for some authorities or for some sections of Brazilian society, except for the families concerned.

4. We stress the importance, if the project is to be taken up again, of ensuring that the Centers are provided with their original multi-professional staff and operate on a network basis. The results achieved by a small team comprising a psychologist, lawyer and social worker, as shown by the indicators, are by themselves evidence of the importance and appropriateness of such a format.

5. The contracting of professional staff and managers was one of the successes of the Centers. At the end of the evaluation process, many coordinators and staff stressed that, even though it was a new project, the efforts and performance of the employees ensured that the services became a reference point leading to effective and important achievements.

6. The role of the Centers was fundamental in establishing an inter-institutional network of necessary and essential services, such as health and legal assistance, which are unfortunately the most difficult to access and which involve delays and poor quality in their provision. We stress again the inadequacy of specialist care for dependent old people with serious physical health problems, and the difficulties of access to mental health services.

7. There are four problems with regard to the day to day running of the Centers that jeopardize their work: the lack of any guarantee of continuity and sustainability on the part of the local authorities working in collaboration with the NSHR; some current policies mean that the Centers serve only certain sections of society; inadequate financial resources for the activities of the Centers; and a lack of investment in qualified personnel, and their replacement by inexperienced staff.

References

Publication Dates

History