Professor Trevor Bailey (Professor of Applied and Computational Statistics, College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK).

Trevor Bailey is Professor of Applied and Computational Statistics at the University of Exeter, UK, where he was Associate Dean (Education) in the College of Engineering, Mathematics and Physical Sciences for several years and lecturer in Advanced Statistical Modelling. His highly cited book on spatial analysis was published, in 1995 ¹1. Bailey TC, Gatrell AC. Interactive spatial data analysis. London: Longman Scientific and Technical; 1995. and led us in Brazil to contact him and invite him here for the first time in 1996. His research interests nowadays are in general applied statistical modelling, but more particularly spatial-temporal epidemiology and environmental impacts on public health. Much of his research has been collaborative involving academics and professionals from a variety of fields such as medicine, public health, geography, computer science, behavioural science, education, commercial organisations and government agencies. Since 2015 he has been receiving a Capes fellowship as a Senior Visiting Researcher under the Science Without Borders program.

Carvalho Professor Trevor Bailey, you’ve engaged in research visits to Brazil for quite a long time. I think 1996 was your first visit. This long term collaboration was funded for the last two years by the Senior Visiting Professor Program from Capes and over some twenty years has been valuable for us in a number of ways. But could you tell us what it means for you? Why this long term cooperation - you wanting to do things together with us?

Bailey Well Professor Carvalho it is a genuine pleasure to talk to you today about this collaboration and other matters. I’ll admit there are some purely selfish reasons for the long cooperation - the UK can be somewhat damp, grey and dark at times and Brazil is none of those! But, more seriously, and as you know, I am an applied statistician with interests in public health and spatio-temporal epidemiology. So, in that professional context, I guess there are at least four reasons why my research relationship with Brazil has persevered for so many years.

The first is that Brazil is a country where there is enormous diversity of problems in the area, ranging from infectious diseases at one end, to chronic diseases at the other, including growing issues concerned with ageing in the population. And, of course, the geography in Brazil is enormous. One of my key interests is the environment and its impact upon the health - environment in the broadest sense including climate and infrastructure. And Brazil is probably one of the few countries in the world where the diversity of related problems is as wide. For example, we could be talking about malaria in the Amazon, dengue in the Southeast, diseases of the elderly or antibiotic resistance in São Paulo. There is an enormous diversity of interesting and important problems.

The second reason, somewhat to my surprise in the early days, is that the data availability in Brazil is actually quite good. Better than I’ve experienced in other countries where infectious diseases are a big problem, in Asia or Malaysia - I’ve done work on some problems there - or in Africa, for example. The data systems in Brazil are actually, I would say, first world. In some ways better than you might be able to get in some parts of Europe, for example. So I know there are always problems with data, but with the quality of integrated data available in Brazil one can actually make some progress.

Carvalho I agree. To obtain some data across Portugal, for instance, can be very difficult.

Bailey The third reason is the huge commitment to public health and associated research in Brazil. Just one example is the work that Fiocruz does around the country. OK, based in Rio, but also very active in other places in Brazil. More generally, the number of people involved in public health in general is substantial in Brazil. If you go to the Epi conferences, for example, you are talking about more than two thousand people.

Carvalho Which places have you visited in Brazil?

Bailey I have visited and worked with people in Rio, Recife, Belo Horizonte, Curitiba, Florianópolis, São Paulo, São José dos Campos, Campinas, Piracicaba. But I was talking more generally about the strong commitment to public health across the whole of Brazil.

The fourth reason is very simple. The people I’ve worked with in Brazil are not only extremely open and accepting and wanting to learn about the sort of things that I am talking about, but they are also exceptionally good scientists. I have learnt more from them than they have from me.

Carvalho It’s from 1996 to now, it’s twenty years.

Bailey Yes, twenty years - a long time. Sure, there are other issues as well, but those four reasons probably explain why my relationship and my collaboration, has continued for as long as it has. Visits haven’t been every year, but the last two years, since we have established this visiting fellowship, has enabled us to to get some more sustained stuff going. I am really very pleased about that.

Carvalho You’ve said that you are very interested in environmental aspects in relation to health. What about climate in particular? We have done two workshops on climate and health that you were involved with. What are your feelings about all this discussion on climate and climate change?

Bailey Climate and health is a complex subject. One of the interesting aspects is that many climate impacts on health are indirect and specific to the particular social and cultural context. For example, the biggest climate impact on health in certain African countries may be through drought and its influence on agriculture which is on a knife edge anyway. Failure in the food supply at a critical time could lead to enormous health impacts. Whereas, in Europe there is much more socio-economic sustainability and climate change is not liable to hit so hard. Once you start thinking about climate and health, you have to start thinking about systems. And I don’t think we are really there yet in terms of the way in which epidemiology in general is actually discussing these kinds of systems issues. Climate impacts certain aspects of the environment, but then you’ve got the mobility of people, you’ve got the infrastructure, you have the socio-economic context before you can unravel potential health impacts. The whole thing is wrapped together and sometimes the connections are hard to see. The traditional kind of epidemiological approach is not necessarily the way of thinking that is going to help in terms of the impact of climate on health. In investigating these systems we need more sophisticated models and we need to bring together different data sources in a much more integrated way than we’ve been able to in the past.

Carvalho I agree. Even dehydration, a typical example of a direct effect of heat on health, only affects people that were not able to respond, to drink enough water, to be taken care of. That’s a huge difference between the direct effects and the environment and the context of people’s life. So, what you’ve just said now is that we need a different way of thinking about problems that's not exactly the way we, as epidemiologists, have traditionally been doing. So, what do you think are the main pitfalls and problems in addressing these kinds of complex problems?

Bailey I don’t think we know the answers to that at the moment - I certainly don’t. But one thing is that we need to involve people with a mixed skill set. We are talking about people who understand aspects of computer science and “big data” issues, people who understand the medical and epidemiological side of things, we need people who can use GIS technologies, we need mathematical and statistical modellers and so on. All these people are around, but we need an environment where we can bring these kinds of people together. And the other thing we need in there is people who understand the climate and what is going to happen to the climate.

Carvalho Following on from what we have been talking about, as an Editor-in-Chief of CSP, we receive lots of papers that are essentially “more of the same”. We’ve published an editorial about this ²2. Carvalho MS, Travassos C, Coeli CM. Mais do mesmo? Cad Saúde Pública 2013; 29:2141-3.. The justification for the methodology is often no more than: “because everybody has done it this way then I will do this way”. It is one more paper along the same lines and using the same methodology that everybody has used before. On the other hand, to bring new stuff is much more difficult than just repeating what someone else has done. This is a real problem. I don’t think it is exclusively ours, in Brazil. I am not an editor in any other place, but it’s something that I’ve seen, not only in Cadernos, but in other health journals around the world. I would like to listen a bit more to your views on that - how do you approach a problem, how do you model a problem?

Bailey It’s not just in the health journals, actually. This whole issue of trying to justify the model that you are using is something that is not particularly well done. This is not just in Brazil and certainly not just in epidemiology. My undergraduates in the UK, for example, can be extremely good mathematicians, but often weak modellers. And the same is true to some extent of science students in general. One of the reasons is because students like the idea of deducing things from a set of rules and they are very good at doing that. If you ask a mathematician or if you ask a physicist - given this set of relationships or laws, then prove this result - then they love that and they can do it. But if you ask the opposite question, if you say: here is something going on in the world, what kind of framework of mathematical relationships might be responsible for this? You are switching from a deductive approach into an inductive approach, which of course you have to do in modelling, whether it is statistical modelling or whether it is mathematical modelling more generally. Then, students are not so good or so comfortable with it. Mathematical/Statistical modelling is indeed very difficult to teach, but maybe we spend too much time teaching students about the details of different models and not enough about creating and justifying them.

So, it is not surprising to me that when you get papers some authors do not spend enough time talking about why the phenomenon on which they are working should actually have certain kinds of behaviour which then equates with the modelling framework which they use. Instead of explaining why their modelling framework is the right one for use here, what they actually do is simply lift a pretty standard model off the shelf and then that drives the whole of the paper and the results. Sometimes a standard model may be appropriate, but even if it is then it still needs to be justified by the science and the epidemiology and not just because it has been used before or it is easy to implement at the touch of a button in a software package. As I have said, it is difficult to teach modelling whether it is to scientists, mathematicians, epidemiologists or whoever. It is difficult because it needs “hands on” experience to develop the skill. So my personal opinion is that we have to make more space in our masters, PhD and short course programmes for students to actually be able to build that experience over and above what they gain in their research dissertation. We need to flip from the individual standing at the front of the classroom delivering “modelling truth” to more emphasis on students discovering for themselves, through their own experience that actually “modelling truth” is rarely well defined.

Carvalho I agree, we thought a bit in my post-graduation programme ³3. Epidemiology Program in Public Health, National School of Public Health Sergio Arouca, Oswaldo Cruz Foundation, Rio de Janeiro, Brazil (http://ensino.ensp.fiocruz.br/ensino-english/cursos/mestrado-e-doutorado/epidemiologia-em-saude-publica).
http://ensino.ensp.fiocruz.br/ensino-eng... about bringing in, for instance, some old studies to give to the students and let them think about what they would do to investigate the problem.

But, perhaps I can move our discussion onto a possibly related area as to why researchers are tempted to “turn the handle” in publications and that is the huge pressure on them for productivity. I accept that we do not want the “job for life” culture that might have prevailed for professors and researchers in Brazil fifty years ago. However, I feel the opposite that you have to publish many papers maybe to some extent killing science, especially if it is just quantitative evaluation. In Brazil this “publish or perish” culture is still less heavy than it is in the USA; but nevertheless it is significant and growing. I wonder how is it in the UK, for young and for older researchers?

Bailey So the old “publish or perish” dilemma! Well, I am coming up to retirement in the UK. When I first started as an academic - my first job was actually in Australia, then I came back to the UK - the pressure on publication at that point was not anywhere near as great as it is for young people today. When I wrote my book on spatial analysis in the 1990’s I also wrote software to go with that book and I think the whole project took about two years.

Carvalho This book was a bestseller in Brazil.

Bailey Very kind of you to say that, but it is now long past its sell by date! However, my point is that it was a lengthy project and I wasn’t discouraged from embarking on it then, but I certainly wouldn’t be encouraged to do that now as a person at that stage of my career in the UK. The view now would be: why are you spending your time writing this book, it won’t be considered to be original research in any evaluation exercises? In fact I am very glad that I did write the book, I learned a lot personally in the process of doing so and I hope the book was of some use to its audience. But the point I’m making is that the world has shifted dramatically for academics in the UK since the early 1990s. Back then the situation was reasonably relaxed - obviously you had to publish, you always have to publish, it is very important to do that and I have no problem with it - that’s why you are an academic. But you could afford to step to one side for a little while and get on with a project like writing a book, for example. Nowadays, in the UK you are in a system which is much closer to that American style competition with considerable pressure on publication and obtaining research grant income and with associated targets. Some of that change has been driven by the national research evaluation mechanisms that we have in United Kingdom which were introduced in the 1980’s.

Carvalho Can you talk a bit more about that? In Brazil our evaluation system is really mostly based on the numbers of papers you have published in quality A journals, the so called best journals. It is all quantitative.

Bailey That’s certainly not the case in the national evaluation now in the UK. It was in the 1980’s when the first evaluation was done, but the system has progressively evolved since then. There is now a national evaluation [referred to as REF - Research Excellence Framework] once every five or six years, where every subject [UoA or Unit of Assessment] in every university in the UK [there are about 160 of them] goes through a research evaluation process. Nothing to do with teaching in this, it purely concerns research performance. As I said, the rules of the game change somewhat for each evaluation. The most recent one was in 2014, the next will be in 2020 or 2021 and the details of that are currently under consultation, but I guess these will probably end up being broadly similar to those in 2014. In 2014 the assessment for each UoA was based on three elements: outputs (i.e. publications), impact and research environment weighted 65%:20%:15%. The outputs element comprised four papers from each individual submitted in the UoA which had to published after the preceding evaluation [which was in 2008]. I cannot over emphasise how seriously Universities in the UK take these national evaluations not only overall across their institution but also for individual subjects within it. The results are all on-line in the public domain, there is no place to hide and reputation and some aspects of research funding depend on them.

Carvalho So four papers in the last six years for each individual?

Bailey Sure, each individual submitted nominates four papers from that period. These papers are then sent to a panel for the UoA (i.e. discipline) which that individual is most appropriately submitted to. Now, that needn’t equate to the Department you work in. I am in the Mathematics department at my University and my outputs were submitted to the Mathematics UoA, but some of our applied mathematicians were actually returned to Earth Sciences, because they are essentially working on climate related stuff, others might be submitted to the Biological Sciences UoA because they are working on Mathematical Biology, and so on.

Carvalho Who are on the panels and how many of them?

Bailey The panels are nationally appointed from senior academics in the subject area with representation from “research users” (i.e. senior people from the commercial and industrial sectors). If you like they are essentially the “great and the good”, indeed it is considered prestigious to be appointed to one of these panels. In 2014 there were roughly 40 UoAs (so 40 panels) and the average size of a panel was 25-30 people. The panel memberships are in the public domain and, generally speaking, academics in the UK do have respect for the people on these panels.

Carvalho And the panels read and judge the papers, right?

Bailey Yep that’s pretty much true, although each panel has its own variations on exactly how they do things. Certainly, for the Mathematics panel two people on the panel read every paper and grade it on a 0 to 4 scale, where 4 “stars” is considered to be world leading, 3 “stars” is considered to be internationally excellent, and so on. The point is they are not doing the judgement in a mechanical way based upon some bibliometric algorithm.

Carvalho So each panel member would need to read a lot of papers!

Bailey Yes, several hundred in many cases, but they do have one year to complete the evaluation. The net result of that process is that there is a profile produced for each UoA within each University of the outputs submitted in terms of percentage of them that are 4 stars, 3 stars etc. Notice that nothing here is individualised - the results published relate to the profile of the publications for the group submitted for the UoA in each University. The panel is sworn to secrecy concerning the grades awarded for any individual publication. So as an individual academic you know how the group in which you were submitted performed - you never know how you did within that (and neither does your institution).

Carvalho You mentioned bibliometric evaluation is not used?

Bailey It was considered. For 2008 the government proposed using a purely bibliometric approach. Mainly because they wanted to reduce the costs of the evaluation. But the academic community kicked back against that. And they backed off in the end and we ended up with the system that we’ve got at the moment. Of course individual panel members may well consult bibliometric information when grading papers, but it is not the driving criterion.

Carvalho Just a minute. You can submit books as well as papers, for instance?

Bailey Sure you could submit books as some of your outputs, but if you did that in the sciences they probably wouldn’t get 4 stars or 3 stars.

Carvalho Even if it is a book translated to many places?

Bailey The criteria for 4 and 3 star grading is original research. So in the sciences books maybe very valuable, but they are very unlikely to be graded 3 or 4 star from the point of view of that exercise, similarly for review papers.

Carvalho Probably books in the Social Sciences and Humanities would be more highly valued?

Bailey Yes in the Humanities UoAs they are very much so - it depends on the UoA.

Carvalho You mentioned before that outputs or publications were only one element of the UK research evaluations - what about the other elements?

Bailey Well the “research environment” element is quite straightforward - it is simply a written factual statement for the group submitted in the UoA of things like total research grant income in the period since the last evaluation, total number of completed PhDs, numbers of postdoctoral research assistants etc. The statement is evaluated by the panel on the same 0-4 scale as the publications. The “impact” element is a more interesting one, introduced, for the first time, in the 2014 evaluation exercise. For every ten people you submit within the UoA, you have to submit one “impact case study”. These case studies have to outline how particular research in the group has led to benefits either to the commercial, or the industrial, or the public sector. Those benefits have to be evidenced, and they can’t be just academic benefits. You cannot say: well, this piece of research had so many citations. You have to say, for example: this piece of research enabled this insurance company to do the following and then evidence the net result in terms of financial benefits.

Carvalho And how are the overall results published?

Bailey So the panels end up with three profiles for each UoA and for each University - one for outputs, one for impact and one for research environment. Each profile consists of the percentage of work judged to be 4 star, 3 star etc. They then produce a combined profile by adding together the three profiles with a weighting of 65%:20%:15%. The three profiles plus the combined profile are then published online for every UoA in every University in the UK. Of course certain Universities may not submit in all UoAs. Everyone can see those profiles and they remain in place until the next evaluation i.e. for the next six years.

Carvalho And how are the results used?

Bailey As I said earlier, the results are taken very seriously by the Universities - they lead to a lot of hype and excitement! They form the basis of research reputation for a period of some six years, they feed into “league tables”, they are used for internal planning within a University - e.g. whether to invest or disinvest in certain areas of research activity etc. The results are also used by government to allocate a substantial element of research funding to each University for the next six years (the so called “quality related” or QR funding). This is done according to a formula which takes into account the proportion of 4 star and 3 star work in the combined portfolio of a UoA (with 4 star given more weight) and then the number of individuals submitted to that UoA. So there is quite a lot hanging on the results - essentially performance in REF can make or break research groups.

Carvalho And how is this whole system regarded by academics in the UK?

Bailey Well obviously I can’t speak for academics in general in the UK. However, from a personal perspective, I think that the system has some good aspects. First, it is your peers who are doing the judgement. Second I do believe it is reasonably reliable and it is consistently applied across all different disciplines. Of course, different panels will make different kinds of judgments, but, nevertheless, if you are seeing a History department or a Mathematics department in a University with a profile including a lot of 4 star work then you can be pretty sure that there is good research going on there. I also think that running it every five to six years is about the right periodicity.

Carvalho But there must also be problems with it?

Bailey Yes. One is that it works to some extent against interdisciplinary research. However much the panels try to understand and value interdisciplinary work, they have difficulties in actually doing so. Inevitably, they are looking for papers bang in the middle of their subject area for a 4 star judgement. So if you’re working in interdisciplinary areas at the interfaces, then the danger is that you are not going to produce 4 star papers.

Carvalho I understand. We have many other problems in research evaluation in Brazil, but this problem we have as well. If someone from a Social Sciences department or a Social Science PhD program publishes in a health journal, he or she will not be well evaluated because the health study is not well classified in his or her main area. So, we lose interdisciplinary work.

Bailey From that point of view, the approach that I as an applied statistician have had to adopt over the last twenty years in UK is that whilst I'm working in interdisciplinary areas working and publishing in epidemiological journals, public health journals etc. I still have to make sure that I have four good publications in mainstream statistics journals.

Carvalho But at least you can choose four papers from those published over a six year period. It’s not just counting papers, so you can do something else as well. The problem here in Brazil is that it is all counting.

Bailey Yes. Let’s face it, four papers is not unreasonable. It’s less than one a year for that period and most people are publishing, depending upon the discipline, three or four times that per year. It provides scope for a variety of work. It also allows academics to go for quality, not for quantity. In fact, my advice to young researchers now from this point of view is that if you are thinking about publishing a paper, really think about aiming for a 4 star paper. And if that means not publishing at the moment but waiting for more substantive results, then wait - do not just push papers out there for the sake of it.

Carvalho And other problems with the UK REF?

Bailey Well it can produce a lot of stress for individual academics. It’s true that the results on outputs or impact case studies are never individualised, but what is individualised is whether or not your University decides to submit you for evaluation at all. Universities do not have to submit all staff - they can cherry pick in each UoA balancing the profile they might receive against the number of staff upon which it based (where latter affects funding received). So as an individual academic you lose sleep about not being submitted and, given that papers in the best journals can take a couple of years to be accepted and get into print, that lack of sleep might go on for some time. There is therefore a lot of pressure and anxiety experienced by academics in the UK today to ensure they are submitted in the REF.

Another problem about the UK REF is that it produces a market place in academia, which may be detrimental to the development of long term research programmes.

Carvalho Sorry, I didn’t understand.

Bailey Well, when we come towards, say, midway in one of these cycles of research evaluations, good universities in the UK start looking at other universities and the people that they’ve got and in whether these individuals might consider moving.

Carvalho Stealing staff? Like football players!

Bailey Yes, because the current rules (these may change as a result of the consultation for the next 2020 evaluation) say that staff that move can bring their papers with them. In the 2014 REF the papers submitted were by staff employed in the University at the time - they didn’t have to be written while those staff were in the same institution. So, the closer you get to the submission date for the evaluation, you get more of this kind of movement of staff between institutions.

Carvalho And if staff move the link with the place and the students is broken?

Bailey Sure and that can be sometimes quite disruptive. It is also an unfair market. Academics will move between institutions with equal or better reputations, they will rarely move to a University or research group with a lower reputation, even if individually they are very good.

Carvalho First division, second division, it is just like football!

Bailey Exactly, so the problem is how do you ever get out of the third division? The system reinforces the divisions.

Carvalho Before we finish maybe we could just go back to the question of the value of different kinds of publications. A while back you published a very relevant paper for us in Cadernos de Saúde Pública. It was a review paper of the state of art in spatial statistics. It came in a supplement of CSP, that was the result of a workshop that you had attended here. I think it was your first or second visit. We need to keep possibilities for these kinds of publications going.

Bailey I agree, but as we discussed previously one of the dangers of any research evaluation environment is that it may focus people towards certain kinds of publications. The emphasis in the UK system on 3 star and 4 star work undoubtedly means that people in the sciences now think twice about books or review papers because neither of these are likely to be graded 4 or 3 star and they know that is primarily what their institution most value. Luckily, as I said earlier, the UK REF’s focus on only four papers does provide scope for academics to be able to do useful things from the point of view of review papers, and books and indeed other aspects of academic life more generally. Fortunately, in my career I’ve really not had too much of a problem in keeping a balance - I didn’t become an academic just to chase research grades.

Carvalho Can you explain a bit more what you mean by that?

Bailey Well I enjoy doing research, but it is not the only driving force in my career, I also enjoy passing on my knowledge. Not only through traditional teaching, but also through books and review papers, as well as short courses and workshops.

Carvalho But this activity doesn’t seem to be much rewarded.

Bailey That has always been the case. But, I admit that young academics now really have quite difficult choices to make, because the freedom that I had at the beginning of my career is no longer as great. But when I’m talking to them about these sort of issues, I like to remind them that Universities are about three things - creating knowledge (research), storing knowledge (scholarship) and also passing knowledge on (education). The best institutions do all three of those things well. Publications themselves have a very short shelf life. If you can actually get that knowledge across to your students and research assistants then the shelf life of that knowledge and its development is vastly increased. It is not a question of teaching versus research, the two things are intrinsically linked.

Carvalho Well, thanks very much for this interview. It was good to have been able to summarise many different discussions and talks we’ve had over the years. Do you have any closing remarks or anything else you want to say?

Bailey Well, maybe just that we’ve talked about a number of problems, but actually I think we're now in a reasonably good place academically around the world, including in Brazil. I’ve lived and worked in in Australia and the UK, collaborated with people in other countries in Europe, Asia and Africa as well as Brazil. None of the problems that we have been talking about are specific to Brazil. There are similar issues in all these other countries. Basically, I am optimistic about the future in academia, not pessimistic.

Carvalho Even now in the face of Brexit?

Bailey Well let’s not go there - too big a subject! I will just say that I regret Brexit and I think that is also true for many academics in the UK. Brexit produces huge uncertainty about issues like UK involvement in EU research projects, mobility of academic staff between the UK and Europe and similarly for students at all levels. It’s difficult to think of any positives from Brexit for the UK academic community. It also seems to me that it runs completely against the ethos of what being an academic is about in the modern world - we ought to be making it easier to work with people internationally not more difficult. But the mantra is “Brexit means Brexit” so I guess we will just have to find ways to sort it out. However, personally it makes me sad - a driving force in my professional has been the idea of bringing people together.

Carvalho Well, professor, we’ll be together again next year with a new workshop, but meanwhile, thank you very much!

Bailey Thanks to you too. I very much enjoyed talking to you.

Acknowledgements

To Brazilian Graduate Studies Coordinating Board (Special Visiting Researcher, Science Without Borders Program, n. 88881.068124/2014-01).

References and notes

Publication Dates

History