Abstract

The aim of this study was to identify and analyze the main constituent elements of text generated by ChatGPT in response to questions on an emerging topic in the academic literature in Portuguese - health literacy - and discuss how the evidence produced can contribute to improving our understanding of the limits and challenges of using artificial intelligence (AI) in academic writing. We conducted an exploratory descriptive study based on responses to five consecutive questions in Portuguese and English with increasing levels of complexity put to ChatGPT. Our findings reveal the potential of the use of widely available, unrestricted access AI-based technologies like ChatGPT for academic writing. Featuring a simple and intuitive interface, the tool generated structured and coherent text using natural-like language. Considering that academic productivism is associated with a growing trend in professional misconduct, especially plagiarism, there is a need too take a careful look at academic writing and scientific knowledge dissemination processes mediated by AI technologies.

Key words:
Health literacy; Artificial Intelligence; Academic communication; Ethics in the scientific publication

Introduction

Health literacy is a widely employed concept that has been increasingly used since the 1990s to refer to the ability of individuals to seek out, understand, evaluate and make sense of information with the aim of caring for their own health or that of others¹1 Peres F, Rodrigues KM, Silva TS. Literacia em saúde. Rio de Janeiro: Ed. Fiocruz; 2021.. It encompasses a broad set of skills and competencies, ranging from the reading and writing skills needed to comprehend health information, to the awareness of the role each citizen plays in guaranteeing public health²2 Zarcadoolas C, Pleasant A, Greer DS. Understanding health literacy: an expanded model. Health Prom Int 2005; 20(2):195-203.^,33 Pleasant A, Kuruvilla S. A tale of two health literacies: public health and clinical approaches to health literacy. Health Promo Int 2008; 23(2):152-159..

Studies of the health literacy of individuals and groups around the planet have demonstrated that the more developed these skills and competencies are in a given population the better individual and collective health outcomes¹1 Peres F, Rodrigues KM, Silva TS. Literacia em saúde. Rio de Janeiro: Ed. Fiocruz; 2021.^,44 Okan O. From Saranac Lake to Shanghai: A brief history of health literacy. In: Okan O, Bauer U, Levin-Zamir D, Pinheiro P, Sørensen K, organizers. International handbook of health literacy. Bristol: Policy Press; 2019. p. 21-38.. The inverse has also been reported, with other studies showing that individuals and groups with a lesser degree of health literacy are more likely to be subjected to inadequate management of chronic conditions and use emergency services, and tend to have greater difficulty adhering to drug therapy⁵5 International Union for Health Promotion and Education (IUHPE). Position statement on health literacy: a practical vision for a health literate world. IUHPE Global Working Group on Health Literacy. Paris: IUHPE; 2018..

Although widely used in academic spheres and by governments in the United States, Canada and various European countries, including Portugal, health literacy is still a relatively new concept in Brazil, being applied mainly to language studies and understanding health information, meaning that research tends to focus on the core domain of the concept⁶6 Peres F. Alfabetização, letramento ou literacia em saúde? Traduzindo e aplicando o conceito de health literacy no Brasil. Cien Saude Colet 2023; 28(5):1563-1573.. Likewise, the concept is neither at the heart of public health policies, nor a strategic element of individual and collective health promotion¹1 Peres F, Rodrigues KM, Silva TS. Literacia em saúde. Rio de Janeiro: Ed. Fiocruz; 2021..

Despite the limited literature on the topic, growing academic interest in health literacy has contributed to the development of knowledge and practices related to various aspects that influence the process of signifying health information. This is especially true now, at a time strongly marked by a pandemic that unfolded in a context of overabundant, widely accessible and easily disseminable information that often lacks theoretical underpinnings or veracity - a context many authors refer to as an “infodemic”⁷7 Zarocostas J. How to fight an infodemic. Lancet 2020; 395(10225):676.^,88 Harper T, Tomkinson S, Attwell K. Communication Is Not a Virus: COVID-19 Vaccine-Critical Activity on Facebook and Implications for the 'Infodemic' Concept. J Health Commun 2022; 27(8):563-573.. And a time equally marked by widespread access to information and communication technologies, particularly mobile devices⁹9 Nunes C, Marta B, Marino P. Promoção da Literacia em Saúde através dos media. Literacia em saúde na prática. In: Lopes C, Almeida CV, coordenadores. Literacia em saúde na prática. Lisboa: Edições ISPA; 2019. p. 97-117.

10 Van Kessel R, Wong BLH, Clemens T, Brand H. Digital health literacy as a super determinant of health: More than simply the sum of its parts. Internet Interv 2022; 27:e100500.^-1111 Yang K, Hu Y, Qi H. Digital Health Literacy: Bibliometric Analysis. J Med Internet Res 2022; 24(7):e35816., and the rapid evolution of even more advanced technologies applied to everyday life, such as those using artificial intelligence (AI)¹²12 Tatnall A, Davey B, editors. Reflections on the History of Computers in Education: Early Use of Computers and Teaching about Computing in Schools. New York: Springer; 2014.^,1313 Kibria MG, Nguyen K, Villardi GP, Zhao O, Ishizu K, Kojima F. Big data analytics, machine learning, and artificial intelligence in next-generation wireless networks. IEEE Access 2018; 6:32328..

In the second half of 2022, the science and technology editorials of the main media outlets around the globe gave widescale coverage to the launch of a new AI-based tool called ChatGPT. Developed by the American technology company OpenAI¹⁴14 OpenAI. ChatGPT - versão 30 jan 2023 [Internet]. [acessado 2023 fev 14]. Disponível em: http://openai.com/blog/chatgpt/.
http://openai.com/blog/chatgpt... , founded in 2015 in Silicon Valley, ChatGPT is an AI chatbot that uses natural language processing to create humanlike conversational dialogue in response to prompts or queries. The tool works by analyzing patterns in large natural language processing data sets (big data), using AI to produce elaborate and “original” responses to queries written into a page with a chat-like structure.

Besides the technological issues, the launch of ChatGPT brought into question - especially within the academic community - the ethical implications of using AI to produce knowledge and academic writing¹⁵15 Thorp HH. ChatGPT is fun, but not an author. Science 2023; 379(6630):313. in different areas, including the field of health¹⁶16 Liebrenz M, Schleifer R, Buadze A, Bhugra D, Smith A. Generating scholarly content with ChatGPT: ethical challenges for medical publishing. Lancet Digit Health 2023; 5(3):e105-e106.. This has added another layer of complexity to the already imbricated context of the production and signification of health information in times of infodemic and low health literacy.

The aim of this study was to identify and analyze the main constituent elements of the answers generated by ChatGPT in response to queries about an emerging theme in the Portuguese language - health literacy - and discuss how the evidence produced can contribute to improving our understanding of the limits and challenges of using AI in academic writing.

Methodology

We conducted an exploratory descriptive study based on responses to questions put to ChatGPT¹⁴14 OpenAI. ChatGPT - versão 30 jan 2023 [Internet]. [acessado 2023 fev 14]. Disponível em: http://openai.com/blog/chatgpt/.
http://openai.com/blog/chatgpt... about health literacy.

The questions were asked on 13 February 2023 using the version of the platform updated on 30 January 2023 (GPT 3), which was made free to the public during its research preview. We devised five questions in Brazilian Portuguese and English with increasing levels of complexity, which were entered in sequence:

We then performed another search using the same five prompts translated into English, given that most of the literature on the topic is in this language and because academic material is the main substrate used to develop the algorithms used by ChatGPT. Finally, we repeated the search replacing the term “literacia em saúde” with “alfabetização em saúde” and “letramento em saúde”, considering the range of translations of health literacy in Brazilian Portuguese, as described by a recent study⁶6 Peres F. Alfabetização, letramento ou literacia em saúde? Traduzindo e aplicando o conceito de health literacy no Brasil. Cien Saude Colet 2023; 28(5):1563-1573..

The responses were compiled in full and analyzed using the Turnitin plagiarism-detection software to verify the originality of the text generated by the tool. The responses were then analyzed using content analysis¹⁷17 Miller T. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 2019; 267:1-38., structured into four stages: organization; codification; categorization; and inferences.

The organization of the analysis includes planning and developing data gathering strategies, as well as the exploration of the collected material (detailed reading and re-reading of the two datasets in Portuguese and English). The material was then codified and categorized according to the topics identified in the responses provided by the tool. Finally, inferences are drawn and the constituent elements of each response were identified. The results were compared to determine the scope of responses in each language, identify differences in the characteristics and constituent elements generated for each term used in Portuguese (literacia/alfabetização/letramento) and assess the similarity between responses and the relevant literature on health literacy reviewed by a recent study in Brazil⁶6 Peres F. Alfabetização, letramento ou literacia em saúde? Traduzindo e aplicando o conceito de health literacy no Brasil. Cien Saude Colet 2023; 28(5):1563-1573..

Results

The findings reveal advances in AI technology as a resource for academic writing, using natural-like language and producing text that is up-to-date and coherent with state-of-the-art literature.

With regard to the originality of the text generated by the ChatGPT, overall similarity in the Turnitin report was 20%. However, the analysis of the similarities showed that they were small passages (no more than five or six consecutive words), including repetitions, such as: a) the full name of organizations like the DHHS (U.S. Department of Health and Human Services); b) commonly repeated definitions of health literacy (e.g. “understanding basic health information and services to inform decision-making”); c) the main applications of health literacy (e.g. “help improve quality of life”); and d) concepts (“literacia em saúde”, “letramento em saúde”, “alfabetização em saúde”). The software did not identify any instances of matching sentences or paragraphs. Based on the above, the responses generated by the tool were considered “original” text with similarities that would be acceptable in an effectively original author’s text substantiated by the relevant literature.

In response to Question 1 (Explain the concept of health literacy), ChatGPT generated a structured and coherent text, both in terms of language and the relevant literature. However, the scope of the response was limited when the multidimensionality of the concept is considered, and the texts contained repetitions of central elements in some sentences (Chart 1). The main constituent elements identified in the response were as follows: a) health literacy as an individual skill; b) health literacy limited to the core domain of the concept (the language domain); c) health literacy as a health care skill; d) health literacy as a medical care skill; e) health literacy as the ability to make informed decisions; and f) health literacy as the ability to navigate the health system.

The response to the same question in English contained other constituent elements in addition to those above (Chart 1): a) health literacy as an individual skill; b) health literacy as a health care skill; c) health literacy as a medical care skill; d) health literacy as the ability to make informed decisions; e) health literacy as the ability to navigate the health system; f) health literacy as a skill for critical thinking; and g) health literacy as the ability to understand basic health concepts and terminology.

The response to the same question when “literacia em saúde” was replaced with “alfabetização em saúde” was more comprehensive, containing the following constituent elements (Chart 1): a) health literacy as an individual skill; b) health literacy as a health care skill; c) health literacy as a medical care skill; d) health literacy as the ability to make informed decisions; e) health literacy as the ability to navigate the health system; f) health literacy as a knowledge and skills and competencies acquisition process; g) health literacy as the ability to prevent disease; h) health literacy as the ability to evaluate health information presented in different forms; and i) health literacy as the ability to involve people in the management of your health and well-being.

The response generated when “literacia em saúde” was replaced by “letramento em saúde” was similar to the first response (using the term “literacia em saúde”) and less comprehensive than the former, including the following constituent elements (Chart 1): a) health literacy as a health care skill; b) health literacy as a medical care skill; c) health literacy as the ability to make informed decisions; d) health literacy as the ability to navigate the health system; e) health literacy as the ability to get people to play an active role in your health; and f) health literacy to improve individual and collective health outcomes.

The responses generated by ChatGPT to each Question 1 and their respective constituent elements are shown in Chart 1.

With regard to Question 2 (What are the origins of the concept of health literacy), ChatGPT generated similar responses, tracing the advent of research into health literacy back to the 1970s and highlighting a surge in interest in the topic from the 1990s, when the concept began to be associated with health education strategies (Chart 2). The main constituent elements of the responses were: a) health literacy as a health care skill; b) health literacy as a medical care skill; c) health literacy as the ability to make informed decisions; and d) health literacy as a key element of health promotion.

Some differences were observed between the responses to the questions using the different terms in Portuguese and English. When the term “literacia em saúde” was used, the tool did not highlight the evolution of the concept from the 1990s. However, when “alfabetização em Saúde” was used, the tool mentioned that the concept began to be developed in the 1990s, citing a World Health Organization (WHO) initiative. When this term was replaced by “letramento em saúde”, ChatGPT also mentioned that the concept began to be developed in the 1990s, citing the development of the concept of letramento as a milestone. Finally, in response to the question in English, the tool traced the concept back to the 1990s, citing a document of the U.S. Department of Health and Human Services (DHHS).

The responses generated by ChatGPT to each Question 2 and their respective constituent elements are shown in Chart 2.

Question 3 (Can you describe the importance of health literacy for individuals, health systems and institutions?) increased the degree of complexity by requesting a response in three different levels (individual, health system and institutional) and received a response divided into these three levels. At the individual level, the responses to the questions using the different terms in Portuguese and English converged towards the following main points: a) increased individual capacity to make informed decisions regarding health and medical care; b) better understanding and use of health information and services; and c) ability to improve overall health and well-being (Chart 3).

At the health system level, the responses to the questions in Portuguese converged towards two points: a) the improvement of services and b) cost reduction. The response to the question in English was more comprehensive, mentioning the following in addition to these two points: a) greater patient adherence to treatment; and b) reduction of health disparities/promotion of equity (Chart 3).

With regard to institutions, the responses to the questions in Portuguese and English converged towards three points: a) promotion of health equity and the reduction of disparities between social and cultural groups; b) improvement of the effectiveness of health programs and services; and c) improvement in the overall quality of health care. The responses generated by ChatGPT to each Question 3 and their respective constituent elements are shown in Chart 3.

Finally, we analyzed the responses to Questions 4 and 5, both of which had a higher level of complexity because they requested ChatGPT to identify the main challenges in promoting health literacy (Question 4) and the prospects for improving public policies on health literacy in a post-pandemic context (Question 5).

The responses to question 4 in Portuguese and English converged towards the following points: a) language and cultural barriers; b) limited access to health information; c) information overload/too much information; d) information complexity/language; e) cognitive limitations or lack of literacy skills; f) socioeconomic disparities; and g) lack of trust in the health system.

Although Question 4 was limited to the identification of challenges, all responses to all questions in Portuguese and English also offered suggestions/recommendations for addressing the challenges, including: a) provide clear information using simple language; b) ensure widespread access to quality information; c) consider cultural and socioeconomic differences between individuals and groups; and d) train/educate health professionals and patients to improve health communication. The main challenges mentioned in the responses and respective suggestions at the end of each response are summarized in Figure 1.

Question 5 requested ChatGPT to identify the main prospects for improving public policies on health literacy in a post-pandemic context. There were both points of convergence and points of divergence in the responses to the questions in Portuguese and English. Points of convergence across all the questions included:

a) the need for greater investment in health literacy education programs; b) the need to improve health communication programs and policies; c) the need to culturally adapt public health policies to the needs of different individuals and groups; and d) the need to engage the community in policy-making processes.

The response to the question using the term “literacia em saúde” also identified the need to promote information and communication technologies within the scope of health policy making. When “literacia em saúde” was replaced by “alfabetização em saúde”, the response also included the need for greater investment in technology and the need to broaden understanding of health, considering social and environmental aspects within the scope of public policy making. And when the term “literacia em saúde” was replaced by “letramento em saúde”, the response also included the need to promote health literacy among traditionally vulnerable groups due to ethic, racial and gender issues.

In the response to the question in English, in addition to the points of convergence mentioned above, ChatGPT mentioned the need to promote digital health literacy and tackle health disparities. The points of convergence and divergence in the responses to Question 5 are shown in Figure 2.

Discussion

Our findings reveal the potential of the use of AI-based technologies for academic writing, particularly for text on emerging themes in Brazil such as health literacy. The results are discussed below from three perspectives: a technological perspective; an academic perspective; and ethical perspective.

From a technological perspective, the announcement of the creation and wide availability of ChatGPT for testing at the end of 2022 was a milestone for the incorporation of technologies into the academic world, just as Google was a leap forward in the search for information on a wide range of topics, including academic knowledge¹²12 Tatnall A, Davey B, editors. Reflections on the History of Computers in Education: Early Use of Computers and Teaching about Computing in Schools. New York: Springer; 2014.. With a user-friendly interface (chat format), easy to navigate and providing wide and unrestricted access (all you need to do is provide an email address on the developer’s website), ChatGPT has the potential to become another technology tool to be incorporated into academia on a global scale. This is despite the risk of representing an ethically reprehensible shortcut in the process of constructing and disseminating academic knowledge¹⁵15 Thorp HH. ChatGPT is fun, but not an author. Science 2023; 379(6630):313. due to the nature and sophistication of the technology, as discussed below.

While AI-based tools used to analyze large volumes of data (big data) and develop forecasts and project trends are being incorporated by academia and acknowledged as strategies to construct scientific knowledge in various areas¹⁷17 Miller T. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 2019; 267:1-38.^,1818 Aggarwal K, Mijwil MM, Al-Mistarehi AH, Alomari S, Gök M, Alaabdin AMZ, Abdulrhman SH. Has the future started? The current growth of artificial intelligence, machine learning, and deep learning. Iraqi J Computer Sci Math 2022; 3(1):115-123., including the field of health¹⁹19 Yu KH, Beam AL, Kohane IS. Artificial intelligence in healthcare. Nature Biomed Engineering 2018; 2(10):719-731.^,2020 Chen M, Decary M. Artificial intelligence in healthcare: An essential guide for health leaders. Healthcare Manage Forum 2020; (1):10-18., their use to generate “original” text, images and graphs used in publications raises a series of questions about the integrity of the knowledge generated, both in terms of the reliability of the responses and ethical aspects of authorship.

Although the use of AI to generate “natural-like language” text is not a recent discovery and ChatGPT is not the first tool of its kind to offer this resource to the academic community²¹21 Paris CL, Swartout WR, Mann WC, editors. Natural language generation in artificial intelligence and computational linguistics. Vol. 119. New York: Springer Science & Business Media; 2013.^,2222 Baclic O, Tunis M, Young K, Doan C, Swerdfeger H, Schonfeld J. Artificial intelligence in public health: Challenges and opportunities for public health made possible by advances in natural language processing. Canada Communicable Disease Report 2020; 46(6):161., its widespread availability and fast-growing user base around the globe raises the need to assess the potential of incorporating the tool into the day-to-day functioning of universities, research institutions, schools and training institutions for use by students, teachers, researchers and knowledge disseminators alike²³23 Pavlik J. Collaborating with ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education. J Mass Commun Educ 2023; 78(1):e10776958221149577..

Our findings revealed technological advances in the tool, which, in response to the five questions, generated coherent text that was consistent with state-of-the-art literature, producing sentence structures and formatting that could very well be the product of an academic study in a degree or post-graduate setting. However, the tool provided some information that has been superseded by advances in knowledge about the topic and showed some inconsistencies, such as tracing the origin of the concept of health literacy (and its variations in Portuguese) back to the 1970s rather than the 1990s, when the term was actually coined and incorporated into knowledge and practices in countries like the United States, Canada and Australia¹1 Peres F, Rodrigues KM, Silva TS. Literacia em saúde. Rio de Janeiro: Ed. Fiocruz; 2021.. This is probably because the tool’s algorithms are influenced by the large volume of studies on literacy and functional literacy (the ability to develop oral and written language skills), as well as by studies in the field of health conducted after the emergence of discussions about health promotion and the social medicine movement, which resulted in the Alma-Ata conference (1978) and the Ottawa Charter (1986)²⁴24 Heidmann IT, Almeida MCPD, Boehs AE, Wosny ADM, Monticelli M. Promoção à saúde: trajetória histórica de suas concepções. Texto Contexto Enferm 2006; 15(2):352-358..

It is therefore evident that the tool was influenced by the large volume of information published about these historical milestones, which strongly influenced the field of Public Health. The influence of big data on the accuracy of the responses generated by ChatGPT is also observed in the scope of factors associated with the concept in English (health literacy) and the use of “alfabetização em saúde” in place of “literacia em saúde” or “letramento em saúde”. It was expected, and anticipated by the study design, that the responses to the questions in English would be more comprehensive, because most of the academic and non-academic literature on health literacy is in this language.

However, the responses to the questions using the term “alfabetização em saúde” were also more comprehensive. This reveals a conceptual inconsistency, because the term alfabetização em saúde is more limited and has been superseded in academic literature in Portuguese¹1 Peres F, Rodrigues KM, Silva TS. Literacia em saúde. Rio de Janeiro: Ed. Fiocruz; 2021.^,99 Nunes C, Marta B, Marino P. Promoção da Literacia em Saúde através dos media. Literacia em saúde na prática. In: Lopes C, Almeida CV, coordenadores. Literacia em saúde na prática. Lisboa: Edições ISPA; 2019. p. 97-117., in which the main terms used are literacia em saúde (mainly in Portugal) and letramento em saúde (mainly in Brazil, being the only term registered in the Health Sciences Descriptors)⁶6 Peres F. Alfabetização, letramento ou literacia em saúde? Traduzindo e aplicando o conceito de health literacy no Brasil. Cien Saude Colet 2023; 28(5):1563-1573.. Some authors suggest that this phenomenon constitutes bias that favors the quantitative to the detriment of qualitative in an environment where algorithms are trained using the number of occurrences and observed trends in available knowledge bases¹³13 Kibria MG, Nguyen K, Villardi GP, Zhao O, Ishizu K, Kojima F. Big data analytics, machine learning, and artificial intelligence in next-generation wireless networks. IEEE Access 2018; 6:32328.^,2020 Chen M, Decary M. Artificial intelligence in healthcare: An essential guide for health leaders. Healthcare Manage Forum 2020; (1):10-18.^,2222 Baclic O, Tunis M, Young K, Doan C, Swerdfeger H, Schonfeld J. Artificial intelligence in public health: Challenges and opportunities for public health made possible by advances in natural language processing. Canada Communicable Disease Report 2020; 46(6):161..

Also from a technological perspective, a positive point is the fact that, in response to Question 3 (What are the main challenges in promoting health literacy?), the tool generated not only a list of challenges but also some paths towards tackling them. This is what some authors describe as the process of algorithm “learning” in AI tools to produce natural-like language text¹⁷17 Miller T. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 2019; 267:1-38.^,1818 Aggarwal K, Mijwil MM, Al-Mistarehi AH, Alomari S, Gök M, Alaabdin AMZ, Abdulrhman SH. Has the future started? The current growth of artificial intelligence, machine learning, and deep learning. Iraqi J Computer Sci Math 2022; 3(1):115-123.^,2121 Paris CL, Swartout WR, Mann WC, editors. Natural language generation in artificial intelligence and computational linguistics. Vol. 119. New York: Springer Science & Business Media; 2013..

From an academic perspective, it is important to highlight the huge potential of ChatGPT as a tool for students, teachers and researchers, who are under increasing pressure to publish study results. Several authors have drawn attention to the risks of so-called “academic productivism”, associated with the need to meet increasingly rigorous quantitative output target, generating evaluation indicators that determine, for example, access to funding, career progression, the accreditation of post-graduate programs and assigning of titles²⁵25 Teixeira TSC, Marqueze EC, Moreno CRC. Produtivismo acadêmico: quando a demanda supera o tempo de trabalho. Rev Saude Publica 2020; 54:117.^,2626 Vieira JDA, Castaman AS, Junges Júnior ML. Produtivismo acadêmico: representação da universidade como espaço de reprodução social. Avaliação (Campinas) 2021; 26:253-269..

Against this backdrop of academic output metrics and pressure to publish, “shortcuts” are constantly made available to academic authors, who, based on ethical and moral values, need to consider the implications of using these resources for the integrity of their work and career. A large part of these shortcuts constitute academic misconduct, including falsification of results, fabrication of data and plagiarism. The latter is the most common type of misconduct in the academic world²⁷27 Awasthi S. Plagiarism and academic misconduct: A systematic review. DESIDOC 2019; 39(2):94-100. and has been the object of growing attention and care in peer review processes and the publication of academic papers²⁸28 Olivia-Dumitrina N, Casanovas M, Capdevila Y. Academic writing and the internet: Cyber-plagiarism amongst university students. J New Approaches Educ Res 2019: 8(2):112-125..

Characterizing plagiarism as misconduct is no easy task. Even an undergraduate student starting out in the academic world without any prior article writing experience knows that copying another author’s words without citation is fraud. However, when an experienced post-graduate student or professor uses AI to produce “original” text from responses to questions put to tools such as ChatGPT, the concept of plagiarism is resignified, requiring a broader lens to assess the integrity of academic output.

According to H. Holden Thorp, editor-in-chief of the Science family of journals, the potential effects of ChatGPT on writing scientific papers is worrisome. In an editorial of the journal Science - one of the most prestigious in the academic world - in January 2023, he reveals that in a recent study abstracts created by ChatGPT were submitted to a select and experienced group of academic reviewers, who only detected 63% of these fakes¹⁵15 Thorp HH. ChatGPT is fun, but not an author. Science 2023; 379(6630):313..

In light of the above, and considering the ethical implications related to the integrity of knowledge generated by AI, the Science family of journals updated its editorial policy in January 2023, reinforcing the understanding that “original” text does not include text produced (fully or in part) by ChatGPT and that the use of content generated by AI is plagiarism and not the product of the author’s ideas, reflections and opinions. On an even more unequivocal note, the updated policy specifies that text generated by ChatGPT (or any other AI tool) cannot be used in any article submitted to the family of journals, nor can figures, images, or graphics be the products of such tools¹⁵15 Thorp HH. ChatGPT is fun, but not an author. Science 2023; 379(6630):313..

Along similar lines, Springer Nature, which includes the equally prestigious journal Nature, also published an editorial in January 2023²⁹29 Nature. Tools such as ChatGPT threaten transparent science; here are our ground rules for their use [Editorial]. Nature 2023; 613(7945):612. stating that tools such as ChatGPT are a threat to transparent science. As a result, the group created a set of ground rules for their use, including that ChatGPT (or any other AI tool) will not be accepted as a credited author and that the use of these tools should be documented in the methods or acknowledgements sections.

It is expected that in coming months other editorial groups will also update their editorial policies to address the limits of using tools such as ChatGPT. This is reason to believe that the debate surrounding the use of AI tools and its implications is still in its infancy, requiring an update and review of the situation in the near future.

Conclusions

This study uncovers some important evidence on the potential and, not always positive, repercussions of the use of AI tools for academic writing based on the analysis of responses to questions put to ChatGPT about an emerging theme in the relevant literature in Portuguese: health literacy. From a technological perspective, the findings show that AI-based tools that generate natural-like language text are well developed and that, by offering unrestricted access to a tool (albeit as a research preview) featuring a simple and intuitive interface, ChatGPT has major potential for dissemination and use in academic settings. This is especially the case considering the extreme pressure caused by the need to meet increasingly rigorous academic output targets, which ends up leading to the subalternation of teachers, researchers and students.

From an academic perspective, this so-called productivism in the academic world ends up leading an increasing number of researchers and knowledge disseminators to seek shortcuts or resources that enable them to meet academic output targets, where publications in indexed journals are among those obtaining the highest index score. The phenomenon of academic productivism is thus associated with a growing trend in professional misconduct, especially when it comes to plagiarism. In an environment highly mediated by technologies, including large databases with unrestricted internet access, the process of producing and publishing academic papers is increasingly subject to fraud, leading editorial groups and academic institutions to adopt rigorous measures to identify and curb misconduct such as plagiarism, falsification and fabrication of data.

Based on our results, the novelty presented here and discussed from three standpoints, including and ethical perspective, is the fact that tools that generate natural-like language text such as ChatGPT produce “original” text based on the analysis of large volumes of information using AI-trained algorithms. Our results show that even for an emerging topic in relevant literature in the Portuguese, it was possible to generate coherent and structured texts, based on five simple guiding questions. The findings also reveal the need to take a more careful look at the scientific knowledge production and dissemination process around the globe. Finally, this analysis also highlights the need to develop skills and competencies to better understand the rationale, limits and ethical implications interwoven into the academic writing and knowledge dissemination process.

References

Publication Dates

History