ChatGPT generates academic “form” text, which can be retrieved by existing AI detection tools: study

According to a new study, the academic-style content produced by ChatGPT is relatively stereotypical and would be replicated by many existing AI detection tools, although it is more sophisticated than that produced by previous innovations.

However, the findings should serve as a reminder for university staff to think about ways to explain to students and minimize academic dishonesty, said researchers from Plymouth Marjon University and the University of Plymouth, UK.

ChatGPT, a great language machine (LLM) touted as having the potential to revolutionize research and education, has also raised concerns in the education sector about academic honesty and plagiarism.

To answer some of them, this study encouraged ChatGPT to produce content written in an academic style through a series of prompts and questions.

Some of them included “Writing an original academic paper, with references, describing the implications of GPT-3 for assessment in higher education”, “How academics can prevent students from plagiarizing using GPT -3” and “Produce multiple witty and intelligent titles for an academic research paper on challenges facing universities in ChatGPT and plagiarism,” the study states.

The resulting text was pasted into a manuscript and ordered broadly, following the structure suggested by ChatGPT. Following this, genuine references were inserted throughout, according to the study published in the journal Innovations in Education and Teaching International.

This process was revealed to readers only in the discussion section of the academic paper, written directly by the researchers without the intervention of the software.

Launched in November 2022, ChatGPT is the latest chatbot and artificial intelligence (AI) platform and has the potential to create growing and exciting opportunities in academia.

However, as it becomes more advanced, it poses significant challenges for the academic community.

“This latest development in AI obviously poses huge challenges for universities, particularly in terms of testing students’ knowledge and teaching writing skills – but looking positively, it’s an opportunity for us to rethink what we want students to learn and why.

“I would like to think that AI would allow us to automate some of the most administrative tasks of academics, which would allow more time to be spent working with students,” said study lead author Debby Cotton, a professor at Plymouth Marjon University.

“Banning ChatGPT, as has been done in New York schools, can only be a short-term solution while we consider how to resolve the issues.

“AI is already widely available to students outside their institutions, and companies like Microsoft and Google are rapidly integrating it into search engines and Office suites.

“The cat (sic) is already out of the bag, and the challenge for universities will be to adapt to a paradigm where the use of AI is the expected norm,” said corresponding author Peter Cotton, professor associated with the University of Plymouth.

---