Research repository ArXiv will ban authors for a year if they let AI do all the work

ArXiv, a widely used open repository for preprint research, is making more efforts to eliminate the careless use of large language models in scientific papers.

Although papers are published on the site before they are peer-reviewed, arXiv (pronounced “archive”) has become one of the main ways through which research in fields such as computer science and mathematics circulates, and the site itself has become a source of data on trends in scientific research.

ArXiv has already taken steps to combat the growing number of low-quality research papers generated by AI, for example by requiring first-time posters to obtain approval from a known author. After being hosted by Cornell University for more than 20 years, the organization has become an independent nonprofit, allowing it to raise more money to tackle problems like artificial intelligence.

In his latest move, Thomas Dietrich — head of arXiv’s Department of Computer Science — posted on Thursday that “if the submission contains indisputable evidence that the authors did not verify the results of creating the LLM, it means we cannot trust anything in the paper.”

This indisputable evidence could include things like “hallucinogenic references” and comments to or from LLM, Dietrich said. If such evidence were found, the paper’s authors would face “a one-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted by a peer-reviewed venue.”

Note that this does not constitute an explicit ban on the use of LLMs, but rather an insistence that authors, in Dietrich’s words, bear “full responsibility” for the content, “regardless of how the contents were created.” So, if researchers copy and paste “inappropriate language, plagiarized content, biased content, errors, mistakes, incorrect references, or misleading content” directly from LLM, they remain responsible for it.

Dietrich told 404 Media this will be a “one strike” rule, but supervisors must report the issue and department heads must confirm evidence before imposing punishment. The authors will also be able to appeal the decision.

Recent peer-reviewed research has found that fabricated citations are on the rise in biomedical research, likely due to MBAs — although to be fair, scientists aren’t the only ones getting caught using AI-authored citations.