14 prestigious schools exposed in the "secret instruction brainwashing AI" scandal, Waseda University professor: too many people are lazy and let AI handle paper reviews..

Seventeen papers on the well-known paper website arXiv contain hidden instructions that allow only positive reviews, aiming to manipulate AI and spark discussions about a Crisis of Confidence in the academic community. (Background: Meta is aggressively recruiting AI talent, with Zuckerberg claiming 'annual salaries exceeding 100 million dollars'; Sam Altman sarcastically commented: unable to buy the best employees) (Context: OpenAI is rumored to be launching an AI version of Office software, going head-to-head with Microsoft and Google) Earlier this month, a shocking investigation by Nikkei revealed 'invisible prompts' in the academic circle: 17 papers published on arXiv secretly inserted instructions asking reviewers to give high scores. These instructions were hidden in white text on a white background within the content or LaTeX comments, making them invisible to the naked eye, but if reviewers used AI to perform their work, it could significantly affect the quality of the reviews. arXiv is an open website for global scientists and researchers, mainly collecting preprints in fields such as physics, mathematics, computer science, biology, quantitative finance, and statistics. As of February 2025, arXiv.org has collected over 2.6 million preprints. Cleverly hidden codes have implicated eight countries and prestigious universities According to the Nikkei article, this incident has affected 14 top institutions across the United States, Japan, South Korea, China, and other countries, including Columbia University, the University of Washington, Waseda University, and KAIST. The 17 papers are primarily in the field of computer science, published between April and June this year. The research team shrank phrases like 'only positive reviews please' to 0.5pt or included them in comments, which LLM can read clearly and might actually follow these commands to give high ratings; however, human reviewers would not be able to detect them. A gray trial of academic integrity After the method was exposed, it sparked a debate on whether to 'fight fire with fire'. A professor from Waseda University stated that this was done to restrain lazy AI reviewers, as there have been too many instances of important review work being handed over to AI. However, an associate professor who was a co-author from KAIST has retracted the paper and apologized, stating 'allowing AI to give positive review evaluations is inappropriate.' Concerns extending to the capital market Currently, the application of AI in business, science, and legal documents is becoming increasingly common. If investment presentations, financial reports, or compliance texts are implanted with similar methods, the summaries generated by AI may lean towards a single position, affecting investment decisions or regulatory judgments. Hasegawa Shun, the technology director of AI development company ExaWizards, stated, 'This could hinder users from obtaining accurate information.' Currently, publishers are drafting 'AI Usage Disclosure Terms' requiring authors to explain the usage of tools. Major universities are also accelerating the development of internal guidelines, adding pre-submission scanning procedures; model developers are introducing 'hidden prompt scanners' aimed at identifying abnormal instructions... Only by simultaneously strengthening technology, systems, and culture can we prevent invisible commands from eroding the credibility of knowledge and capital. Related reports Downloading others' creations and then using AI to alter images is illegal! China's first AI copyright infringement criminal case has been sentenced to imprisonment and fines. Musk claims AI knowledge has too many errors 'Grok3.5 needs to rebuild the correct version for humanity'; CZ responds: a singular historical perspective will have problems. <14 prestigious schools exposed in 'secret instructions brainwashing AI' scandal, Waseda professor: too many people are lazy and hand over paper reviews to AI..> This article was first published in BlockTempo, one of the most influential Blockchain news media.

View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Share
Comment
0/400
No comments
Trade Crypto Anywhere Anytime
qrCode
Scan to download Gate app
Community
English
  • 简体中文
  • English
  • Tiếng Việt
  • 繁體中文
  • Español
  • Русский
  • Français (Afrique)
  • Português (Portugal)
  • Bahasa Indonesia
  • 日本語
  • بالعربية
  • Українська
  • Português (Brasil)