Wikipedia urges AI companies to use its paid API, and stop scraping

Wikipedia on Monday laid out a simple plan to ensure its website remains supported in the age of artificial intelligence, despite declining traffic.

In a blog post, the Wikimedia Foundation, the organization that runs the popular online encyclopedia, called on AI developers to use its content “responsibly” by ensuring their contributions are properly attributed and the content is accessed through its paid product, the Wikimedia Enterprise platform.

The paid subscription product allows companies to use Wikipedia content at scale without “heavily taxing Wikipedia servers,” as the Wikimedia Foundation blog post explains. Additionally, the paid nature of the product allows AI companies to support the nonprofit’s mission.

While the post stops short of threatening penalties or any type of legal action for using material through scraping, Wikipedia recently noted that AI bots have been scraping their website while trying to appear like a human. After updating its bot detection systems, the organization found that unusually high traffic in May and June came from AI bots that were trying to “evade detection.” Meanwhile, it said “human page views” were down 8% year over year.

Now Wikipedia is setting out its guidelines for AI developers and providers, saying that generative AI developers must provide attribution to give credit to the human contributors whose content it uses to create its output.

“For people to trust information shared online, platforms must make it clear where the information comes from and increase the chances of visiting and participating in those sources,” the post said. “With fewer visits to Wikipedia, fewer volunteers may be growing and enriching the content, and fewer individual donors may be supporting this work.”

Earlier this year, the organization released its AI for Editors strategy, which it said would use AI to help editors with workflows around tedious tasks, translation automation, and other tools that help editors, not replace them.