Photo via The Straits Times

AI giants promise publishers new way to block some scraping

Read the original at The Straits Times ↗
Cite

The Straits Times. (2026, October 31). AI giants promise publishers new way to block some scraping. The Straits Times. https://www.straitstimes.com/world/ai-giants-promise-publishers-new-way-to-block-some-scraping

Dated from the outlet's own feed. Check the original for its byline.

The Straits Times · collected 2026-10-31 · by The Straits Times

Quick Summary

On October 30, major AI companies including OpenAI, Google, Microsoft, and Anthropic announced plans to help publishers block content scraping more easily by providing a standardized command recognized by all participating firms. This new protocol aims to streamline the process of preventing AI models from training on copyrighted material without permission; however, full implementation is not expected until 2027 for some companies like Microsoft. The move comes amid legal challenges and growing concerns over unauthorized use of published content by AI systems.
Written by AI on 2026-11-01, from this article's own text only (the story summary below draws on all the coverage).

AI analysis

Story summary

Google, OpenAI, Microsoft, and Anthropic pledged on October 30, 2026, to provide publishers with a single standardized method to prevent their content from being scraped by AI models. Currently, publishers must use multiple blocking commands for each bot that collects data, but the new protocol will allow them to enter one standard instruction recognized and followed by any adopting AI company. Microsoft plans to implement this starting February 1, 2027, while Google aims to do so sometime in 2027; OpenAI and Anthropic have already begun or plan to roll it out soon. This announcement comes amid growing criticism and legal battles from publishers who are facing a surge in bots scraping their articles for AI training purposes. For instance, the New York Times sued OpenAI and Microsoft in December 2023 over allegations of using millions of its articles without permission.

Written for “AI Giants Aid Publishers Scraping” on 2026-11-22, grounded in this article and the 0 other(s) covering the same event.

Compared with similar articles

This article reads unscored and hedges 14% of its claims. Each row says how that neighbour differs.
New York Post
Leans strongly right Hedges 4 of 21 claims Outlet record 66/100
“While both articles discuss AI companies' interaction with news publishers regarding web scraping, Article A focuses on conservative groups warning against it in court filings, whereas Article B reports on a promise by AI giants to provide a new method for publishers to block scraping.”

Story

AI Giants Aid Publishers Scraping
Technology · 1 article(s) covering the same event.
How this article was measured signals, leaning charts, publisher record, byline, topics and claims

Signals How these are calculated →

Claims extracted
14
sentences that state something as happening or true
Uncertain
14%
2 of 14 hedged ("reportedly", "may", "according to")
Leaning
not political
takes no side on a contested political question
Publisher signals
64/100
The Straits Times's corrections and hedging, across everything collected from it. Not a measure of accuracy.
Outlets on this story
1
Technology
Same framing elsewhere
1
articles that frame the event the way this one does
Analyzed 2026-11-01 · how these are computed

How this is being covered How these are calculated →

Article leaning vs. publisher reliability
Source leaning vs. consistency

Publisher

The Straits Times · 5694 article(s) · 7 correction(s) detected
Running correction rate · 7 correction(s)

Who wrote this

The Straits Times
3995 article(s) here · 1 carrying a prediction
The latest draft expressed “deep concern” that global warming would cross 1.5 deg C above pre-industrial levels in the next few years.
2026-11-22 · assertive framing · Fossil fuels still missing from latest COP31 draft deal
Mejia had also overstayed her visa, ICE said, adding that she would be charged with assaulting a federal officer.
She will be charged with assaulting a federal officer, according to ICE.
TEHRAN - Iran will impose rolling power cuts from Nov 22 as damage to key infrastructure during the Middle East war deepens an already severe energy crunch, state media reported.
2026-11-22 · assertive framing · Iran to ration electricity as war hits energy network
Party strategists anticipate that the field of 2028 aspirants could be far wider than what was expected at the start of the administration, when Trump joked about seeking a constitutionally-barred third term.
Former prime minister Naftali Bennett, an important figure in Eisenkot’s camp, on Nov 21 called for a “Zionist unity government headed by Eisenkot” and said his party would refuse to join up with Netanyahu or his bloc.
The Wall Street Journal reported on the investigation earlier this week, although it was unable to establish its precise focus. The probe could end without any charges being brought against the company. Questions about Nvidia’s financial disclosures have intensified amid scrutiny of its dealings with Megaspeed, a Singapore-based company investigated by American authorities over suspected shipments of restricted AI chips to China.
Almost all flights are set to be cancelled in Belgium on Nov 23 after the country’s air traffic controller said its staff will take part in a national strike against government austerity measures.
Climate talks in Antalya, Turkey, approached a final deal on Sunday with the announcement of a new package that COP31 presidents Australia and Turkey hope will achieve consensus on how to advance the fight against global warming.
Turkey on Nov 21 dismissed a Malaysian announcement that it would take in 10,000 Rohingya refugees currently living in Malaysia.
Also by The Straits Times
Nothing else under this byline is closely related to this article, so these are simply their most recent.
All 3995 articles by The Straits Times →

Topics

Anthropic ChatGPT Google Microsoft OpenAI

Subjects

Microsoft ORG · 5× OpenAI ORG · 3× Anthropic ORG · 2× Google ORG · 2× California GPE · 1× ChatGPT ORG · 1× Cloudflare ORG · 1× SAN FRANCISCO GPE · 1× Times ORG · 1× the New York Times ORG · 1×

Narrative

Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol.
framing: assertive · carried by 1 article(s) · first seen 2026-11-01
Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol.
2026-11-01 · The Straits Times
AI giants promise publishers new way to block some scraping · assertive framing

Claims (14 extracted, 2 hedged)

AI giants promise publishers new way to block some scraping AI generated SAN FRANCISCO, California – asserted
giants → promise → scraping
OpenAI, Google, Microsoft and Anthropic pledged on Oct 30 to give media outlets and other websites a single way to prevent their content from being scraped to train artificial intelligence models, amid growing criticisms and legal battles from news publishers. asserted
OpenAI → pledge → publishers
Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol. asserted
that → have → protocol
The solution will not fully take effect for months, though. asserted
solution → take → months
Microsoft will implement it beginning Feb 1, 2027, and Google sometime in 2027. asserted
Microsoft → implement → 2027
ChatGPT-maker OpenAI said it will roll it out “in the coming months”, while Claude-maker Anthropic said it has “already started” rolling it out. asserted
it → say → it
The announcement comes as publishers are facing a surge in bots that scrape their articles to feed AI models. asserted
that → come → models
Some, like the New York Times, have taken the matter to court. asserted
Some → take → court
In December 2023, the Times sued OpenAI and Microsoft for allegedly using millions of its articles without authorisation or payment to train ChatGPT. uncertain
Times → sue → ChatGPT
The two tech companies are invoking an exception to copyright law known as fair use. asserted
companies → invoke → use
Meanwhile, internet infrastructure provider Cloudflare has blocked AI bots by default on new customer sites since July 2025, and a standard called Really Simple Licensing already allows publishers to charge bots that visit their pages. asserted
that → block → pages
The new setting will apply only to future training of models but does not prevent bots from crawling otherwise, and content scraping will be allowed by default until the setting is changed by a publisher, Microsoft said in a blog post. Opting out of AI training could reduce the chances that a publisher’s content will surface in AI apps and experiences, Microsoft and Google said in separate blog posts. uncertain
Microsoft → apply → posts
“We believe creating a common signal that conveys such preferences to multiple providers at once is an important step. asserted
that → believe → providers
And we look forward to working with other AI companies on the standards and their implementation across the industry,” Meta spokesman Andy Stone wrote on X, conveying his support for the effort. asserted
Stone → look → effort
Give feedback · no sign-in needed

No sign-in needed for now. Your note is sent without a name, so it cannot be answered.

History Support