The Aporia · AI giants promise publishers new way to block some scraping

On October 30, major AI companies including OpenAI, Google, Microsoft, and Anthropic announced plans to help publishers block content scraping more easily by providing a standardized command recognized by all participating firms. This new protocol aims to streamline the process of preventing AI models from training on copyrighted material without permission; however, full implementation is not expected until 2027 for some companies like Microsoft. The move comes amid legal challenges and growing concerns over unauthorized use of published content by AI systems.

Written by AI on 2026-11-01, from this article's own text only (the story summary below draws on all the coverage).

Story summary

Google, OpenAI, Microsoft, and Anthropic pledged on October 30, 2026, to provide publishers with a single standardized method to prevent their content from being scraped by AI models. Currently, publishers must use multiple blocking commands for each bot that collects data, but the new protocol will allow them to enter one standard instruction recognized and followed by any adopting AI company. Microsoft plans to implement this starting February 1, 2027, while Google aims to do so sometime in 2027; OpenAI and Anthropic have already begun or plan to roll it out soon. This announcement comes amid growing criticism and legal battles from publishers who are facing a surge in bots scraping their articles for AI training purposes. For instance, the New York Times sued OpenAI and Microsoft in December 2023 over allegations of using millions of its articles without permission.

Written for “AI Giants Aid Publishers Scraping” on 2026-11-02, grounded in this article and the 0 other(s) covering the same event.

📰 AI Giants Aid Publishers Scraping

Technology · 1 article(s) covering the same event.

This article reads unscored and hedges 14% of its claims. Each row says how that neighbour differs.

Conservative groups warn letting AI giants ‘recklessly crawl the internet’ would jeopardize news outlets, national security: court filling Different event

New York Post

⚖️ Leans strongly right Hedges 4 of 21 claims Outlet record 66/100

“While both articles discuss AI companies' interaction with news publishers regarding web scraping, Article A focuses on conservative groups warning against it in court filings, whereas Article B reports on a promise by AI giants to provide a new method for publishers to block scraping.”

Article leaning vs. publisher reliability

Source leaning vs. consistency

The Straits Times

2683 article(s) here · 1 carrying a prediction

🔮 How a blue wave could test Trump in his final two years AI

2026-11-02 · assertive framing · How a blue wave could test Trump in his final two years

🔮 Israel elections chief says inquiry into irregularities will not undermine result AI generated JERUSALEM –

2026-11-01 · assertive framing · Israel elections chief says inquiry into irregularities will not undermine result

🔮 Democrats are seeking to wrest control of both chambers of Congress from Republicans in Nov 3’s midterms, with the Michigan race a key battleground that could help them flip control of the Senate.

2026-11-01 · assertive framing · Democratic divisions shake up key Michigan Senate race

🔮 Republicans made a final push on Nov 1 to prevent a predicted thumping in US midterm elections where Democrats are gunning to win Congress and impose a first political check on President Donald Trump’s unprecedented pursuit of power.

2026-11-01 · assertive framing · Trump’s Republicans scramble to stop Democratic midterms wave

🔮 German Interior Minister Alexander Dobrindt, a member of the conservative Christian Social Union, said perpetrators would be identified and held to account.

2026-11-01 · assertive framing · German ministers vow tough response to Frankfurt riots and looting

🔮 US President Donald Trump said a US-Canada trade deal is “very close” after talks with Canadian Prime Minister Mark Carney, who said both sides would work on a “mutually beneficial” outcome in the weeks ahead.

2026-11-01 · assertive framing · Trump says US-Canada trade deal is close after talks with Carney

🔮 Zelensky says Ukraine will stop hitting Russia’s refineries if US provides missiles WASHINGTON - Ukrainian President Volodymyr Zelensky said on Nov 1 that his country would stop hitting Russian oil refineries – a key demand of US President Donald Trump – if Washington provided Kyiv with powerful Patriot interceptor missiles.

2026-11-01 · assertive framing · Zelensky says Ukraine will stop hitting Russia’s refineries if US provides missiles

🔮 Throughout the eight-month war, Trump has pursued dual tracks with Tehran: a campaign of air strikes and other attacks, while seeking to reach a deal that would jumpstart peace talks and fully open the Strait of Hormuz to vital oil and gas traffic.

2026-11-01 · assertive framing · Trump holds call with Netanyahu over state of Iran war: Report

🔮 “The FAA and National Transportation Safety Board will investigate,” the crash, the Federal Aviation Administration said in a statement.

2026-11-01 · assertive framing · Nine dead in Learjet plane crash near Trump’s Florida home

🔮 While You Were Sleeping: 5 stories you might have missed, Nov 2, 2026 Longest truce of Ukraine war ends in Kherson

2026-11-01 · assertive framing · While You Were Sleeping: 5 stories you might have missed, Nov 2, 2026

Also by The Straits Times

Putin breaks down his isolation, with a big assist from Trump

2026-11-02 · The Straits Times

How a blue wave could test Trump in his final two years

2026-11-02 · The Straits Times

Democratic divisions shake up key Michigan Senate race

2026-11-01 · The Straits Times

Israel elections chief says inquiry into irregularities will not undermine result

2026-11-01 · The Straits Times

Nothing else under this byline is closely related to this article, so these are simply their most recent.

All 2683 articles by The Straits Times →

Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol.

framing: assertive · carried by 1 article(s) · first seen 2026-11-01

🔮 Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol.

2026-11-01 · The Straits Times

AI giants promise publishers new way to block some scraping · assertive framing

AI giants promise publishers new way to block some scraping AI generated SAN FRANCISCO, California – asserted

giants → promise → scraping

OpenAI, Google, Microsoft and Anthropic pledged on Oct 30 to give media outlets and other websites a single way to prevent their content from being scraped to train artificial intelligence models, amid growing criticisms and legal battles from news publishers. asserted

OpenAI → pledge → publishers

that → have → protocol

The solution will not fully take effect for months, though. asserted

solution → take → months

Microsoft will implement it beginning Feb 1, 2027, and Google sometime in 2027. asserted

Microsoft → implement → 2027

ChatGPT-maker OpenAI said it will roll it out “in the coming months”, while Claude-maker Anthropic said it has “already started” rolling it out. asserted

it → say → it

The announcement comes as publishers are facing a surge in bots that scrape their articles to feed AI models. asserted

that → come → models

Some, like the New York Times, have taken the matter to court. asserted

Some → take → court

In December 2023, the Times sued OpenAI and Microsoft for allegedly using millions of its articles without authorisation or payment to train ChatGPT. uncertain

Times → sue → ChatGPT

The two tech companies are invoking an exception to copyright law known as fair use. asserted

companies → invoke → use

Meanwhile, internet infrastructure provider Cloudflare has blocked AI bots by default on new customer sites since July 2025, and a standard called Really Simple Licensing already allows publishers to charge bots that visit their pages. asserted

that → block → pages

The new setting will apply only to future training of models but does not prevent bots from crawling otherwise, and content scraping will be allowed by default until the setting is changed by a publisher, Microsoft said in a blog post. Opting out of AI training could reduce the chances that a publisher’s content will surface in AI apps and experiences, Microsoft and Google said in separate blog posts. uncertain

Microsoft → apply → posts

“We believe creating a common signal that conveys such preferences to multiple providers at once is an important step. asserted

that → believe → providers

And we look forward to working with other AI companies on the standards and their implementation across the industry,” Meta spokesman Andy Stone wrote on X, conveying his support for the effort. asserted

Stone → look → effort

AI giants promise publishers new way to block some scraping

Quick Summary

AI analysis

Story

Compared with similar articles

Signals How these are calculated →

How this is being covered How these are calculated →

Publisher

Who wrote this

Topics

Subjects

Narrative

Claims (14 extracted, 2 hedged)