AI training opt-out: what the standard accepted by Anthropic, Google, Microsoft and OpenAI means for your site

If you publish text or images on your own site, you have probably noticed AI crawlers in your logs without knowing exactly what they were after. A new step has just been taken: according to Thurrott.com, Anthropic, Google, Microsoft and OpenAI have agreed to let website owners opt out of AI model training. The key point for a creator is that this refusal would be separate from indexing and archiving for search.

Be careful not to overread the announcement, though. It is a voluntary commitment, not a legal obligation, and the source does not describe the precise technical workings of the signal. Here is what can reasonably be taken from it, and what remains for you to check on your own side.

What was actually announced

The opt-out mechanism was developed by an IETF working group (the body that standardises internet protocols) and will be proposed as an international standard once finalised. In practice, it should let a site tell AI crawlers about its refusal, separately from indexing and archiving for search.

The companies' positions, as reported by the source, overlap:

• Microsoft says publishers should not have to choose between being visible in Bing and deciding how their work is used. According to the company, the new preferences cover training, grounding (using content to answer a query) or both, while keeping eligibility for Bing results.
• OpenAI says it will respect a common signal expressing sites' preferences for training foundational generative AI, and presents the approach as voluntary, with the benefit of communicating one preference to several participating providers.
• Anthropic has already implemented the mechanism.
• Google, Microsoft and OpenAI plan to implement it in 2027.

Why it matters for a creator — and why caution is warranted

Until now, a creator who wanted to say "don't train on my content" often had to improvise: preference files, user-agent blocking, statements in terms of use. The problem was that each provider could have its own convention, and a refusal expressed one way was not necessarily read by the others.

The value of a common signal is therefore twofold: you express a preference once, and you make it readable by several players at the same time. The fact that this refusal is separate from search indexing also matters: you may want to remain findable in search engines without feeding the training of generative models.

But the source makes a point that tempers the enthusiasm: these companies have previously collected content at scale, which makes the change "largely symbolic". In other words, the opt-out mainly protects the future of your publications, not necessarily what has already been scraped. And because the scheme is voluntary, no penalty is described for non-compliance.

Open questions to keep in mind

• The exact technical name of the signal and its detailed operation are not specified by the source.
• It is unknown whether the mechanism will actually be adopted as an international standard, and in what final form.
• The 2027 timeline for Google, Microsoft and OpenAI is reported by the source, with no guarantee it will be met.
• The voluntary nature means no binding obligation is described.
• The source details no concrete method for a creator to verify that their opt-out is respected.

How to check, concretely, what happens on your site

In the absence of a detailed official procedure in the source, here is a cautious approach based on observations you can make yourself. It does not guarantee your choice will be respected, but it gives you checkpoints.

1. Watch crawler visits in your logs

Look at the user-agents hitting your pages. Identify those that present themselves as AI or training crawlers, and note their frequency. A crawler that keeps coming after your opt-out is a signal worth documenting (date, user-agent, page concerned).

2. Check that your preference is actually published

Make sure the file or signal you use is publicly reachable where expected, and that it is not blocked by mistake (firewall, cache rule, error page). An opt-out that crawlers cannot see is useless.

3. Separate search from training

Test your visibility in search engines after setting your preference. If your site disappears from results when you only wanted to refuse training, that suggests your configuration mixes the two signals — exactly what Microsoft says it wants to avoid.

4. Document and date

Keep a record of the date you expressed your refusal, and of the observations that followed. This is useful if you later need to report a breach, or simply to compare how practices evolve.

Key takeaways

The agreement between Anthropic, Google, Microsoft and OpenAI is the first multi-stakeholder deal on AI training opt-out, and that in itself is progress for creators who publish on their own site. Anthropic already applies the mechanism; the other three announce 2027. But it is a voluntary framework, whose technical details are not yet public, and whose effect on already-collected content remains limited.

In practice: express your preference cleanly, keep it separate from your search visibility, monitor your logs, and keep records. Do not treat the opt-out as automatic protection until you have observed its effect on your own site.

Source

Anthropic, Google, Microsoft, and OpenAI Agree to Let Sites Opt Out of AI Training