Reported News
OpenAI will honor a shared robots.txt signal that lets sites opt out of AI training
OpenAI said on October 30, 2026 that it will honor a shared robots.txt signal for AI training preferences. The mechanism is an IETF Internet-Draft published the same day: site owners write robots.txt groups keyed by a policy token, usage_ai-training, to allow or refuse training per path.
- Changes reported
- 5
- listed below, each from a source we read
- We tested
- 0
- we did not run it ourselves
- Gaps we flag
- 2
- things we could not verify
- Sources
- 2
- 2 primary, linked below
What changed
| # | What the source says | Status |
|---|---|---|
| 01 | OpenAI's news feed says it will honor a shared robots.txt signal so website operators can tell participating AI providers their training preferences. | Reported |
| 02 | The signal is defined in draft-kelley-aipref-training-preference-signal-00, an individual IETF Internet-Draft published October 30, 2026 and expiring May 3, 2027. | Reported |
| 03 | Sites write ordinary robots.txt groups whose User-Agent line is a policy token, usage_ai-training, then Disallow or Allow paths; the draft's example blocks training everywhere except a press-kit folder. | Reported |
| 04 | The draft says these training preferences are separate from crawl rules, so a site can allow crawling while refusing training. | Reported |
| 05 | The draft is not an IETF standard; its token follows the definition the IETF aipref Working Group has reached. | Reported |
reported by the sourcemeasured or run by usnot verified
Primary source: OpenAI News RSS feed (October 30, 2026 entry). We are the fast, independent index; the vendor’s own page is the source of record for this release.
In the source’s words
“binds usage preferences (e.g., "no AI training") to paths by writing ordinary robots.txt groups keyed by a per-purpose policy token in the User-Agent line.”
What we have not verified
- OpenAI's announcement page on openai.com blocks our fetcher; OpenAI's commitment is taken from its own news feed entry.
- We did not check which other providers have committed to the signal, or when OpenAI's crawlers start reading it.
This brief is Reported: written from the sources below, not from running the release ourselves.
Sources
- OpenAI News RSS feed (October 30, 2026 entry)primary source · fetched 2026-10-31
- IETF Internet-Draft: AI training preference signalprimary source · fetched 2026-10-31
Related
Spotted an error? See our corrections policy or email the address on the contact page. Markdown version for agents: /news/openai-robots-txt-ai-training-preference-signal.md
Want these briefs in your Google Top Stories? Add Agentic Weekly as a preferred source on Google.
The Weekly Verdict · free
5 new-model use cases. Every Monday.
A free email every Monday: the week's few big AI releases, then 5 short use cases for the new models, each with a prompt you can paste.
Free. Double opt-in. One click to unsubscribe.