OpenAI says it will honour a proposed cross-provider robots.txt signal that lets website owners express preferences about using site content for generative-AI training. Implementation is still forthcoming.
Website owners may soon have a simpler way to express whether their content should be used to train generative-AI models. On October 30, 2026, OpenAI announced that it will honour a proposed shared signal in robots.txt, the file publishers already use to communicate crawler preferences. The idea is to give multiple participating AI providers a common way to read a site’s stated training preference.
Today, website operators often need to maintain separate rules for individual crawlers. A shared policy token could reduce that duplication by letting a publisher state a training preference in one place, rather than writing a separate rule for each provider.
What the proposed signal does
The Internet-Draft describes a machine-readable preference token called __usage_ai-training. A website could use a robots.txt group associated with that token to express a preference for or against AI training on particular paths. The approach reuses the existing robots.txt structure rather than introducing an entirely new file format.
That distinction matters: the document is currently an Internet-Draft, not an approved IETF standard. It is work in progress and can change. OpenAI says it aims to implement the shared signal in the coming months and will publish further details when its implementation is ready.
What it does not do
The proposed preference is not a replacement for existing crawler controls. OpenAI says its current robots.txt controls will continue to apply. The draft also distinguishes content-use preferences from ordinary crawling rules: a training preference is not itself a general instruction to block a crawler from fetching a page.
In practical terms, publishers should not assume that adding the new token today will automatically enforce a universal block across every AI provider. The value depends on which providers recognise the signal and how they implement it. Website owners should continue using documented crawler-specific controls where they need to manage access.
Why publishers should follow this
A common preference format could make it easier for independent publishers, newsrooms and businesses to communicate how their content may be used. The next things to watch are the IETF working-group discussion, final wording of the draft and the implementation details published by participating providers.