OpenAI documents OAI-SearchBot for surfacing websites in ChatGPT search. GPTBot serves a separate model-improvement purpose, so the two access decisions need not be the same. (OpenAI crawler documentation ↗)
Jump to a section
Separating search access from training
A publisher wants to permit OpenAI’s crawler for ChatGPT search and separately decides whether to permit GPTBot. It checks each token in the live robots.txt file and does not treat the search rule as proof the article will be cited.
The search and training controls are separate
Check both the robots.txt rule and the infrastructure serving the page. A firewall block can prevent access even when the file allows the crawler. Conversely, allowing access establishes eligibility to be crawled, not a promise that an answer will cite the article.
Write down what you want to allow
Start with one public article you want readers to discover. Document its intended search access and check the live rules for that route. Record a separate training preference so a later rule change can be tested against both intentions. Include the date and URL in the test record. This gives you a baseline if discovery changes after a later deployment.