# What is OAI-SearchBot? OpenAI search crawler

OAI-SearchBot is OpenAI’s crawler for discovering pages for ChatGPT search.

OpenAI documents OAI-SearchBot for surfacing websites in ChatGPT search. GPTBot serves a separate model-improvement purpose, so the two access decisions need not be the same.

## Separating search access from training

A publisher wants to permit OpenAI’s crawler for ChatGPT search and separately decides whether to permit GPTBot. It checks each token in the live robots.txt file and does not treat the search rule as proof the article will be cited.

## The search and training controls are separate

Check both the robots.txt rule and the infrastructure serving the page. A firewall block can prevent access even when the file allows the crawler. Conversely, allowing access establishes eligibility to be crawled, not a promise that an answer will cite the article.

## Write down what you want to allow

Start with one public article you want readers to discover. Document its intended search access and check the live rules for that route. Record a separate training preference so a later rule change can be tested against both intentions. Include the date and URL in the test record. This gives you a baseline if discovery changes after a later deployment.

## Related terms

[GPTBot](https://digitalpublishingnews.com/dictionary/gptbot/)

[robots.txt](https://digitalpublishingnews.com/dictionary/robots-txt/)

## Continue reading

[OAI-SearchBot and GPTBot: choose search access separately from training access](https://digitalpublishingnews.com/articles/openai-search-training-crawlers/): It explains the separate OpenAI crawler choices for search discovery and model improvement.

## Sources

https://developers.openai.com/api/docs/bots

https://developers.openai.com/api/docs/bots

Editorial method: https://digitalpublishingnews.com/how-we-use-ai/

Canonical: https://digitalpublishingnews.com/dictionary/oai-searchbot/