What is GPTBot?

By the Variask team · Updated

Definition

GPTBot is OpenAI's web crawler that collects publicly available pages that may be used to train OpenAI's models. It is separate from OAI-SearchBot, which powers search in ChatGPT.

GPTBot vs other OpenAI bots

  • GPTBot: crawls for model training.
  • OAI-SearchBot: crawls to show sites in ChatGPT search results.
  • ChatGPT-User: visits pages when a user's request needs them.

You control each one separately in robots.txt using its user agent name.

What it means for brands

  • Allow OAI-SearchBot if you want to be found and cited in ChatGPT search, whatever you decide about GPTBot.

Questions

GPTBot:
common questions

Add a robots.txt rule with User-agent: GPTBot and Disallow: / . This does not block OAI-SearchBot.

Related terms

Keep exploring

Technical

AI crawlers

AI crawlers are bots run by AI companies that visit websites to collect content, either to train models or to fetch pages live when an assistant answers a question.

Read

Technical

robots.txt

robots.txt is a text file at the root of a website that tells crawlers, including AI crawlers, which parts of the site they may or may not visit.

Read

How AI search works

Large language model (LLM)

A large language model (LLM) is an AI model trained on very large amounts of text to understand and write language. LLMs power assistants such as ChatGPT, Gemini, Claude, Grok and DeepSeek.

Read
Full glossary