OpenAI's three web crawlers: GPTBot, OAI-SearchBot, and ChatGPT-User

Most discussion of "blocking AI" treats OpenAI as a single crawler. It isn't. OpenAI publishes three distinct user agents and each one represents a different ingestion path with different consequences if you block it.

The three bots

GPTBot — Used to crawl pages for model training. Anything GPTBot fetches may end up as part of the training corpus for future GPT models. Blocking GPTBot keeps your content out of future model weights but does not affect whether ChatGPT can cite you in answers today.

OAI-SearchBot — Used to maintain the ChatGPT search index — the corpus ChatGPT consults when answering questions that involve current information. Blocking OAI-SearchBot removes your domain from that index, which means ChatGPT cannot cite you in real-time search-grounded answers.

ChatGPT-User — Fired when an individual user clicks a citation, asks ChatGPT to read a specific URL, or otherwise triggers an on-demand fetch. Blocking ChatGPT-User prevents end-users from pulling your page into their ChatGPT conversation.

Why the distinction matters

Three failure modes look identical to a publisher who only sees "ChatGPT mentions us less" but each requires a different fix:

• If GPTBot is blocked but OAI-SearchBot is allowed: future model training is locked out, but search-grounded ChatGPT can still cite you. This is the "I want to opt out of training but stay discoverable" stance.
• If OAI-SearchBot is blocked: search-grounded answers won't surface your URLs even when GPT's parametric knowledge mentions you.
• If ChatGPT-User is blocked: a user asking "summarise this article: yourdomain.com/post" will get an error.

A blanket User-agent: * block hits all three, which is rarely what publishers actually want. Differentiating in robots.txt lets you choose which ingestion paths to allow.

Example robots.txt patterns

Allow training, allow search, allow user-initiated fetches (the default if you do nothing):

# Implicit — no blocks needed.

Block training, allow search + user fetches:

User-agent: GPTBot
Disallow: /

Allow training, block search index, allow user fetches (uncommon):

User-agent: OAI-SearchBot
Disallow: /

Block all three:

User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /

How AI Visibility Monitor checks this

The Crawler Access tool in Monitor parses your robots.txt and reports which of these three user agents are allowed. The "AI discovery" score weights OAI-SearchBot most heavily because it's the surface that drives same-day citations in ChatGPT — the highest-leverage of the three for visibility.