What does your robots.txt tell AI crawlers?

Run a per-crawler check against your robots.txt: for each AI crawler token, find the rule group that applies to it and whether the page path is allowed. This checker does that for all 9 identities in one pass. Keep in mind that a permissive robots.txt does not guarantee delivery — a firewall or CDN can still block the request, which is why the checker reports evidence states rather than assuming access.

The checker fetches the robots.txt file for the domain you submit, then evaluates — separately for each supported AI crawler — which rule group applies to that crawler and whether the page path you submitted is allowed or blocked under those rules. The evaluation follows the RFC 9309 robots exclusion conventions: most-specific matching user-agent group, longest matching path rule wins, and Allow overrides Disallow at equal length.

The checker evaluates all 9 robots.txt tokens covered by the Optimus Pass visibility engine: Googlebot, OAI-SearchBot, PerplexityBot, Claude-SearchBot and Bingbot for search and discovery; GPTBot, Google-Extended and ClaudeBot for model training and other AI uses; and OAI-AdsBot for ChatGPT Ads landing-page validation. Each is reported with its purpose so you do not confuse a training crawler with a search crawler.

No, the checker cannot tell you that. robots.txt controls whether a crawler may request your pages. Whether an AI system retrieves, ranks, or cites your content depends on many factors outside robots.txt, and no tool can measure those from a robots file. Allowed means the door is open, not that anyone walked through it — the same no-guarantee rule applies across all 9 tokens.

The two controls are independent. According to OpenAI's published crawler documentation, OAI-SearchBot and GPTBot are independent controls: OAI-SearchBot surfaces sites in ChatGPT search results, while GPTBot crawls content that may be used for model training. You can allow one and block the other. The checker shows both states separately so the distinction stays visible.

No readable robots.txt exists for that origin — the file is missing, unreachable, or returns no readable text. In that case the engine reports all 9 crawlers as not evaluated instead of assuming access. That is a deliberate honesty rule: no evidence, no confirmation.

Yes. Your first check is free with no account and no credit card — run it on any public domain or page URL. After that, create a free account to keep checking. The full visibility audit — which adds the 3-lens SEO, AEO, and GEO diagnostics alongside this crawler view — also has a free snapshot.