Does your robots.txt block GPTBot and friends?
Paste your robots.txt. The check tells you for every relevant AI crawler whether it may read the page. It runs in your browser — nothing is uploaded.
Found at your-domain.com/robots.txt. The text stays in your browser.
Result
Paste your robots.txt above — the result appears immediately.
How this works
What counts as a block
Only a full block counts, meaning Disallow: / — either in the bot’s own section or via User-agent: * without a milder exception for it. Individual disallowed directories are not an exclusion: the crawler may still read everything else.
The Google-Extended trap
Google-Extended controls use by Gemini and AI Overviews only. Blocking that token does NOT remove you from normal Google Search — that is Googlebot. The reverse also holds: allowing Googlebot does not mean the content may appear in AI Overviews.
robots.txt is a request, not a lock
The file is a voluntary agreement. Reputable operators honour it, others do not — Bytespider has been reported to ignore it repeatedly. To truly lock a crawler out, block it at server or CDN level as well.
What open access does not yet mean
A crawler that may read still finds nothing if the content only appears through JavaScript. No major AI crawler executes JavaScript. Access is the precondition, not the result.
Frequently asked questions
- Where do I find my robots.txt?
- At your-domain.com/robots.txt. If nothing is there, no file exists — and no crawler is blocked.
- Should I block AI crawlers?
- If you want to appear in AI answers, do not block them. Blocking makes sense when content must not be used for model training and visibility in AI answers is secondary. Having both is only partly possible: you can allow OAI-SearchBot while keeping GPTBot blocked.
- Does blocking hurt my Google ranking?
- No, as long as Googlebot stays allowed. Google-Extended is a separate token and only affects Gemini and AI Overviews.
- Is my robots.txt uploaded anywhere?
- No. The check runs entirely in your browser; nothing is sent and nothing is stored.
Check the whole page
This tool checks one aspect. The free GEO Check scores a whole URL across 25 checks in six categories. For pages behind a login, on staging or on localhost, the browser extension computes the identical score inside your own tab.