Go to ShadowSync SEO → AI Bot Access.
Each AI crawler has an Allow and a Block option. Your robots.txt is updated to match.
| Crawler | Belongs to |
|---|---|
| GPTBot | ChatGPT (OpenAI) |
| ChatGPT-User | ChatGPT browsing |
| ClaudeBot | Claude (Anthropic) |
| anthropic-ai | Anthropic (older crawler) |
| Google-Extended | Google Gemini and AI Overviews |
| PerplexityBot | Perplexity |
| CCBot | Common Crawl |
| Bytespider | ByteDance |
The screen groups them into Training & Search Crawlers, which visit regularly, and Live Browsing & Citation Crawlers, which fetch a page when someone asks the AI a question.
Allowing a crawler lets that AI read and cite your content. Blocking it stops it reading your pages.