Free tool · Runs in your browser
Choose allow or block for each of 18 AI crawlers, or start from a preset that allows the crawlers feeding answers and blocks the ones that only feed training. helloAISearch writes the robots.txt in your browser and re-checks it, so you see the resulting verdict for every crawler before you publish.
Want this checked on every engine, on a schedule? Start the free 3-day trial. No credit card.
One check is a snapshot. A schedule is a measurement.
helloAISearch runs your prompts on ChatGPT, Claude, Gemini and Perplexity on a schedule, highlights every mention, and turns each gap into one move per day.
Questions
It allows the crawlers that feed live answers and citations (OAI-SearchBot, ChatGPT-User, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Googlebot, Bingbot, Amazonbot, DuckAssistBot) and blocks the ones that only collect training data (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, meta-externalagent, CCBot, Bytespider, cohere-ai). Flip any row you disagree with.
A crawler with no group of its own follows the * group, so listing it is not required. The generator lists it anyway so the choice is documented and survives a later edit that tightens the * group.
Paste your current file into the existing field and it is kept, unchanged, above the generated block. The verification list re-parses the whole file, so you see the combined result per crawler before you publish, including any group in your file that overrides a choice.
The major vendors state that theirs do; some smaller crawlers do not. robots.txt is a request, not a wall. For a hard block use your CDN or firewall, and remember the same rule can block the crawlers that would have cited you.
Related tools
Read next
Free 3-day trial · No credit card · Cancel anytime
Three minutes from signup to reading what ChatGPT and Gemini say about your brand. Everything the trial finds is saved whether or not you subscribe.
hello, AI search.