Question 1

Should I allow GPTBot to crawl my site?

Accepted Answer

If you want your content to be reachable by ChatGPT and future OpenAI models, yes. If you sell that content or object to it training models, blocking GPTBot is a reasonable choice — just know that OAI-SearchBot (ChatGPT search) is a separate bot, so you can allow search visibility while opting out of training.

Question 2

What is llms.txt?

Accepted Answer

A proposed standard: a markdown file at /llms.txt that gives AI systems a curated map of your most important pages with short summaries. It is not universally consumed yet, but it costs ten minutes to publish and several AI tools already read it — an easy, low-risk win.

Question 3

What does a score of 80+ actually mean?

Accepted Answer

That AI engines can fetch your homepage, parse structured data about it, and read the basic signals they quote in answers. It means you are eligible to be cited, not guaranteed to be — content quality and topical authority decide the rest.

Question 4

I put "Disallow: /" under "User-agent: GPTBot" but left "User-agent: *" wide open — is GPTBot still blocked?

Accepted Answer

Yes, and this is the most common robots.txt mistake this tool exists to catch. A group naming a bot specifically always wins over the wildcard group, no matter which one looks more permissive or which one appears first in the file — so a targeted Disallow overrides a generous wildcard Allow every time. It cuts the other way too: an "Allow: /" under a bot-specific group does nothing to override a blanket "Disallow: /" sitting under a different group; it only helps the bot it names. Every card in the report states which group decided, so you can check this against your own file.

Question 5

Why is my score low but my Google traffic is fine?

Accepted Answer

Classic Googlebot and AI crawlers are different user-agents with different robots rules. Many sites (or their CDN/firewall defaults) block GPTBot, CCBot or PerplexityBot without ever touching Googlebot, so organic traffic stays healthy while AI engines cannot read the site at all.

Question 6

Does blocking AI bots protect my content from AI completely?

Accepted Answer

No. robots.txt is a request that reputable crawlers honour, not an enforcement mechanism, and your content can still enter training data through third-party datasets, syndication or scraping. It is still worth setting deliberately — it is the standard, documented signal of your intent.

Question 7

What is the difference between the noai signal and blocking a crawler in robots.txt?

Accepted Answer

robots.txt is a fetch-level gate, checked before a crawler requests the page at all. noai (and noimageai) is a page-level request sent inside the meta robots tag or the X-Robots-Tag header, asking a crawler that has already fetched the page not to use its content for AI training. Neither is enforced by anything on your server — both depend on the crawler operator choosing to honour them. This checker reports noai but does not score it, because opting out is a business decision, not a defect. A noindex or nofollow signal in that same tag is reported separately and much more severely, since it can suppress a page from AI answers entirely regardless of every other check.

Question 8

Can this tool check more than my homepage?

Accepted Answer

Not right now — every check runs against your homepage only. A full-site crawl is a heavier, different product (a queue, not a single request), and one homepage is the honest scope for a free tool that runs synchronously with no email gate.

AI Visibility Checker

Exactly what this does

The 10 crawlers we evaluate

What the score is made of

Check your AI visibility

Browse all the tools