The record · 7 September 2026
Not readableDoes gtop100.com block AI crawlers?
We cannot say. gtop100.com answered us, but it did not give us its robots.txt, so there is nothing to report.
Read straight out of the CrawlState record, a dated sweep of 1,163,843 domains on 7 September 2026. Nothing on this page is estimated, and nothing about gtop100.com is inferred from what similar sites do.
What we wrote down
One of five: block, charge, signal, open, silent. The record keeps the strongest thing a site said, not every thing it said.
The note the sweep wrote at the moment it decided, in its own words.
403 means gtop100.com answered but refused to serve us the file. The verdict still stands on what we could read.
A record has a date on it or it is worthless. This one has this date.
What that actually means
A website's instruction to AI lives in a file called robots.txt. This site answered us, but its homepage came back as an error rather than a page, so that file was never read. That is a check we could not run, not a fault we found — and it is certainly not permission. A site whose robots.txt we cannot read may be refusing every AI crawler in the world, or none of them. Anyone telling you which, from outside, is guessing.
How gtop100.com compares
136,479 of the 899,525 domains that answered us are in the same position, which is 15.2%. Until recently CrawlState counted these as declaring nothing, which was wrong: it published a permission that was never measured.
Shares are of the 899,525 domains that answered. A further 264,318 never answered at all, which is why they are not in the bars above.
Find out what happens now
The record says what gtop100.com declared on 7 September 2026. It does not say whether anyone obeyed it. The live report asks the site for its own homepage under three named AI crawlers and tells you what each one was actually served, which is usually the surprise.
If this is your site and the answer above is not the one you wanted, we will set the policy properly for you — robots.txt, Cloudflare and licence terms, done and checked. See what that costs.
Method: homepage, robots.txt and sitemap, on 7 September 2026. We record what came back and nothing else. How a verdict is decided · The awkward questions · The same record over an API