For developers
The same record,
in JSON.
Almost everybody who lands on CrawlState owns a website and wants an answer about it, which is what the rest of the site is for. This page is for the small number of people who would rather query the record than read a page: agencies with a few hundred client sites, hosting companies, and anybody building something that needs to know what a domain declares.
What you can ask it
One domain per call, or a list. Each one comes back with what we read in robots.txt, what the response headers carried, the date we read it, and — where we have measured it — what the origin actually served to a named AI crawler. Declared and actual are separate fields on purpose, because on plenty of domains they disagree.
- Verdict and the evidence behind it, never a score.
- The sweep date attached to every reading.
- A flag for domains sitting behind Cloudflare.
- Nothing inferred: if we could not see it, the field says so.
Access is by arrangement while the schedule settles down. Tell us roughly how many domains and how often, and we will tell you whether it is free or what it costs.
GET /v1/domain/casamonica.com
{
"domain": "casamonica.com",
"declared": "block",
"evidence": "robots.txt names AI crawlers",
"actual": "served 200 to all 3",
"agrees": false,
"cloudflare": true,
"seen": "2026-09-06" }Not a developer?
Start here instead.
Type your domain and read the answer in English.