Does bbc.co.uk block AI crawlers?

Yes. bbc.co.uk names 8 AI crawlers in its robots.txt and tells them to stay out.

Read straight out of the CrawlState record, a dated sweep of 449,070 domains on 6 September 2026. Nothing on this page is estimated, and nothing about bbc.co.uk is inferred from what similar sites do.

What we wrote down

Verdictblock

One of five: block, charge, signal, open, silent. The record keeps the strongest thing a site said, not every thing it said.

Reasonnames 8 AI crawler(s)

The note the sweep wrote at the moment it decided, in its own words.

HTTP status200

What bbc.co.uk answered with when we asked for its rules.

Swept6 September 2026

A record has a date on it or it is worthless. This one has this date.

What that actually means

Every website carries a small text file at the top of it called robots.txt, and it is the first thing an automated visitor reads. Naming a crawler there is a website saying, out loud, that it does not want that crawler taking its pages. It is a request rather than a lock. Companies that respect the file turn around and leave. Companies that do not respect it walk straight in, and nothing on the site stops them.

How bbc.co.uk compares

58,098 of the 347,935 domains that answered us do the same, which is 16.7%. That puts bbc.co.uk in the sixth of the web that has actually made this decision, rather than the four in five that have never opened the file.

Name an AI crawler and refuse it — bbc.co.uk58,098 · 16.7%

Publish a licence or return a price1,848 · 0.53%

Carry a Content-Signal preference only1,135 · 0.3%

Declare nothing at all286,854 · 82.4%

Shares are of the 347,935 domains that answered. A further 101,135 never answered at all, which is why they are not in the bars above.

Find out what happens now

The record says what bbc.co.uk declared on 6 September 2026. It does not say whether anyone obeyed it. The live report asks the site for its own homepage under three named AI crawlers and tells you what each one was actually served, which is usually the surprise.

Run the live report on bbc.co.uk Look up another domain

If this is your site and the answer above is not the one you wanted, we will set the policy properly for you — robots.txt, Cloudflare and licence terms, done and checked. See what that costs.

Method: two requests per domain, on 6 September 2026. We record what came back and nothing else. How a verdict is decided · The awkward questions · The same record over an API