Claude-User

Questions about Claude-User

What happens if I block Claude-User?

Anthropic says Claude then can't retrieve your content in response to user queries, which may reduce your site's visibility for user-directed web search. Anthropic says its bots honour robots.txt.

Anthropic documents Claude-User as the crawler that lets Claude access a website when a user directs it to do so as part of their query — a user-initiated request, not a bulk crawl.

Why this is different from ClaudeBot

Claude-User is tracked separately from the training-purpose ClaudeBot and the search-purpose Claude-SearchBot. A website can restrict one without restricting the others — CrawlPact’s “Allow Search, Block Training” preset relies on exactly this kind of distinction.

Site-owner controls

Standard robots.txt disallow rules apply. See /limitations for what a robots.txt rule can and cannot guarantee.

How to verify Claude-User requests

Anthropic publishes its crawlers' IP addresses at https://claude.com/crawling/bots.json. Operator documentation.

A user-agent string alone can be copied by anyone; confirming a request really came from Anthropic needs your server or CDN logs. CrawlPact audits declared policy and does not see your traffic.

Example robots.txt configuration

To disallow Claude-User specifically, without affecting any other crawler:

User-agent: Claude-User
Disallow: /

If no dedicated User-agent: Claude-User group exists in a domain'srobots.txt, this crawler falls back to whatever the wildcard User-agent: * group says (RFC 9309) — see robots.txt syntax basics for how group selection works. A robots.txt rule is a published instruction; it does not technically prevent access.

Official source

https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Verified against the source above as of 24 July 2026 — see how CrawlPact verifies crawler information.