CrawlPactCrawlPact audits and monitors what a website's own declared signals — robots.txt, meta robots tags, X-Robots-Tag headers, llms.txt, RSL, and Content Signals — say about AI crawler access, and reports that honestly against a versioned, source-backed registry of documented AI crawlers.
What CrawlPact is not
CrawlPact does not use an AI model to produce its results. Every finding traces to a deterministic parser and rules engine evaluating your site's own declared files against a registry entry with a cited official source — see methodology. CrawlPact also does not modify your website, and it does not guarantee that any crawler obeys a declared instruction — see limitations for what a result can and cannot prove.
CrawlPact is also not a web application firewall, a live crawler blocker, a server-log analytics service, or a general-purpose SEO crawler — it reads and reports on declared public policy files, it does not sit in front of your traffic or process your server logs. See the full list on limitations, and how CrawlPact relates to tools that do those jobs in Compare.
Principles
- Vendor neutral. The crawler registry is built from each operator's own published documentation, not any single operator's product interests.
- Evidence based. Every finding cites the specific matched rule, line number, and source behind it — not a summary judgement without a trail back to the evidence.
- Honest about uncertainty. A crawler CrawlPact cannot verify against an official source is marked unverified, not guessed. An unimplemented or disabled feature says so, rather than fabricating a result.
Who operates CrawlPact
The CrawlPact service, this website and its published content are operated by CrawlPact, which is responsible for the crawler registry, the guides, Insights and comparisons, and every report the product generates. No crawler operator or compared vendor pays for placement — see the comparison methodology.
- Support: support@crawlpact.com
- Privacy and security reports: info@crawlpact.com
- Corrections to content or the crawler registry: support@crawlpact.com — see how corrections are handled
- Payments are processed by Paddle as merchant of record; CrawlPact never sees card details.
Questions
See security, privacy, status, and contact for operational, privacy, security-reporting, and content-correction details, or run a free audit to see the methodology applied to a real domain.