CrawlPactFree tools
Free tools
Free validators for the public signals CrawlPact's full audit evaluates together. Each one runs a full scan of the domain you enter and takes you straight to the relevant section of the report.
A tool, or the full audit?
Use an individual tool when you already know which signal you care about — for example, confirming a single crawler is blocked, or checking whether an llms.txt file validates. Use the full audit when you want every signal evaluated together, a Policy Health Score, and conflict detection across signals (for example, robots.txt allowing a crawler that RSL prohibits).
AI crawler checker
See which documented AI crawlers your robots.txt allows or blocks, by purpose, with the rule and line behind each.
Robots.txt AI validator
Validate robots.txt groups, rules, sitemap lines and syntax problems by line, following RFC 9309.
RSL validator
Find the RSL licence a site declares and validate it against RSL 1.0, with its usage and payment terms.
llms.txt validator
Validate llms.txt and llms-full.txt structure and linked resources.
Content Signals checker
Read robots.txt Content-Signal lines (search, ai-input, ai-train) per User-agent group, with conflicts.
How these work
Enter a domain or full URL. CrawlPact retrieves the relevant public resource directly — no account, no installation, and no server-log access required. Only the specific, bounded resources relevant to AI crawler policy are retrieved and stored; see scanner informationfor exactly what is and isn't fetched, and privacy for retention.
Every tool here uses the same parser and rules as the full audit — there is no separate, simplified logic that could disagree with it. See methodology for how each signal is evaluated, and limitations for what a result does and doesn't prove.