Troubleshooting guide
My RSL or Content Signals declaration isn't being detected — troubleshooting
Common reasons CrawlPact reports no RSL licence or no Content Signals for a site that publishes them, and how to check each one.
Both declarations only count if they are served where clients look for them. This guide covers the checks CrawlPact has run since ruleset 2026.10.1 (15 October 2026). A report from an earlier scan says so and shows the result as it was originally evaluated — run a new audit to apply the current checks.
Why does CrawlPact say no RSL licence is declared?
CrawlPact follows RSL 1.0’s discovery rules. It looks for a License: line in your robots.txt, a
Link header with rel="license" and type="application/rsl+xml" on your homepage response, and
a matching <link> or inline <script> in your homepage HTML. Only if none exists does it try the
legacy /.well-known/rsl.xml location. Common causes:
- No association. The XML file exists but nothing points to it. Add a
License: https://yourdomain.com/license.xmlline torobots.txt(see how to publish an RSL declaration). - A relative URL. RSL 1.0 requires an absolute URI in
License:lines and HTML links;License: /license.xmlis ignored and reported as a warning. - The association is scoped to one crawler. A
Licenseline inside a named crawler’s group applies only to that crawler. CrawlPact lists it but evaluates the licence that applies to crawlers in general (theUser-agent: *group, or a line before any group). - The link is missing its type. A
<link rel="license">withouttype="application/rsl+xml"could be any licence (Creative Commons, for example) and is not treated as RSL.
Why is my RSL licence reported as not conformant?
The licence was found but does not meet RSL 1.0. The report lists each error with its line. The usual ones:
- The
<rsl>root has noxmlns="https://rslstandard.org/rsl", or uses a different (earlier draft) namespace. <permits>or<prohibits>has notypeattribute, or uses words outside the vocabulary.- A
<content>element has nourl, or no<license>. - The file is not well-formed XML, or includes a
DOCTYPE— CrawlPact does not process DTDs.
Why does CrawlPact say there are no Content Signals?
CrawlPact reads Content-Signal lines in your live robots.txt, the form Cloudflare documents.
Common causes:
- The line is outside a group. A
Content-Signalline before the firstUser-agentline applies to no crawler; move it into theUser-agent: *group (or the group it is meant for). - The line sits at the end of the file. A
Content-Signalline placed after the last group’s rules — even after a blank line or a comment — belongs to that last group under RFC 9309 grouping, so it may apply to one crawler rather than all of them. CrawlPact warns about this; put the line inside theUser-agent: *group instead. - The values are not
yesorno.ai-train=falseorsearch=maybeis reported as a warning and treated as unspecified. - Your CDN or platform serves a different
robots.txt. Fetchhttps://yourdomain.com/robots.txtand confirm the line is in the live file — a managed or generatedrobots.txtcan replace the one in your repository. - You only send a response header. Cloudflare documents Content Signals in
robots.txt, not as a header. CrawlPact still shows a homepageContent-Signalheader if you send one, but readers of Content Signals look inrobots.txt.
A robots.txt without any Content-Signal line is not an error: it simply means the site neither
grants nor restricts those uses through Content Signals.
Verify
Run CrawlPact’s RSL validator or Content Signals checker after making a change — both fetch your live site on every scan.
Related guides
- A crawler shows 'Resource unavailable' — troubleshootingWhat it means when CrawlPact can't determine a crawler's access result because robots.txt itself couldn't be fetched, and how to fix it.
- My llms.txt file isn't validating — troubleshootingCommon structural issues CrawlPact's llms.txt validator flags, and how to fix each one.
- My Policy Health Score dropped between scans — troubleshootingWhat causes a saved domain's score to change on a scheduled re-scan, and how to find exactly what changed.
See how this applies to your own site
Run a free audit to check your declared AI crawler policy against your own domain.