CrawlPactApple · Training crawler
Applebot-Extended
Controls use of website content for training Apple Intelligence and other Apple generative AI models.
Questions about Applebot-Extended
What is Applebot-Extended?
A robots.txt control token, not a separate crawler. Apple says Applebot-Extended does not crawl webpages; it only determines how data crawled by Applebot may be used.
Does blocking Applebot-Extended remove my site from Apple search?
No. Apple says pages that disallow Applebot-Extended can still be included in search results in Spotlight, Siri and Safari, and that Applebot-Extended rules are not considered in Search ranking.
What does disallowing Applebot-Extended do?
Apple says it opts your content out of being used to train Apple's generative foundation models.
How can I verify Apple's crawler?
Apple says Applebot traffic is identified by reverse DNS in the *.applebot.apple.com domain, or by matching the IP address against https://search.developer.apple.com/applebot.json.
Applebot-Extended is documented by Apple as a control token for opting website content out of
use in training Apple’s generative AI models, separate from the base Applebot crawler used for
Siri and Spotlight indexing.
Site-owner controls
Disallowing Applebot-Extended opts this content out of training Apple Intelligence and other
Apple generative AI models specifically. As with Google’s pairing of Googlebot/Google-Extended,
it leaves standard Apple indexing for Siri and Spotlight Suggestions (Applebot) unaffected — the
two tokens are evaluated independently, and Applebot-Extended does not itself perform a separate
crawl; it layers a training-use restriction on top of Apple’s existing Applebot access. See
/limitations for what a robots.txt rule can and cannot guarantee.
Example robots.txt configuration
To disallow Applebot-Extended specifically, without affecting any other crawler:
User-agent: Applebot-Extended
Disallow: /If no dedicated User-agent: Applebot-Extended group exists in a domain'srobots.txt, this crawler falls back to whatever the wildcard User-agent: * group says (RFC 9309) — see robots.txt syntax basics for how group selection works. A robots.txt rule is a published instruction; it does not technically prevent access.
Official source
https://support.apple.com/en-us/119829
Verified against the source above as of 1 July 2026 — see how CrawlPact verifies crawler information.
Related guides
- Applebot vs. Applebot-Extended: Search/Siri vs. Apple IntelligenceApple separates its long-standing search crawler from a newer, generative-AI-specific opt-out token. A decision guide for telling them apart.
- Blocking AI training while staying visible in AI searchChoosing between CrawlPact's presets when the goal is opting out of model training without losing AI-search discoverability.
- How to block only AI training crawlers, without blocking AI searchA step-by-step implementation guide for writing robots.txt groups that restrict training-purpose crawlers while leaving search-purpose crawlers untouched.
Other training crawlers
See how this applies to your own site
Run a free audit to check your declared AI crawler policy against your own domain, or use the AI crawler checker to see how your robots.txt treats Applebot-Extended and every other documented crawler.