Applebot-Extended in robots.txt
Applebot-Extended is run by Apple. Purpose: AI training control. Source: Apple documentation.
What blocking Applebot-Extended changes
This is a control token, not a separate crawler. Blocking it tells the operator not to use your pages for its AI models. Regular search crawling is not affected.
Block Applebot-Extended
User-agent: Applebot-Extended Disallow: /
Block Applebot-Extended from one folder
User-agent: Applebot-Extended Disallow: /private/
Allow Applebot-Extended
If no group names Applebot-Extended, it follows your "User-agent: *" rules. To allow it while other bots are blocked:
User-agent: Applebot-Extended Allow: /
More: the guide AI crawlers in robots.txt and the AI crawler checker.
Questions
What is Applebot-Extended?
Applebot-Extended is the user agent token for an agent run by Apple. Its purpose: AI training control. See the Apple documentation linked on this page.
How do I block Applebot-Extended?
Add a group to robots.txt with "User-agent: Applebot-Extended" and "Disallow: /". The rule applies to agents that follow robots.txt.
How do I check if my site blocks Applebot-Extended?
Run the free check on this site. It tests your robots.txt against 8 AI agents on your homepage. The $19 deep report tests 18 agents per path.
Get an email when AI crawlers, robots.txt rules, or llms.txt practice change. At most twice a month.
We send a confirmation link first. See the privacy policy.