Meta-ExternalAgent in robots.txt
Meta-ExternalAgent is run by Meta. Purpose: training. Source: Meta documentation.
What blocking Meta-ExternalAgent changes
Blocking it asks the operator not to use your pages for AI model training. Search and live-fetch agents, where the operator runs them, use separate tokens with their own rules.
Block Meta-ExternalAgent
User-agent: Meta-ExternalAgent Disallow: /
Block Meta-ExternalAgent from one folder
User-agent: Meta-ExternalAgent Disallow: /private/
Allow Meta-ExternalAgent
If no group names Meta-ExternalAgent, it follows your "User-agent: *" rules. To allow it while other bots are blocked:
User-agent: Meta-ExternalAgent Allow: /
More: the guide AI crawlers in robots.txt and the AI crawler checker.
Questions
What is Meta-ExternalAgent?
Meta-ExternalAgent is the user agent token for an agent run by Meta. Its purpose: training. See the Meta documentation linked on this page.
How do I block Meta-ExternalAgent?
Add a group to robots.txt with "User-agent: Meta-ExternalAgent" and "Disallow: /". The rule applies to agents that follow robots.txt.
How do I check if my site blocks Meta-ExternalAgent?
Run the free check on this site. It tests your robots.txt against 8 AI agents on your homepage. The $19 deep report tests 18 agents per path.
Get an email when AI crawlers, robots.txt rules, or llms.txt practice change. At most twice a month.
We send a confirmation link first. See the privacy policy.