Home / Guides

How to get cited by Claude: what Anthropic says, in 6 checks

To get cited by Claude, let Claude-SearchBot and Claude-User reach your pages. Anthropic's crawler help page says disabling either one "may reduce your site's visibility" in Claude's search results. ClaudeBot, the third bot, collects content for training and is a separate choice.

No one can promise a citation from Claude. These checks remove the reasons it can't use your pages. Type your domain to see where you stand first:

What Anthropic says decides it

Anthropic runs three bots, each with its own robots.txt token:

Bot	What Anthropic says it does	If you block it
Claude-SearchBot	"Navigates the web to improve search result quality for users."	"Prevents our system from indexing your content for search optimization, which may reduce your site's visibility and accuracy in user search results."
Claude-User	Visits sites "when individuals ask questions to Claude."	"Prevents our system from retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search."
ClaudeBot	Collects web content that may feed model training, per Anthropic.	Signals "that the site's future materials should be excluded from our AI model training datasets."

When Claude searches the web, Anthropic's web search help page says it uses "content from the live web" and that "every response includes citations." Those citations are the links you want.

The 6 checks

Allow Claude-SearchBot and Claude-User in robots.txt.
Add the rules on every subdomain.
Don't put a CAPTCHA or bot challenge in front of them.
Use robots.txt, not IP blocks, to set your rules.
Decide on ClaudeBot as a separate choice.
Put your main text in the HTML and give crawlers a sitemap and structured data.

Step 1: allow Claude-SearchBot and Claude-User

Open yoursite.com/robots.txt. Look for groups that name Claude-SearchBot or Claude-User with Disallow: /, and for a blanket User-agent: * with Disallow: /. To allow both bots in plain terms:

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

In our scan of 500 software homepages on October 18, 2026, 0 of 467 scored sites blocked Claude-SearchBot. See the Claude-SearchBot and Claude-User pages for the details, and test your file with the AI crawler robots.txt checker.

Step 2: cover every subdomain

robots.txt works per host. Anthropic's opt-out instructions say to add the rules "for every subdomain that you wish to opt out from." The same logic runs the other way: if your docs, blog, or shop sit on their own subdomain, check the robots.txt file on each one.

Step 3: don't wall them off with a challenge page

Anthropic says its bots "respect anti-circumvention technologies (e.g., we will not attempt to bypass CAPTCHAs for the sites we crawl.)" A bot challenge from your CDN stops Claude-SearchBot and Claude-User at the door, whatever robots.txt says.

In our October 18 scan, 20 of 500 homepages showed our checker a bot challenge page in place of the site: 16 from Cloudflare, 2 from Vercel, 1 from Akamai, and 1 we could not name. Our checker is not an Anthropic bot, so this does not prove Claude was blocked. It shows how common these walls are. If you use a bot shield, check what it does to AI search crawlers.

Step 4: use robots.txt, not IP blocks

Anthropic says blocking its IP addresses may not give a lasting opt-out, because a block stops its bots from reading your robots.txt file. Its bots honor robots.txt, so that file is the place to set your rules, whether you allow or block. If crawl load worries you, Anthropic supports the Crawl-delay line:

User-agent: ClaudeBot
Crawl-delay: 1

Step 5: decide on ClaudeBot as a separate choice

ClaudeBot is the training bot. Anthropic describes it apart from the two search bots, so you can block training and keep search:

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

In our scan, 3 of 467 scored sites blocked ClaudeBot: figma.com, bardeen.ai, and june.so. All 3 left Claude-SearchBot allowed. The ClaudeBot page has the user agent details.

Step 6: make your pages easy to read

Anthropic's help pages do not list page format rules. The standard ways to help any crawler apply: serve your main text in the HTML, list your pages in a sitemap, write a plain meta description, and add JSON-LD. In our scan, 10 of 467 scored homepages had little or no text without JavaScript, 23 had no sitemap we could find, and 101 (22%) had no JSON-LD. Our AI SEO audit checklist walks through each one.

How to check if Claude cites you

Turn on web search in Claude and ask the questions your buyers ask. Anthropic's help page says to click the "+" button in the lower left of the chat and pick "Web search." It adds: "If you have the new Claude experience, there's no web search toggle. Claude searches the web when it helps." Look at the cited links in each answer.

Anthropic's help page names Bing as the source for image results. It does not name a provider for text results, so the steps above are the parts you control.

The other AI answer engines

How to show up in ChatGPT search: OAI-SearchBot and OpenAI's IP ranges.
How to rank in Perplexity: PerplexityBot, Perplexity-User, and firewall rules.
How to rank in Gemini: the Google Search index and Google-Extended.
AI crawlers in robots.txt: rules for every major AI bot in one file.

Check your site

Run the free check on your domain. It tests ClaudeBot, Claude-SearchBot, and six other AI user agents in robots.txt, text without JavaScript, JSON-LD, sitemap, meta description, and llms.txt, and tells you when a challenge page blocks it. The $19 deep report crawls 25 pages and tests 18 AI crawlers, Claude-User included.

FAQ

How do I get my website cited by Claude? Allow Claude-SearchBot and Claude-User in robots.txt and don't block them with a CAPTCHA or bot challenge. Anthropic says disabling either bot may reduce your site's visibility in Claude's search results.

Does blocking ClaudeBot remove me from Claude's answers? Anthropic says restricting ClaudeBot signals that your future content should be excluded from training. It describes search visibility under Claude-SearchBot and Claude-User, which are separate bots.

Does Claude follow robots.txt? Anthropic says its bots "respect 'do not crawl' signals by honoring industry standard directives in robots.txt."

Does Claude cite its sources? Anthropic says that when Claude searches the web, "every response includes citations."

Should I block Anthropic by IP address? Anthropic says IP blocks may not work as an opt-out, because they stop its bots from reading your robots.txt. Use robots.txt rules.

Checked 2026-10-23. Prices and policies change. Confirm on the source before you buy.

Sources

More guides

More from us: Trades With AI (AI tools for trades businesses) · SaaS Price Watch (SaaS pricing, tracked weekly) · WrapMockup (Free vehicle wrap mockups for trades) · SkillsPicker (Agent skills picked for your project)