Training
How a model is developed. Allowing a training crawler does not guarantee that your business will appear in an answer.
THE AI PLATFORMS FIELD GUIDE
Different AI.
Different ways of finding you.
ChatGPT, Claude, Gemini, Perplexity. Familiar names, different routes to information. Here’s what that means for your website.
Reviewed 28 September 2026 · For business owners, not just developers
01 / GET YOUR BEARINGS
A model is not
the whole product.
A large language model, or LLM, generates responses from patterns learned during training and the context it receives. An assistant can add tools: web search, page retrieval, uploaded files or connected apps. The tools available change what it can use.
Web search
Finding sources for a question. Search access and training permission are separate questions for several providers.
Page retrieval
Opening a specific page, sometimes at a user’s request. A visit is evidence of access, not an endorsement.
Taking action
An agent may navigate or use a form. Clear labels and usable journeys matter alongside readable information.
02 / THE DIRECTORY
Know who’s
doing the looking.
These are product and access distinctions, not a ranking of which AI is “best”. Open each card for the crawler details, then check the provider’s documentation.
01OpenAI
ChatGPT
An assistant with a search option.
ChatGPT can answer using its model or use web search. When it searches, it can link to source pages. That does not mean every answer starts with a fresh visit to your website.
How website access works
OAI-SearchBot supports search discovery. GPTBot is used for potential model training. ChatGPT-User handles certain visits initiated by users. Search and training settings are independent. OpenAI notes that robots.txt may not apply to user-initiated visits.
Your next check
OpenAI crawler documentation Check search access separately from your training policy. Make the service, audience, location and supporting evidence easy to find on the page.
02Anthropic
Claude
Research depends on the tools in use.
Claude can use web search and retrieve pages when those capabilities are available. A response without web research is a different situation from an answer grounded in linked sources.
How website access works
Anthropic distinguishes ClaudeBot for model development, Claude-SearchBot for search, and Claude-User for user-directed retrieval. Its published guidance describes robots.txt controls for these bots.
Your next check
Anthropic crawler documentation Review which type of access you want to allow. Keep useful explanations on accessible pages, and check the cited source rather than assuming Claude read the whole site.
03Perplexity
Perplexity
Answers built around finding sources.
Perplexity is an answer and research product that uses web sources. A linked citation is useful evidence that a page supported that particular answer. It is not proof that the whole business is being recommended.
How website access works
Perplexity documents PerplexityBot for search discovery and Perplexity-User for user-requested retrieval. It says PerplexityBot is not used to crawl for foundation-model training; user-requested retrieval generally ignores robots.txt.
Your next check
Perplexity crawler documentation Publish specific answers with clear evidence, dates where relevant and identifiable authorship. Compare the question asked with the exact page cited.
04AI Overviews & AI Mode
Google AI Search
Search visibility still matters.
AI Overviews and AI Mode are experiences within Google Search. They may explore related searches to assemble an answer. They are not the same product as the standalone Gemini app.
How website access works
Google says a page must be indexed and eligible for a search snippet to qualify as a supporting link. Existing SEO practices apply; no special AI schema or new AI text file is required.
Your next check
Google Search guidance Start with indexing, internal links and useful visible content. Check Googlebot access and preview controls before buying an AI-only optimisation fix.
05Google
Gemini
One model family. Different experiences.
The Gemini name covers models and assistant experiences. The standalone app, a Search feature and an API application can use different tools and sources. Record which experience you tested.
How website access works
Google-Extended is a robots.txt control for specified Gemini training and grounding uses. It is not a separate crawler user-agent, and it does not control inclusion or ranking in Google Search.
Your next check
Google-Extended documentation Treat Search access and Google-Extended as separate decisions. In a test, note whether the answer includes sources and verify what those sources actually say.
06Microsoft & Bing
Microsoft Copilot
Check the product and the source.
Copilot spans several Microsoft products. Consumer web answers and a workplace assistant using organisational files are different contexts. Do not combine their results into a single visibility score.
How website access works
For public-web discovery, Bing Webmaster Tools provides a place to inspect your site’s search presence. Microsoft also documents publisher controls for Bing and generative AI.
Your next check
Bing Webmaster guidance Check Bing indexing and the current publisher guidance. For a Copilot test, save the question, answer, source links and date so the result can be checked again.
A SMALL CAVEAT. A BIG DIFFERENCE.
Being crawled isn’t
being chosen.
Access, inclusion, citation, recommendation and an enquiry are different events. One does not prove the next.
What you can observe
A verified crawler request. A source link in an answer. A referral visit. A completed enquiry. Keep the evidence and the date.
What you can’t promise
A permanent position in AI answers, a secret ranking formula or a guaranteed recommendation. Answers vary with the question, tools, location and product changes.
Independent guide by AI Visibility Studio. We are not affiliated with these platforms. Names belong to their owners. Provider documentation describes intended behaviour; your server logs and repeatable tests help establish what happened on your site.
03 / MAKE IT USEFUL
Test a question.
Keep the evidence.
Use a small, repeatable set of questions your customers actually ask. A single flattering answer is a starting point, not a trend.
AN EXAMPLE, NOT A LIVE RESULT
“Who can rebuild my business website without locking me into their platform?”
Ask the same question in selected products. Record the product, date, search mode, answer and cited URLs. Then check whether each source supports what the answer says.
- Was your business mentioned?Record the description, even if there is no link.
- Was your website cited?Save the exact URL and the claim it supports.
- Was the information correct?Check services, audience, location and ownership claims.
- Did anyone take the next step?Measure referrals and enquiries separately from mentions.
04 / START ON YOUR OWN WEBSITE
Less guesswork.
Better foundations.
Make the offer understandable. Say what you do, who you help and what the next step involves.
Make the evidence inspectable. Use specific examples, accurate details and sources that support your claims.
Check actual access. Review robots.txt, indexing and firewall responses. Permission in one place is not proof of access everywhere.
Keep the enquiry path usable. Labels, links and forms should work for the person arriving after an AI answer.
YOUR NEXT MOVE
Give your website
a closer look.
Start with the free Lighthouse and agent-readiness check on our audit page. It checks technical foundations. It does not measure whether an AI recommends your business.