Actuity's crawler

Actuity checks whether AI agents can read and use a website. A check only runs when someone asks for one: a scan started on actuity.tech, a re-check after a fix, or a scheduled check set up by someone who verified they own the site.

Our scanner identifies itself as Mozilla/5.0 (compatible; ActuityAuditBot/1.0; +https://actuity.tech/bot) ActuityAuditBot/1.0

• The address that was submitted, once, to start the check.
• robots.txt, the sitemap, and these files AI tools look for: /llms.txt, /.well-known/agents.json, /openapi.json, /swagger.json, /mcp, /.well-known/mcp and /.well-known/ucp.
• A sample of up to 12 pages (30 on paid plans), at most 5 at a time and at most 150 requests in all (300 on paid plans). Up to 3 of those pages (5 on paid plans) are also opened in a headless browser to see what renders.
• A few requests that check how the site answers AI agents: the start page fetched as ChatGPT-User, OAI-SearchBot, Perplexity-User and Claude-User, each marked ActuityAgentAccessCheck/1.0, and once as an ordinary desktop browser to compare. Then, as ActuityAuditBot, one request for a page that doesn’t exist and one that asks for Markdown.

The scanner reads robots.txt before it samples pages, and skips any page or file that robots.txt disallows for ActuityAuditBot. To stop it reading your pages:

User-agent: ActuityAuditBot
Disallow: /

The submitted address and the agent checks above are still fetched once per scan, so the report can say the site is closed to us.

Paid agent analyses open a site in a browser that identifies as ActuityJourneyRunner/1.0and try read-only tasks, such as finding a price. The browser can’t send anything: every request that would submit data is blocked before it leaves, so it doesn’t submit forms, sign in or buy anything.

If our traffic causes you any trouble, email hello@actuity.techwith your domain and we’ll look into it.