How to recognize it
Every request says who it is in the user agent:
Mozilla/5.0 (compatible; CiteloopBot/1.0; +https://citeloop.ai/bot/)Requests come from these addresses:
62.146.238.2012407:3640:2349:7133::1
When it visits
Only when someone enters your website at app.citeloop.ai to run a free audit, or when we test our own websites. The same website is audited at most once an hour.
CiteloopBot does not crawl the web on its own, does not build a search index, and what it fetches is not used to train AI models. We keep it for 90 days so the person who ran the audit can see the report, then delete it.
What it fetches
- robots.txt first, then the homepage, your sitemap, llms.txt and a sample of up to 20 pages, plus one request to see whether http:// redirects to https://.
- At most 30 requests per audit, one per second, at most 2 MB per response, and it follows at most 5 redirects.
- Only public pages. It does not log in, submit forms or click anything, and it never connects to private network addresses.
To check whether your content depends on JavaScript, it opens up to 3 of those pages in a headless browser. Like any browser, it then loads the scripts, styles and images those pages use. These requests carry the same user agent.
Comparison requests
To see whether your website treats AI crawlers differently, CiteloopBot requests your homepage once more for each of a few crawlers: OAI-SearchBot, ClaudeBot, PerplexityBot and Googlebot. It keeps its own name first and adds that crawler's published user agent after it, for example:
Mozilla/5.0 (compatible; CiteloopBot/1.0; +https://citeloop.ai/bot/) compare-as (Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot))It skips a crawler when your robots.txt does not let that crawler fetch the homepage. These requests do not come from those companies, so the report treats any difference only as a hint.
How to block it
CiteloopBot follows the rules in robots.txt for User-agent: CiteloopBot, or the rules for * if there is no group for it. To block it everywhere:
User-agent: CiteloopBot
Disallow: /It reads robots.txt again at the start of every audit, so a change applies to the next one. If you block it, an audit of your website will only say that it could not get in.
Contact
If CiteloopBot causes a problem on your website, use the contact page linked from our About page. How we handle data is explained in our privacy policy.