What it checks
- Can AI crawlers get in?Whether robots.txt allows GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Google-Extended, Bingbot and Googlebot; whether your site sits behind a CDN such as Cloudflare, and how its AI crawler settings are configured.
- Is the content there without JavaScript?Many AI crawlers do not run JavaScript. We compare the raw HTML with what a browser renders to see whether your content only exists in scripts.
- Titles, descriptions, h1 and canonicalsPresent on every page, clearly written, with canonicals pointing to the right URL.
- Structured dataValid, and consistent with what the page shows. For example, FAQ markup should match the questions visible on the page.
- Sitemaps, IndexNow and webmaster toolsWhether your sitemap exists and its dates are accurate; whether you notify Bing through IndexNow; whether the site is verified in Bing Webmaster Tools and Google Search Console with the sitemap submitted.
- Language, hreflang and llms.txtWhether pages declare the right language, whether language versions point to each other with hreflang, and whether you have an llms.txt for AI.
- Broken linksWhether internal links on the sampled pages work.
How it works
- Fetch public pagesWe request your home page and sample 20 pages from your sitemap: at most 30 requests per site, one at a time, at least a second apart. Our requests say they come from Citeloop and we follow your robots.txt.
- Check each itemWe go through the list above. We also request a few pages using each crawler's name for comparison, but CDNs such as Cloudflare recognize real crawlers by signature and IP, so those results are only hints. Whether AI crawlers actually reach you is settled by your CDN settings and logs, Bing Webmaster Tools and Google Search Console, which needs a screenshot or read-only access from you.
- Write the reportYou first get a one-page summary with the three most serious issues. The full report lists every issue with the request, status code and page excerpt, and explains the fix for the technology your site runs on.
What we never do
- Log in, fetch pages behind a login, or change your site or CDN settings.
- Publish your report. Reports are private and not indexed by default.
- Promise rankings, or promise that AI will recommend you.
In the meantime
Until the free audit opens, read the FAQ or the pilot program.