Documentation

Understand every check.

Start with a tool, learn what it inspects and understand what the result means before changing a production website.

01

Getting started

  1. Choose a toolPick the public signal you want to inspect.
  2. Run the checkEnter a URL or paste the configuration.
  3. Review the evidenceRead the detected rule, header or source before acting.

02

Tools

AI Crawler CheckerCheck whether AI crawlers can access your website.CSP DebuggerAnalyze CSP errors and get practical fix suggestions.Robots.txt TesterValidate and inspect your robots.txt rules.Sitemap ExplorerView and analyze your website sitemap.HTTP HeadersInspect HTTP headers and security configuration.Structured DataFetch or paste JSON-LD and validate its structure.

03

Concepts

AI crawlers
How automated agents discover content and how robots.txt can affect access.
robots.txt
The rules websites publish for crawlers, including allows, disallows and sitemap references.
HTTP headers
Response metadata used by browsers, crawlers and security policies.
Sitemaps
XML documents that help search engines and other crawlers discover public URLs.
Browse guides