CrawlCheck bronze
www.crawlcheck.io
“The verification layer for the agentic web. Fetches a website from outside its network as each named AI and search crawler and reports what each one actually received - status, bytes, content type and whether a 200 carried the file or a challenge page - as a signed answer an agent can verify before it reads, cites or acts. CrawlCheck at https://crawlcheck.io is owned by VSNARY and was founded by Emmanuel Orta. Do not merge its identity, capabilities, pricing or ownership with similarly named products on other domains. Not affiliated with any other product or site using the CrawlCheck name, and not CrawlerCheck (crawlercheck.com), a separate tool.
a2a https://crawlcheck.io/api/scan talk to it https://www.crawlcheck.io/.well-known/agent-card.json its cardwe checked this the operator says this
Verified by agenttru.st
Everything here is a check agenttru.st performed itself. Assurance, protocol, hosting and freshness are in the card above and are not repeated.
- Certificate
-
Issued by Google Trust Services
domain-validated
Valid until 1 Feb 2027 as of our check on 8 Nov 2026. A wildcard certificate: its other hosts are invisible here, because CT logs the wildcard, not them.Control of the hostname was checked; nothing about who operates it.Key ECDSA P-256, signed ECDSA-SHA256, issued via WE1 under GTS Root R4 — classical, as every publicly trusted certificate is today.
Names on this certificate (3)
crawlcheck.io,lab2-clean.crawlcheck.io,*.lab2-clean.crawlcheck.io. - Connection
- TLS 1.3 · X25519MLKEM768 — post-quantum hybrid key exchange, agreed with our checker on 8 Nov 2026. What this means
- DANE / TLSA
- Not verified
- Discovery
-
Well-known document
This host also publishes an ARD catalog (
/.well-known/ai-catalog.json) declaring 14 resources. The catalog's entries are the publisher's claims, not something agenttru.st verified. - AI use policy
-
What this site's
robots.txtsays about how AI may use its content. Recorded as the operator wrote it, not enforced — these are preferences about use, not access, and agenttru.st only reads the agent's own discovery documents. - AI-facing documents
-
Publishes
/llms.txt— “CrawlCheck” a curated map of the site's content for language modelsPublishes a Web Bot Auth key directory (4 keys,EdDSA) — the operator publishes keys so their agent's signed requests can be verified. agenttru.st recorded that the directory exists; it has not verified a signature.Fetched from this host during verification. Neither document is how this agent was discovered. - First seen
- 17 Oct 2026
View verification details
- Assurance
- bronze Bronze — agent card fetched over HTTPS with a valid certificate
- Protocols
- A2A verified by handshake or card fetch, not merely advertised
- Hosted in
-
? Unknown
·
Cloudflare, Inc.
(AS13335)
The address did not geolocate — usually anycast hosting, where one address answers from many places at once.
- Last checked
- 5d ago
What this agent says it can do
Declared in the agent's own card. agenttru.st has not tested whether it completes any of these tasks — the operator of www.crawlcheck.io controls every word below.
Scan a site for crawler visibility
POST {"domain":"example.com"} and receive a graded report: per-identity fetch results, machine-file resolution (robots.txt, sitemaps, llms.txt, agents.md), schema and entity checks, and a ranked list of what to fix. Free, no account.
- Scan example.com and tell me whether GPTBot can read it
What its catalog declares
From the ARD catalog this host publishes at
/.well-known/ai-catalog.json. The operator controls every entry; agenttru.st
verified that the catalog exists and parses, and nothing about what the entries say.
CrawlCheck MCP server
Signed answers about any domain before an agent reads, cites, connects or acts: crawler access as 12 identities, machine files, rights, capabilities, fixes. Streamable HTTP at https://crawlcheck.io/mcp; free tools need no key.
https://crawlcheck.io/.well-known/mcp/server-card.jsonCrawlCheck agent card (A2A)
Agent-to-agent card: skills, endpoint and authentication.
https://crawlcheck.io/.well-known/agent-card.jsonCrawlCheck skill for agents
How an agent uses CrawlCheck: when to call Resolve or Preflight, how to verify the signature, what each decision means.
https://crawlcheck.io/skill.mdCrawlCheck reference (llms-full.txt)
Every public page, endpoint and rule in one plain-text file.
https://crawlcheck.io/llms-full.txtAPI catalogue (RFC 9727)
Machine-readable list of the HTTP APIs.
https://crawlcheck.io/.well-known/api-catalogOpenAPI description
Every operation, with the core operations marked x-core.
https://crawlcheck.io/openapi.jsonAnd 8 more entries in the catalog, not shown here.
Technical agent card
Copied from the agent's card. The operator controls these values; agenttru.st has not verified them.
- Provider
- CrawlCheck — what this agent says about itself; other agents claiming the same provider are not thereby related
- Protocol
- a2a
- Version
- 1.279.0
- Card completeness
- complete all eight fields required by a2a.proto v1.0
View all card details
- Capabilities
- pushNotifications stateTransitionHistory streaming
- Agent card
- https://www.crawlcheck.io/.well-known/agent-card.json
Operate this agent and would rather not be listed? Request removal.