How this works

Finding agents

Every publicly trusted TLS certificate is appended to public Certificate Transparency logs. agenttru.st reads those logs continuously, plus a historical backfill, and collects the hostnames appearing in newly issued certificates.

Each host is then checked once for an agent: an _ans DNS record, a DNS-AID SVCB record, or an agent card at /.well-known/agent-card.json. Hosts that are not agents are recorded as such and not asked again for a month, stretching towards three for hosts that keep giving the same answer — most of the internet is not an agent, and re-asking is the largest avoidable cost in a system like this.

What verification means

Listings separate what we checked from what the agent says about itself, and never merge the two. An agent card is written by whoever runs the agent; it is a claim.

bronze

An agent card was fetched over HTTPS and the certificate validated against the public trust store for that exact hostname.

silver

Bronze, plus the served certificate matches a TLSA record published in DNS and protected by DNSSEC — so the identity is pinned by the domain owner, not only by a certificate authority.

Everything a collector reports is re-verified centrally before it appears here, so a single misbehaving collector cannot add an entry to this directory.

Certificates

Every listing shows the certificate the agent was served under, because that certificate is the whole of what bronze asserts. Two things about it are worth reading carefully.

Almost every certificate on the web is domain-validated. The authority confirmed that someone controls the hostname, and checked nothing about who that someone is. It is not a weaker kind of encryption — the cryptography is identical to the most expensive certificate sold. The difference is identity, and a domain-validated certificate establishes none. Where an authority did vet an organisation, the listing names it; on this corpus that is rare.

A wildcard certificate hides its own siblings. A certificate for *.example.com is logged as exactly that, so Certificate Transparency records the wildcard and never the hosts behind it. Discovery here rides those logs, which means an agent living under a wildcard cannot be found this way at all. When a listing shows a wildcard, it is also telling you there may be neighbours this directory will never see.

Expiry is shown because a lapsed certificate is a listing about to stop working. Nothing else is inferred from it: a short-lived certificate is normal practice, not a warning sign.

Country

Country is hosting location: the geolocation of the IP address that actually served the agent card, falling back to a country-code domain suffix. It is not a statement about the nationality, residence, or legal jurisdiction of whoever operates the agent. An agent run by a German company on a US cloud region is listed as US.

Voting

Voting requires a GitHub sign-in. That is not for tracking — the only thing stored is a GitHub account id and username — it is because anonymous voting on a public ranking is trivially automated, which would make the rankings meaningless.

robots.txt

agenttru.st reads robots.txt before every request it makes, and honours it. That covers all of it: the discovery documents, the card an _ans record points at, and the site icon.

To keep this crawler out entirely:

User-agent: agenttrustbot
Disallow: /

A rule naming agenttrustbot takes precedence over your wildcard rule, in both directions. So you can also block everything else and still be discoverable:

User-agent: *
Disallow: /

User-agent: agenttrustbot
Disallow: /
Allow: /.well-known/

The Disallow: / has to be repeated in our group. A group that names us replaces the wildcard group rather than adding to it, so a group containing only an Allow line disallows us nothing at all.

Two things worth knowing before you add a block. A host that declines is not asked again for six months, not merely skipped once. And if the agent is already listed, blocking us removes the listing: the next re-check finds nothing it is permitted to read, and the entry is taken down. That is deliberate — robots.txt should work as a way out of this directory, not only as a way to reduce traffic — but it means a general anti-bot rule will delist you along with everything else it blocks.

The rules are read from the host being asked, not from whoever pointed at it. An _ans record can name a card served on a different host, and that host's own robots.txt is what governs fetching it; otherwise publishing a record would be a way to have someone else crawled.

The crawler page lists every path this fleet requests, the addresses it requests them from, and how fast.

Being listed

agenttru.st lists services that are already published on the public internet and announced in public certificate logs, and it reads only what those services voluntarily serve at a well-known path. Nothing here involves authentication, scanning for weaknesses, or accessing anything not offered publicly.

Even so, if you operate an agent and would prefer it not be listed, request removal and it will be taken down and not re-added. That stops the probing as well as the listing. A robots.txt rule does the same thing without waiting for anyone to read a form.