agent-tracking

Guides

How much AI bot traffic does a typical website get?

Nobody can tell you a reliable number for a typical site, because it depends on what the site is: a documentation site is fetched by crawlers every night, a shop sees live assistants answering product questions, a blog sees referrals after it is cited. What you can do is measure your own in a day: crawler fetches from the server log, verified against vendor address ranges, plus assistant referrals and tool calls from a script tag.

By · Updated 2026-09-08

Diesen Guide auf Deutsch lesen

Why the published averages do not help

Industry reports count requests at the edge across millions of sites, which says something about the internet and nothing about you. A crawler that reads 400 pages of a documentation site every night is normal there and alarming on a five-page brochure site. The useful number is yours, split by agent and by page, with a trend.

The three components

Origin-log fetches from named crawlers or user-triggered fetchers, with explicit verification status where supported. Identifiable assistant referrals are human landings and do not prove a citation. Instrumented tool calls form a third event source. Google-Extended is a robots policy token, not a crawler identity.

Measure it in a day

Add the site, paste the snippet, upload yesterday's access log. The dashboard shows fetches per agent with verification, bursts, referrals and pages within minutes, and a trend after a week. One public example is this site's own live stats page, updated daily, and it is small because the domain is new.

curl -sS -X POST https://agenttracking.co/api/logs/example.com \
  -H "Authorization: Bearer wmt_your_token" \
  -H "Content-Type: text/plain" --data-binary @/var/log/nginx/access.log

Reading the result

Fetch volume alone does not reveal training, indexing or use in an answer. A burst groups nearby requests but does not reveal a prompt or citation. Referrals show identifiable human landings; server receipts show confirmed business outcomes. An unverified bot claim needs investigation and is not automatically an impostor.

In short

Can you share benchmarks from your customers?

Not yet, and not without their consent. The pilot is days old. When there are enough sites and owners agree, aggregate ranges by site type will be published here.

Do page views by people count against my quota?

No. Only agent events count. Plain page views appear as a total for context.