How much AI bot traffic does a typical website get?

Why the published averages do not help

Industry reports count requests at the edge across millions of sites, which says something about the internet and nothing about you. A crawler that reads 400 pages of a documentation site every night is normal there and alarming on a five-page brochure site. The useful number is yours, split by agent and by page, with a trend.

The three components

Crawler fetches: GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended and their kind, visible only in the server log because they run no JavaScript. Live fetches: ChatGPT-User, Perplexity-User and other assistants answering a question about you right now, usually three to ten pages in a few seconds. Referrals: people arriving from an assistant that cited you. Most sites have the first two long before the third.

Measure it in a day

Add the site, paste the snippet, upload yesterday's access log. The dashboard shows fetches per agent with verification, bursts, referrals and pages within minutes, and a trend after a week. One public example is this site's own live stats page, updated daily, and it is small because the domain is new.

curl -sS -X POST https://agenttracking.co/api/logs/example.com \
  -H "Authorization: Bearer wmt_your_token" \
  -H "Content-Type: text/plain" --data-binary @/var/log/nginx/access.log

Reading the result

A high, steady fetch count from one crawler is training or indexing; whether you want that is a robots.txt decision. Bursts of a few pages in seconds are questions being answered about you; those pages are what assistants consider relevant. Referrals are the part that turns into business, and they usually lag the other two by weeks. Unverified lines claiming a known bot are impostors and worth a look.

In short